publications
preprints/submitted works
- Recreating Neural Activity During Speech Production with Language and Speech Model EmbeddingsOwais Mujtaba Khanday, Pablo Rodríguez-San Esteban, Zubair Ahmad Lone, and 2 more authorsarXiv May 2025
Understanding how neural activity encodes speech and language production is a fundamental challenge in neuroscience and artificial intelligence. This study investigates whether embeddings from large-scale, self-supervised language and speech models can effectively reconstruct high-gamma neural activity characteristics, key indicators of cortical processing, recorded during speech production. We leverage pre-trained embeddings from deep learning models trained on linguistic and acoustic data to represent high-level speech features and map them onto these high-gamma signals. We analyze the extent to which these embeddings preserve the spatio-temporal dynamics of brain activity. Reconstructed neural signals are evaluated against high-gamma ground-truth activity using correlation metrics and signal reconstruction quality assessments. The results indicate that high-gamma activity can be effectively reconstructed using large language and speech model embeddings in all study participants, generating Pearson’s correlation coefficients ranging from 0.79 to 0.99.
- UGR-MINDVOICE: A Multimodal EEG-Audio Dataset for Overt and Covert Iberian Spanish Speech ProductionIbon Vales Cortina, Owais Mujtaba Khanday, Marc Ouellet, and 6 more authorsMay 2025
We present UGR-MINDVOICE, the University of Granada (UGR) multi-modal electroencephalography (EEG) and audio dataset for overt and covert speech in Iberian spanish intended for basic neuroscience and brain-computer interface (BCI) research. The dataset features EEG and audio recordings from 15 native Spanish speakers engaged in both overt and covert speech production tasks. This dataset is unique in its inclusion of all Spanish phonemes and a diverse set of words spanning various semantic categories and different usage frequencies. Validation of the dataset confirmed the presence of the P100 event-related potential component, a distinct positive peak occurring around 100 milliseconds after visual stimulus onset, indicating participant attention to the visual stimuli. Additionally, the EEG data were classified into rest, covert speech, and overt speech conditions with an accuracy of 75.26%, demonstrating active participant engagement in the tasks. By providing synchronized EEG and audio data for overt speech, along with EEG data for the same stimuli during covert speech, UGR-MINDVOICE constitutes a valuable resource for advancing research in basic neuroscience and brain-computer interfaces, particularly in the domain of silent speech communication.
peer-reviewed publications
2025
- Neural representation of consciously seen and unseen informationPablo Rodríguez-San Esteban, José A. González-López, and Ana B. ChicaScientific Reports Mar 2025
Machine learning (ML) techniques have steadily gained popularity in Neuroscience research, particularly when applied to the analysis of neuroimaging data. One of the most discussed topics in this field, the neural correlates of conscious (and unconscious) information, has also benefited from these approaches. Nevertheless, further research is still necessary to better understand the minimal neural mechanisms that are necessary and sufficient for experiencing any conscious percept, and which mechanisms are comparable and discernible between conscious and unconscious events. The aim of this study was two-fold. First, to explore whether it was possible to decode task-relevant features from electroencephalography (EEG) signals, particularly those related to perceptual awareness. Secondly, to test whether this decoding could be improved by using time-frequency representations instead of voltage. We employed a perceptual task in which participants were presented with near-threshold Gabor stimuli. They were asked to discriminate the orientation of the grating, and report whether they had perceived it or not. Participants’ EEG signal was recorded while performing the task and was then analysed by using ML algorithms to decode distinctive task-related parameters. Results demonstrated the feasibility of decoding the presence/absence of the stimuli from EEG data, as well as participants’ subjective perception, although the model failed to extract relevant information related to the orientation of the Gabor. Unconscious processing of unseen stimulation was observed both behaviourally and at the neural level. Moreover, contrary to conscious processing, unconscious representations were less stable across time, and only observed at early perceptual stages ( 100 ms) and during response preparation. Furthermore, we conducted a comparative analysis of the performance of the classifier when employing either raw voltage signals or time-frequency representations, finding a substantial improvement when the latter was used to train the model, particularly in the theta and alpha bands. These findings underscore the significant potential of ML algorithms in decoding perceptual awareness from EEG data in consciousness research tasks.
2024
- The role of white matter variability in TMS neuromodulatory effectsMar Martín-Signes, Pablo Rodríguez-San Esteban, Cristina Narganes-Pineda, and 4 more authorsBrain Stimulation Nov 2024
Background Transcranial Magnetic Stimulation (TMS) is a widely used tool to explore the causal role of focal brain regions in cognitive processing. TMS effects over attentional processes are consistent and replicable, while at the same time subjected to individual variability. This individual variability needs to be understood to better comprehend TMS effects, and most importantly, its clinical applications. Objective Hypothesis: This study aimed to explore the role of white matter variability in TMS neuromodulatory effects on behavior in healthy participants (N=50). Methods Participants completed an attentional task in which orienting and alerting cues preceded near-threshold targets. Continuous Theta Burst Stimulation (cTBS) was applied over the left frontal eye field (FEF) or an active vertex condition. White matter was explored with diffusion-weighted imaging tractography and Tract-Based Spatial Statistics (TBSS). Results Behaviourally, TMS over the left FEF slowed down reaction times (especially in the alerting task), impaired accuracy in the objective task, and reduced the proportion of seen targets (as compared to the vertex condition). Attentional effects increased, overall, when TMS was applied to the left FEF as compared to the vertex condition. Correlations between white matter and TMS effects showed i) reduced TMS effects associated with the microstructural properties of long-range white matter pathways such as the superior longitudinal fasciculus (SLF), and interhemispheric fibers of the corpus callosum (CC), and ii) increased TMS effects in participants with high integrity of the CC connecting the stimulated region with the opposite hemisphere. Additionally, variability in attentional effects was also related to white matter, showing iii) increased alerting effects in participants with low integrity of association, commissural, and projection fibers, and iv) increased orienting effects in participants with high integrity of the right SLF III. Conclusion All these observations highlight the importance of taking into account individual variability in white matter for the understanding of cognitive processing and brain neuromodulation effects.
- The role of brain oscillations in feature integrationMaría I. Cobos, María Melcón, Pablo Rodríguez-San Esteban, and 2 more authorsPsychophysiology Mar 2024
Our sensory system is able to build a unified perception of the world, which although rich, is limited and inaccurate. Sometimes, features from different objects are erroneously combined. At the neural level, the role of the parietal cortex in feature integration is well-known. However, the brain dynamics underlying correct and incorrect feature integration are less clear. To explore the temporal dynamics of feature integration, we studied the modulation of different frequency bands in trials in which feature integration was correct or incorrect. Participants responded to the color of a shape target, surrounded by distractors. A calibration procedure ensured that accuracy was around 70% in each participant. To explore the role of expectancy in feature integration, we introduced an unexpected feature to the target in the last blocks of trials. Results demonstrated the contribution of several frequency bands to feature integration. Alpha and beta power was reduced for hits compared to illusions. Moreover, gamma power was overall larger during the experiment for participants who were aware of the unexpected target presented during the last blocks of trials (as compared to unaware participants). These results demonstrate that feature integration is a complex process that can go wrong at different stages of information processing and is influenced by top-down expectancies.
2023
- Functional characterization of correct and incorrect feature integrationPablo Rodríguez-San Esteban, Ana B. Chica, and Pedro M. Paz-AlonsoCerebral Cortex Feb 2023
Our sensory system constantly receives information from the environment and our own body. Despite our impression to the contrary, we remain largely unaware of this information and often cannot report it correctly. Although perceptual processing does not require conscious effort on the part of the observer, it is often complex, giving rise to errors such as incorrect integration of features (illusory conjunctions). In the present study, we use functional magnetic resonance imaging to study the neural bases of feature integration in a dual task that produced ∼30% illusions. A distributed set of regions demonstrated increased activity for correct compared to incorrect (illusory) feature integration, with increased functional coupling between occipital and parietal regions. In contrast, incorrect feature integration (illusions) was associated with increased occipital (V1–V2) responses at early stages, reduced functional connectivity between right occipital regions and the frontal eye field at later stages, and an overall decrease in coactivation between occipital and parietal regions. These results underscore the role of parietal regions in feature integration and highlight the relevance of functional occipito-frontal interactions in perceptual processing.