Neural correlates of semantic number: A cross-linguistic investigation

One aspect of natural language comprehension is understanding how many of what or whom a speaker is referring to. While previous work has documented the neural correlates of general number comprehension and quantity comparison, we investigate semantic number from a cross-linguistic perspective with the goal of identifying cortical regions involved in distinguishing plural from singular nouns. We use three fMRI datasets in which Chinese, French, and English native speakers listen to an audiobook of a children's story in their native language. We select these three languages because they differ in their number semantics. While Chinese lacks nominal pluralization, French and English nouns are overtly marked for number. We find a number of known semantic processing regions in common, including dorsomedial prefrontal cortex and the pars orbitalis, in which cortical activation is greater for plural than singular nouns and posit a cross-linguistic role for number in semantic comprehension.

GPT-2’s activations predict the degree of semantic comprehension in the human brain

Language transformers, like GPT-2, have demonstrated remarkable abilities to process text, and now constitute the backbone of deep translation, summarization and dialogue algorithms. However, whether these models actually understand language is highly controversial. Here, we show that the representations of GPT-2 not only map onto the brain responses to spoken stories, but also predict the extent to which subjects understand the narratives. To this end, we analyze 101 subjects recorded with functional Magnetic Resonance Imaging while listening to 70 min of short stories. We then fit a linear model to predict brain activity from GPT-2 activations, and correlate this mapping with subjects’ comprehension scores as assessed for each story. The results show that GPT-2’s brain predictions significantly correlate with semantic comprehension. These effects are bilaterally distributed in the language network and peak with a correlation above 30% in the infero-frontal and medio-temporal gyri as well as in the superior frontal cortex, the planum temporale and the precuneus. Overall, this study provides an empirical framework to probe and dissect semantic comprehension in brains and deep learning algorithms.

Correlated Brain Indexes of Semantic Prediction and Prediction Error: Brain Localization and Category Specificity

With strong and valid predictions, grasping a message is easy, whereas more demanding processing is required in the absence of robust expectations. We here demonstrate that brain correlates of the interplay between prediction and perception mechanisms in the understanding of meaningful sentences. Sentence fragments that strongly predict subsequent words induced anticipatory brain activity preceding the expected words; this potential was absent if context did not strongly predict subsequent words. Subjective reports of certainty about upcoming words and objective corpus-based measures correlated with the size of the anticipatory signal, thus establishing its status as a semantic prediction potential (SPP). Crucially, there was an inverse correlation between the SPP and the N400 brain response. The main cortical generators of SPP and N400 were found in inferior prefrontal cortex and posterior temporal cortex, respectively. Interestingly, sentence meaning was reflected by both measures, with additional category-specific sources of SPPs and N400s falling into parieto-temporo-occipital (visual) and frontocentral (sensorimotor) areas for animal- and tool-related words, respectively. These results show that the well-known brain index of semantic comprehension, N400, has an antecedent with different brain localization but similar semantic discriminatory function. We discuss whether N400 dynamics may causally depend on mechanisms underlying SPP size and sources.

Semantic comprehension of idioms

This study intends to test the Graded Salience Hypothesis, in order to investigate the factors involved in comprehension. This research considered predictions derived from this hypothesis by evaluating the salience of idioms in the Persian language. We intended to measure Reading Time (RTs), and the design comprised 2 Contexts (figurative, literal), 3 Types of Statements (familiar vs. unfamiliar vs. less familiar) and RTs (long, short, equal). Two types of contexts (figuratively inviting and literally inviting contexts) were prepared. The software for this experiment was prepared for the purpose of self-paced reading experiments. Two pretests were performed. In the first pretest, participants rated the expressions on a 1–7 familiarity scale. The second pretest was designed to confirm that contexts are equally supportive. Then, expressions were divided according to their familiarity (familiar, less-familiar, unfamiliar). Sentences were used so that, according to the second pretest, their contexts would be equally supportive. Sentences were displayed on a PC, controlled by Windows 7. The self-paced reading task was applied using the Moving Windows software. In the first part of the experiment, participants read each idiom in figuratively inviting contexts and their RTs were recorded. In the second part of the experiment, participants read each idiom in literally inviting contexts and their RTs were recorded. Results of testing these idioms support the Graded Salience Hypothesis, but not entirely. Such findings suggested that sometimes context affects the access of salient information and a semi serial process is witnessed. Results indicate that the salient meaning of both familiar and less familiar idioms is figurative. In addition, salient meanings in the space following the unfamiliar idiom and the first word of the next (spillover) sentence, were both, figurative and literal.

Unconstrained multivariate EEG decoding can help detect lexical-semantic processing in individual children

In conditions such as minimally-verbal autism, standard assessments of language comprehension are often unreliable. Given the known heterogeneity within the autistic population, it is crucial to design tests of semantic comprehension that are sensitive in individuals. Recent efforts to develop neural signals of language comprehension have focused on the N400, a robust marker of lexical-semantic violation at the group level. However, homogeneity of response in individual neurotypical children has not been established. Here, we presented 20 neurotypical children with congruent and incongruent visual animations and spoken sentences while measuring their neural response using EEG. Despite robust group-level responses, we found high inter-individual variability in response to lexico-semantic anomalies. To overcome this, we analysed our data using temporally and spatially unconstrained MVPA, supplemented by descriptive analyses to examine the timecourse, topography, and strength of the effect. Our results show that neurotypical children exhibit heterogenous responses to lexical-semantic violation, implying that any application to heterogenous disorders such as ASD will require individual-subject analyses that are robust to variation in topology and timecourse of neural responses.