Processing Speech and Thoughts during Silent Reading: Direct Reference Effects for Speech by Fictional Characters in Voice-Selective Auditory Cortex and a Theory-of-Mind Network

Stories transport readers into vivid imaginative worlds, but understanding how readers create such worlds—populating them with characters, objects, and events—presents serious challenges across disciplines. Auditory imagery is thought to play a prominent role in this process, especially when representing characters' voices. Previous research has shown that direct reference to speech in stories (e.g., He said, “I'm over here”) may prompt spontaneous activation of voice-selective auditory cortex more than indirect speech [Yao, B., Belin, P., & Scheepers, C. Silent reading of direct versus indirect speech activates voice-selective areas in the auditory cortex. Journal of Cognitive Neuroscience, 23, 3146–3152, 2011]. However, it is unclear whether this effect reflects differential processing of speech or differences in linguistic content, source memory, or grammar. One way to test this is to compare direct reference effects for characters speaking and thinking in a story. Here, we present a multidisciplinary fMRI study of 21 readers' responses to characters' speech and thoughts during silent reading of short fictional stories. Activations relating to direct and indirect references were compared for both speaking and thinking. Eye-tracking and independent localizer tasks (auditory cortex and theory of mind [ToM]) established ROIs in which responses to stories could be tracked for individuals. Evidence of elevated auditory cortex responses to direct speech over indirect speech was observed, replicating previously reported effects; no reference effect was observed for thoughts. Moreover, a direct reference effect specific to speech was also evident in regions previously associated with inferring intentions from communication. Implications are discussed for the spontaneous representation of fictional characters and the potential roles of inner speech and ToM in this process.

Download Full-text

Silent Reading of Direct versus Indirect Speech Activates Voice-selective Areas in the Auditory Cortex

Journal of Cognitive Neuroscience ◽

10.1162/jocn_a_00022 ◽

2011 ◽

Vol 23 (10) ◽

pp. 3146-3152 ◽

Cited By ~ 62

Author(s):

Bo Yao ◽

Pascal Belin ◽

Christoph Scheepers

Keyword(s):

Auditory Cortex ◽

Interdisciplinary Research ◽

Silent Reading ◽

Mental Simulation ◽

Human Communication ◽

Direct Speech ◽

Indirect Speech ◽

Inner Voice ◽

Starting Point ◽

Human Volunteers

In human communication, direct speech (e.g., Mary said: “I'm hungry”) is perceived to be more vivid than indirect speech (e.g., Mary said [that] she was hungry). However, for silent reading, the representational consequences of this distinction are still unclear. Although many of us share the intuition of an “inner voice,” particularly during silent reading of direct speech statements in text, there has been little direct empirical confirmation of this experience so far. Combining fMRI with eye tracking in human volunteers, we show that silent reading of direct versus indirect speech engenders differential brain activation in voice-selective areas of the auditory cortex. This suggests that readers are indeed more likely to engage in perceptual simulations (or spontaneous imagery) of the reported speaker's voice when reading direct speech as opposed to meaning-equivalent indirect speech statements as part of a more vivid representation of the former. Our results may be interpreted in line with embodied cognition and form a starting point for more sophisticated interdisciplinary research on the nature of auditory mental simulation during reading.

Download Full-text

Theory of Mind impairments in early course schizophrenia: An fMRI study

Journal of Psychiatric Research ◽

10.1016/j.jpsychires.2021.02.010 ◽

2021 ◽

Vol 136 ◽

pp. 236-243

Author(s):

Rachal R. Hegde ◽

Synthia Guimond ◽

Deepthi Bannai ◽

Victor Zeng ◽

Shezal Padani ◽

...

Keyword(s):

Theory Of Mind ◽

Early Course ◽

Fmri Study

Download Full-text

Investigating neural mechanisms that interface between self and others: An fMRI study on the theory of mind and empathy

Neuroscience Research ◽

10.1016/j.neures.2010.07.357 ◽

2010 ◽

Vol 68 ◽

pp. e26-e27

Author(s):

Noriaki Yahata

Keyword(s):

Theory Of Mind ◽

Neural Mechanisms ◽

Fmri Study

Download Full-text

Short-term functional plasticity in the human auditory cortex: an fMRI study

Cognitive Brain Research ◽

10.1016/s0926-6410(01)00092-1 ◽

2001 ◽

Vol 12 (3) ◽

pp. 479-485 ◽

Cited By ~ 89

Author(s):

L Jäncke ◽

N Gaab ◽

T Wüstenberg ◽

H Scheich ◽

H.-J Heinze

Keyword(s):

Auditory Cortex ◽

Functional Plasticity ◽

Short Term ◽

Fmri Study ◽

Human Auditory Cortex

Download Full-text

Chapter 9 - Theory of Mind and Joint Attention - The Handbook on Socially Interactive Agents

10.31234/osf.io/az8qy ◽

2021 ◽

Author(s):

Jairo Pérez-Osorio ◽

Eva Wiese ◽

Agnieszka Wykowska

Keyword(s):

Theory Of Mind ◽

Joint Attention ◽

Cognitive Neuroscience ◽

Computational Models ◽

Social Cognitive ◽

Virtual Agents ◽

Artificial Agents ◽

Future Directions ◽

Social Cognitive Neuroscience ◽

Intelligent Virtual Agents

The present chapter provides an overview from the perspective of social cognitive neuroscience (SCN) regarding theory of mind (ToM) and joint attention (JA) as crucial mechanisms of social cognition and discusses how these mechanisms have been investigated in social interaction with artificial agents. In the final sections, the chapter reviews computational models of ToM and JA in social robots (SRs) and intelligent virtual agents (IVAs) and discusses the current challenges and future directions.

Download Full-text

Are cognitive representations of friends' request refusals implemented in the orbitofrontal and dorsolateral prefrontal cortices? A cognitive neuroscience approach to ‘theory of mind’ in relationships

Journal of Social and Personal Relationships ◽

10.1177/0265407506062476 ◽

2006 ◽

Vol 23 (2) ◽

pp. 249-265 ◽

Cited By ~ 11

Author(s):

Alan D. Heisel ◽

Michael J. Beatty

Keyword(s):

Theory Of Mind ◽

Brain Imaging ◽

Social Relationships ◽

Cognitive Neuroscience ◽

Process Model ◽

Critical Role ◽

Brain Wave ◽

Cognitive Representations ◽

Dorsolateral Prefrontal ◽

Prefrontal Cortices

Formulating cognitive representations of others' mental states when interpreting behavior (i.e., ‘theory of mind’) rather than merely focusing on the behavior is considered a distinctly human trait which both interpersonal scholars and cognitive neuroscientists agree plays a critical role in the development and maintenance of social relationships. Although brain-imaging studies have led to huge advances in the understanding of memory and language, theories of social relationships remain relatively uninformed by cognitive neuroscience. In the present study, hypotheses regarding the implementation of theory of mind in a relationship context are (a) derived from extant theory and research, and (b) tested via brain-imaging technology. Specifically, spectrum analyses were conducted using brain wave recordings collected by an electroencephalograph (EEG) monitoring oscillations in the gamma range for the orbitofrontal and dorsolateral prefrontal cortices while participants attempted to construct cognitive representations regarding a friend's request refusal. Results indicated statistically greater electrical activity in both cortical regions for participants engaged in the task than for participants in the control condition. The implications of the findings for building a fully elaborated sequential process model of cognitive representations in interpersonal contexts, among other theoretical endeavors, are discussed.

Download Full-text