100 Peers: An open set of controlled emotional facial expressions for use in psychology and neuroscience experiments

Human faces are a ubiquitous stimulus category in experimental psychology and neuroscience, routinely applied in many research domains including perception, emotion, social cognition, and memory. Here, we present a set of portrait photographs of 100 identities with each identity making 9 emotional expressions (anger, calm, contempt, disgust, fear, happiness, sadness, surprise, and neutral), amounting to 900 total photographs. Unlike many existing face databases, these photographs are highly controlled for perceptual features, in that (1) they were taken in a laboratory setting with identical professional lighting, (2) all participants are wearing the exact same neutral clothing (a black t-shirt), (3) all faces have been manually adjusted such that the eyes are approximately at the center of a square without cropping any part of the face from the photograph, and (4) the background has been colored to be exactly the same gray hue in all photographs with no shadows. Given their highly controlled nature, these faces are optimal for experiments aiming to homogenize non-face properties, such as lighting or clothing. We are making the face set open for the benefit of the broader research community, and request that in turn, data collected in response to the faces be made open and linked to the original face database (please cite this technical report). This report presents a high-level summary of the faces using behavioral data (free-response descriptions and emotion categorizations), but these data do not necessarily capture the scope of potential research designs that the stimuli could support. If authors collect relevant pilot data on these stimuli for specific research questions, our hope is that such data can becollaboratively linked to the face set for the community to utilize.

Naturalistic Audiovisual Stimulation Reveals the Topographic Organization of Human Auditory Cortex

Despite the importance of audition in spatial, semantic, and social function, there is no consensus regarding the detailed organisation of human auditory cortex. Using a novel computational model to analyse a high-powered naturalistic audiovisual movie-watching dataset, we simultaneously estimate spectral tuning properties and category selectivity to reveal the modes of organisation and computational motifs that characterise human auditory cortex. We find that regions more remote from the auditory core exhibit more compressive, non-linear response properties and finely-tuned, speech-selective receptive fields in low frequency portions of the tonotopic map. These patterns of organisation mirror aspects of the visual cortical hierarchy, wherein tuning properties progress from a stimulus category-agnostic front end towards more advanced regions increasingly optimised for behaviorally relevant stimulus categories.

Response Bias Reflects Individual Differences in Sensory Encoding

Humans exhibit substantial biases in their decision making even in simple two-choice tasks, but the origin of these biases remains unclear. I hypothesized that one source of bias could be individual differences in sensory encoding. Specifically, if one stimulus category gives rise to an internal-evidence distribution with higher variability, then responses should optimally be biased against that stimulus category. Therefore, response bias may reflect a previously unappreciated subject-to-subject difference in the variance of the internal-evidence distributions. I tested this possibility by analyzing data from three different two-choice tasks ( ns = 443, 443, and 498). For all three tasks, response bias moved in the direction of the optimal criterion determined by each subject’s idiosyncratic internal-evidence variability. These results demonstrate that seemingly random variations in response bias can be driven by individual differences in sensory encoding and are thus partly explained by normative strategies.

Contributions of representational distinctiveness and stability to memory performance and age differences

Long-standing theories of cognitive aging suggest that memory decline is associated with age-related differences in the way information is represented in the brain. In the last years, these hypotheses have been substantiated by novel neuroscientific evidence that was derived from multivariate pattern similarity analyses. This approach enabled researchers to take a representational perspective on brain and cognition, and allowed them to study the properties of neural representations that support successful episodic memory. In young adults, two prominent representational properties have been identified as crucial for memory performance, namely the distinctiveness and the stability of neural representations. Distinctiveness describes the relation of neural representations to each other, i.e., how similar or dissimilar they are, while stability characterizes how much or how little representations change over time. However, researchers have only recently started to explore age differences in these representational properties and how they interact to support episodic memory in old age. Here, we review studies that used multivariate analysis tools for different neuroimaging techniques to clarify how representational distinctiveness, stability, and their interactions relate to memory performance across adulthood, and specifically during aging. While most evidence on age differences in neural representations involved how stimulus category information is represented, recent studies demonstrated that particularly item-level stability and specificity of activity patterns are positively linked to memory success and decline during aging. Overall, multivariate methods offer a promising and versatile tool for our understanding of age differences in the neural representations underlying episodic memory.

Endogenous memory reactivation during sleep in humans is clocked by slow oscillation-spindle complexes

Sleep is thought to support memory consolidation via reactivation of prior experiences, with particular electrophysiological sleep signatures (slow oscillations (SOs) and sleep spindles) gating the information flow between relevant brain areas. However, empirical evidence for a role of endogenous memory reactivation (i.e., without experimentally delivered memory cues) for consolidation in humans is lacking. Here, we devised a paradigm in which participants acquired associative memories before taking a nap. Multivariate decoding was then used to capture endogenous memory reactivation during non-rapid eye movement (NREM) sleep in surface EEG recordings. Our results reveal reactivation of learning material during SO-spindle complexes, with the precision of SO-spindle coupling predicting reactivation strength. Critically, reactivation strength (i.e. classifier evidence in favor of the previously studied stimulus category) in turn predicts the level of consolidation across participants. These results elucidate the memory function of sleep in humans and emphasize the importance of SOs and spindles in clocking endogenous consolidation processes.

Emoji as Affective Symbols: Affective Judgments of Emoji, Emoticons, and Human Faces Varying in Emotional Content

An important function of emoji as communicative symbols is to convey emotional content from sender to receiver in computer-mediated communication, e. g., WhatsApp. However, compared with real faces, pictures or words, many emoji are ambiguous because they do not symbolize a discrete emotion or feeling state. Thus, their meaning relies on the context of the message in which they are embedded. Previous studies investigated affective judgments of pictures, faces, and words suggesting that these stimuli show a typical distribution along the big two emotion dimensions of valence and arousal. Also, emoji and emoticons have been investigated recently for their affective significance. The present study extends previous research by investigating affective ratings of emoji, emoticons and human faces and by direct comparison between them. In total, 60 stimuli have been rated by 83 participants (eight males, age: 18–49 years), using the non-verbal Self-Assessment Manikin Scales for valence and arousal. The emotionality of the stimuli was measured on a 9-point Likert scale. The results show significant main effects of the factors “stimulus category” and “discrete emotion” including emotionality, valence and arousal. Also, the interaction between these two main factors was significant. Emoji elicited highest arousal, whereas stimuli related to happiness were rated highest in valence across stimulus categories. Angry emoji were rated highest in emotionality. Also, the discrete emotion was best recognized in emoji, followed by human face stimuli and lastly emoticons.

The Level of Processing Modulates Visual Awareness: Evidence from Behavioral and Electrophysiological Measures

The level of processing hypothesis (LoP) proposes that the transition from unaware to aware visual perception is graded for low-level (i.e., energy, features) stimulus whereas dichotomous for high-level (i.e., letters, words, meaning) stimulus. In this study, we explore the behavioral patterns and neural correlates associated to different depths (i.e., low vs. high) of stimulus processing. The low-level stimulus condition consisted of identifying the color (i.e., blue/blueish vs. red/reddish) of the target, whereas the high-level stimulus condition consisted of identifying stimulus category (animal vs. object). Behavioral results showed that the levels of processing manipulation produced significant differences in both the awareness rating distributions and accuracy performances between tasks, the low-level task producing more intermediate subjective ratings and linearly increasing accuracy performances and the high-level task producing less intermediate ratings and a more nonlinear pattern for accuracies. The electrophysiological recordings revealed two correlates of visual awareness, an enhanced posterior negativity in the N200 time window (visual awareness negativity [VAN]), and an enhanced positivity in the P3 time window (late positivity [LP]). The analyses showed a double dissociation between awareness and the level of processing hypothesis manipulation: Awareness modulated VAN amplitudes only in the low-level color task, whereas LP amplitude modulations were observed only in the higher level category task. These findings are compatible with a two-stage microgenesis model of conscious perception, where an early elementary phenomenal sensation of the stimulus (i.e., the subjective perception of color) would be indexed by VAN, whereas stimulus' higher level properties (i.e., the category of the target) would be reflected in the LP in a later latency range.

The effect of task demands on the neural patterns generated by novel instruction encoding

Verbal instructions allow fast and optimal implementation of novel behaviors. Previous research has shown that different control-related variables organize neural activity in frontoparietal regions during the preparation of novel instructed task sets. Little is known, however, about how such variables organize brain activity under different task demands. In this study, we assessed the impact of implementation and memorization demands in the neural representation of novel instructions. We combined functional Magnetic Resonance Imaging (fMRI) with an instruction-following paradigm to compare the effect of three relevant control-related variables (integration of dimensions, response complexity, and stimulus category) across demands, and to explore the degree of overlap between these. Our results reveal, first, that the implementation and memorization of novel instructions share common neural patterns in several brain regions. Importantly, they also suggest that the preparation to implement instructions results in a strengthened coding of relevant control-related information in frontoparietal areas compared to their mere memorization. Overall, our study shows how the content of novel instructions proactively shapes brain activity based on multiple dimensions and how these organizational patterns are strengthened during implementation demands.

Early Evaluation of Fearfulness in Future Guide Dogs for Blind People

Fear is the leading cause of guide dog failure. Detecting the nature and causes of these fears as early as possible is the first step in preventing their occurrence. The process of habituation is a fundamental part of fear prevention. In this study, 11 puppies, all five months of age, underwent an emotional reactivity test (ERT) composed of 12 scored items, classified into three categories: unknown person (UP), sound and visual stimuli (SVS), and body sensitivity (BS). Salivary cortisol was also measured. Foster families were asked to complete a questionnaire concerning puppies’ habituation. The physiological data were correlated with UP (r = 0.71) and BS scores (r = 0.67), but not with SVS scores (r = 0.16), suggesting the ability of these dogs to control themselves when faced with the latter stimulus category. Additionally, the more time a puppy spent alone, the more likely it was to be afraid of SVS (p = 0.05). A correlation, albeit moderate, was detected between cortisol and habituation scores (r = 0.48). These results give us interesting avenues to explore, particularly regarding the importance of focusing on early puppy socialization and habituation to improve the numbers of guide dog candidates becoming successful guide dogs.

Category-specific item encoding in the medial temporal lobe and beyond: The role of reward

Forming new memories is a fundamental part of human life, and the medial temporal lobe (MTL) is central to memory formation. Recent research suggests that within MTL, the perirhinal and parahippocampal cortices (PRC, PHC) process object and scene memory, respectively, whereas the hippocampus (HC) is agnostic to stimulus category. It is unclear, however, whether MTL category specificity extends to item encoding. Furthermore, MTL does not act in isolation: Reward-related memories are formed in interplay with the dopaminergic midbrain (substantia nigra/ventral tegmental area, SNVTA) and amygdala (AMY), but it is unclear whether reward modulates neural item encoding in a category-specific way. To address these questions, we had 39 healthy volunteers (27 for all memory-based analyses) undergo functional magnetic resonance imaging while they solved an incidental encoding task, which paired objects or scenes with high or low reward, followed by a next-day surprise recognition test. Behaviourally, high reward preferably enhanced object memory. Importantly, neural activity in PRC and PHC reflected item encoding of objects and scenes, respectively. Moreover, AMY encoding effects were selective for high-reward objects, with a similar pattern in PRC. SNVTA and HC showed no clear evidence of item encoding. The behavioural and neural asymmetry of reward-related encoding effects may be conveyed through an anterior-temporal memory system, including AMY and PRC, potentially in interplay with the ventromedial prefrontal cortex (vmPFC).