Micro-state-based neural decoding of speech categorization using Bayesian non-parametrics

Understanding the many-to-many mapping between patterns of functional brain connectivity and discrete behavioral responses is critical for speech-language processing. We present a microstate-based analysis of EEG recordings to characterize spatio-temporal dynamics of neural activities that underly rapid speech categorization decisions. We implemented a data driven approach using Bayesian non-parametrics to capture the mapping between EEG and the speed of listeners phoneme identification [i.e., response time (RT)] during speech labeling tasks. Based on our empirical analyses, we show task-relevant events such as resting-state, stimulus coding, auditory-perceptual object (category) formation, and response selection can be explained using patterns of micro-state dwell-time and are decodable as unique time segments during speech perception. State-dependent activities localize to a fronto-temporo-parietal circuit (superior temporal, supramarginal, inferior frontal gyri) exposing a core decision brain network (DN) underlying rapid speech categorization. Furthermore, RTs were inversely proportional to the frequency of state transitions, such that the rate of change between brain microstates was higher for trials with slower compared to faster RTs. Our findings imply that during rapid speech perception, higher uncertainty producing prolonged RTs (slower decision-making) is associated with staying in the DN longer compared lower RTs (faster decisions). We also show that listeners perceptual RTs are highly sensitive to individual differences. Our computational method opens a new avenue in segmentation and dynamic brain connectivity for modeling neuroimaging data and understanding task-related cognitive events.

A spurious correlation between difference scores in evidence accumulation model parameters

Evidence-accumulation models are a useful tool for investigating the cognitive processes that give rise to behavioural data patterns in reaction times (RTs) and error rates. In their simplest form, evidence-accumulation models include three parameters: The average rate of evidence accumulation over time (drift rate) and the amount of evidence that needs to be accumulated before a response becomes selected (boundary) both characterise the response-selection process; a third parameter summarises all processes before and after the response-selection process (non-decision time). Researchers often compute experimental effects as simple difference scores between two within-subject conditions and such difference scores can also be computed on model parameters. In the present paper, we report spurious correlations between such model parameter difference scores, both in empirical data and in computer simulations. The most pronounced spurious effect is a negative correlation between boundary difference and non-decision difference, which amounts to r = –.70 or larger. In the simulations, we only observed this spurious negative correlation when either (a) there was no true difference in model parameters between simulated experimental conditions, or (b) only drift rate was manipulated between simulated experimental conditions; when a true difference existed in boundary separation, non-decision time, or all three main parameters, the correlation disappeared. We suggest that care should be taken when using evidence-accumulation model difference scores for correlational approaches, because the parameter difference scores can correlate in the absence of any true inter-individual differences at the population level.

The working memory costs of a central attentional bottleneck in multitasking

When two (or more) tasks, each requiring a rapid response, are performed at the same time then serial processing may occur at certain processing stages, such as the response selection. There is accumulating evidence that such serial processing involves additional control processes, such as inhibition, switching, and scheduling (termed the active scheduling account). The present study tested whether the existence of serial processing in multitasking leads to a requirement for processes that coordinate processing in this way (active scheduling account) and, furthermore, whether such control processes are linked to the executive functions (EF) of working memory (WM). To test this question, we merged the psychological refractory period (PRP) paradigm with a WM task, creating a complex WM span task. Participants were presented with a sequence of letters to remember, followed by a processing block in which they had to perform either a single task or a dual task, and finally were asked to recall the letters. Results showed that WM performance, i.e. the amount of letters recalled in the correct order, decreased when performing a dual task as compared to performing a single task during the retention interval. Two further experiments supported this finding using manipulations of the dual task difficulty. We conclude that the existence of serial processing in multitasking demands additional control processes (active scheduling) and that these processes are strongly linked to the executive functions of working memory.

Response Ranking with Multi-types of Deep Interactive Representations in Retrieval-based Dialogues

Building an intelligent dialogue system with the ability to select a proper response according to a multi-turn context is challenging in three aspects: (1) the meaning of a context–response pair is built upon language units from multiple granularities (e.g., words, phrases, and sub-sentences, etc.); (2) local (e.g., a small window around a word) and long-range (e.g., words across the context and the response) dependencies may exist in dialogue data; and (3) the relationship between the context and the response candidate lies in multiple relevant semantic clues or relatively implicit semantic clues in some real cases. However, existing approaches usually encode the dialogue with mono-type representation and the interaction processes between the context and the response candidate are executed in a rather shallow manner, which may lead to an inadequate understanding of dialogue content and hinder the recognition of the semantic relevance between the context and response. To tackle these challenges, we propose a representation [ K ] -interaction [ L ] -matching framework that explores multiple types of deep interactive representations to build context-response matching models for response selection. Particularly, we construct different types of representations for utterance–response pairs and deepen them via alternate encoding and interaction. By this means, the model can handle the relation of neighboring elements, phrasal pattern, and long-range dependencies during the representation and make a more accurate prediction through multiple layers of interactions between the context–response pair. Experiment results on three public benchmarks indicate that the proposed model significantly outperforms previous conventional context-response matching models and achieve slightly better results than the BERT model for multi-turn response selection in retrieval-based dialogue systems.

Dialogue History Matters! Personalized Response Selection in Multi-Turn Retrieval-Based Chatbots

Existing multi-turn context-response matching methods mainly concentrate on obtaining multi-level and multi-dimension representations and better interactions between context utterances and response. However, in real-place conversation scenarios, whether a response candidate is suitable not only counts on the given dialogue context but also other backgrounds, e.g., wording habits, user-specific dialogue history content. To fill the gap between these up-to-date methods and the real-world applications, we incorporate user-specific dialogue history into the response selection and propose a personalized hybrid matching network (PHMN). Our contributions are two-fold: (1) our model extracts personalized wording behaviors from user-specific dialogue history as extra matching information; (2) we perform hybrid representation learning on context-response utterances and explicitly incorporate a customized attention mechanism to extract vital information from context-response interactions so as to improve the accuracy of matching. We evaluate our model on two large datasets with user identification, i.e., personalized Ubuntu dialogue Corpus (P-Ubuntu) and personalized Weibo dataset (P-Weibo). Experimental results confirm that our method significantly outperforms several strong models by combining personalized attention, wording behaviors, and hybrid representation learning.

Categorical encoding of speech sounds: beyond auditory cortices

What processes lead to categorical perception of speech sounds? Investigation of this question is hampered by the fact that categorical speech perception is normally confounded by acoustic differences in the stimulus. By using ambiguous sounds, however, it is possible to dissociate acoustic from perceptual stimulus representations. We used a binaural integration task, where the inputs to the two ears were complementary so that phonemic identity emerged from their integration into a single percept. Twenty-seven normally hearing individuals took part in an fMRI study in which they were presented with an ambiguous syllable (intermediate between /da/ and /ga/) in one ear and with a meaning-differentiating acoustic feature (third formant) in the other ear. Multi-voxel pattern searchlight analysis was used to identify brain areas that consistently differentiated between response patterns associated with different syllable reports. By comparing responses to different stimuli with identical syllable reports and identical stimuli with different syllable reports, we disambiguated whether these regions primarily differentiated the acoustics of the stimuli or the syllable report. We found that BOLD activity patterns in the left anterior insula (AI), the left supplementary motor cortex, the left ventral motor cortex and the right motor and somatosensory cortex (M1/S1) represent listeners' syllable report irrespective of stimulus acoustics. The same areas have been previously implicated in decision-making (AI), response selection (SMA), and response initiation and feedback (M1/S1). Our results indicate that the emergence of categorical speech sounds implicates decision-making mechanisms and auditory-motor transformations acting on sensory inputs.

Biased credit assignment in motivational learning biases arises through prefrontal influences on striatal learning

Actions are biased by the outcomes they can produce: Humans are more likely to show action under reward prospect, but hold back under punishment prospect. Such motivational biases derive not only from biased response selection, but also from biased learning: humans tend to attribute rewards to their own actions, but are reluctant to attribute punishments to having held back. The neural origin of these biases is unclear; in particular, it remains open whether motivational biases arise solely from an evolutionarily old, subcortical architecture or also due to younger, cortical influences. Simultaneous EEG-fMRI allowed us to track which regions encoded biased prediction errors in which order. Biased prediction errors occurred in cortical regions (ACC, vmPFC, PCC) before subcortical regions (striatum). These results highlight that biased learning is not a mere feature of the basal ganglia, but arises through prefrontal cortical contributions, revealing motivational biases to be a potentially flexible, sophisticated mechanism.

Increased decision latency in alcohol use disorder reflects altered resting-state synchrony in the anterior salience network

Increased decision latency in alcohol use disorder (AUD) has been generally explained in terms of psychomotor slowing. Recent results suggest that AUD patients’ slowed decision-making might rather reflect alterations in the neural circuitry underlying the engagement of controlled processing by salient stimuli. We addressed this hypothesis by testing a relationship between decision latency at the Cambridge Gambling Task (CGT) and intrinsic brain activity in 22 individuals with AUD and 19 matched controls. CGT deliberation time was related to two complementary facets of resting-state fMRI activity, i.e. coherence and intensity, representing early biomarkers of functional changes in the intrinsic brain architecture. For both metrics, we assessed a multiple regression (to test a relationship with deliberation time in the whole sample), and an interaction analysis (to test a significantly different relationship with decision latency across groups). AUD patients’ slowed deliberation time (p < 0.025) reflected distinct facets of altered intrinsic activity in the cingulate node of the anterior salience network previously associated with the “output” motor stage of response selection. Its heightened activity in AUD patients compared with controls, tracking choice latency (p < 0.025 corrected), might represent a compensation mechanism counterbalancing the concurrent decrease of its internal coherent activity (p < 0.025 corrected). These findings provide novel insights into the intrinsic neural mechanisms underlying increased decision latency in AUD, involving decreased temporal synchronicity in networks promoting executive control by behaviourally relevant stimuli. These results pave the way to further studies assessing more subtle facets of decision-making in AUD, and their possible changes with rehabilitative treatment.

Negative affordance effect: automatic response inhibition triggered by handle orientation of non-target object

Habituated response tendency associated with affordance of an object is automatically inhibited if this affordance cue is extracted from a non-target object. This study presents two go/no-go experiments investigating whether this response control operates in response selection processes and whether it is linked to conflict-monitoring mechanisms. In the first experiment, the participants performed responses with one hand, and in the second experiment, with two hands. In addition, both experiments consisted of two blocks with varying frequency of go conditions (25%-go vs. 75%-go). The non-target-related response inhibition effect was only observed in Experiment 2 when the task required selecting between two hands. Additionally, the results did not reveal patterns typically related to conflict monitoring when go-frequency is manipulated and when a stimulus–response compatibility effect is examined relative to congruency condition of the previous trial. The study shows that the non-target-related response inhibition assists hand selection and is relatively resistant to conflict-monitoring processes.