The Impact of Pitch and Timbre Cues on Auditory Grouping and Stream Segregation

In multi-talker listening environments, the culmination of different voice streams may lead to the distortion of each source’s individual message, causing deficits in comprehension. Voice characteristics, such as pitch and timbre, are major dimensions of auditory perception and play a vital role in grouping and segregating incoming sounds based on their acoustic properties. The current study investigated how pitch and timbre cues (determined by fundamental frequency, notated as F0, and spectral slope, respectively) can affect perceptual integration and segregation of complex-tone sequences within an auditory streaming paradigm. Twenty normal-hearing listeners participated in a traditional auditory streaming experiment using two alternating sequences of harmonic tone complexes A and B with manipulating F0 and spectral slope. Grouping ranges, the F0/spectral slope ranges over which auditory grouping occurs, were measured with various F0/spectral slope differences between tones A and B. Results demonstrated that the grouping ranges were maximized in the absence of the F0/spectral slope differences between tones A and B and decreased by 2 times as their differences increased to ±1-semitone F0 and ±1-dB/octave spectral slope. In other words, increased differences in either F0 or spectral slope allowed listeners to more easily distinguish between harmonic stimuli, and thus group them together less. These findings suggest that pitch/timbre difference cues play an important role in how we perceive harmonic sounds in an auditory stream, representing our ability to group or segregate human voices in a multi-talker listening environment.

The effects of aging and musicianship on the use of auditory streaming cues

Auditory stream segregation, or separating sounds into their respective sources, and tracking them over time is a fundamental auditory ability. Previous research has separately explored the impacts of aging and musicianship on the ability to separate and follow auditory streams. The current study evaluated the simultaneous effects of age and musicianship on auditory streaming induced by three physical features: intensity, spectral envelope and temporal envelope. In the first study, older and younger musicians and non-musicians with normal hearing identified deviants in a four-note melody interleaved with distractors that were more or less similar to the melody in terms of intensity, spectral envelope and temporal envelope. In the second study, older and younger musicians and non-musicians participated in a dissimilarity rating paradigm with pairs of melodies that differed along the same three features. Results suggested that auditory streaming skills are maintained in older adults but that older adults rely on intensity more than younger adults while musicianship is associated with increased sensitivity to spectral and temporal envelope, acoustic features that are typically less effective for stream segregation, particularly in older adults.

Resetting of Auditory and Visual Segregation Occurs After Transient Stimuli of the Same Modality

In the presence of a continually changing sensory environment, maintaining stable but flexible awareness is paramount, and requires continual organization of information. Determining which stimulus features belong together, and which are separate is therefore one of the primary tasks of the sensory systems. Unknown is whether there is a global or sensory-specific mechanism that regulates the final perceptual outcome of this streaming process. To test the extent of modality independence in perceptual control, an auditory streaming experiment, and a visual moving-plaid experiment were performed. Both were designed to evoke alternating perception of an integrated or segregated percept. In both experiments, transient auditory and visual distractor stimuli were presented in separate blocks, such that the distractors did not overlap in frequency or space with the streaming or plaid stimuli, respectively, thus preventing peripheral interference. When a distractor was presented in the opposite modality as the bistable stimulus (visual distractors during auditory streaming or auditory distractors during visual streaming), the probability of percept switching was not significantly different than when no distractor was presented. Conversely, significant differences in switch probability were observed following within-modality distractors, but only when the pre-distractor percept was segregated. Due to the modality-specificity of the distractor-induced resetting, the results suggest that conscious perception is at least partially controlled by modality-specific processing. The fact that the distractors did not have peripheral overlap with the bistable stimuli indicates that the perceptual reset is due to interference at a locus in which stimuli of different frequencies and spatial locations are integrated.

Auditory Streaming Complexity and Renaissance Mass Cycles

How did Renaissance listeners experience the polyphonic mass ordinary cycle in the soundscape of the church? We hypothesize that the textural differences in complexity between mass movements allowed listeners to track the progress of the service, regardless of intelligibility of the text or sophisticated musical knowledge. Building on the principles of auditory scene analysis, this article introduces the Auditory Streaming Complexity Estimate, a measure to evaluate the blending or separation of each part in polyphony, resulting in a moment-by-moment tally of how many independent streams or sound objects might be heard. When applied to symbolic scores for a corpus of 216 polyphonic mass ordinary cycles composed between c. 1450 and 1600, we show that the Streaming Complexity Estimate captures information distinct from the number of parts in the score or the distribution of voices active through the piece. While composers did not all follow the same relative complexity strategy for mass ordinary movements, there is a robust hierarchy emergent from the corpus as a whole: a shallow V shape with the Credo as the least complex and the Agnus Dei as the most. The streaming complexity of masses also significantly increased over the years represented in this corpus.

Pupillometry in auditory multistability

In multistability, a constant stimulus induces alternating perceptual interpretations. For many forms of visual multistability, the transition from one interpretation to another (“perceptual switch”) is accompanied by a dilation of the pupil. Here we ask whether the same holds for auditory multistability, specifically auditory streaming. Two tones were played in alternation, yielding four distinct interpretations: the tones can be perceived as one integrated percept (single sound source), or as segregated with either tone or both tones in the foreground. We found that the pupil dilates significantly around the time a perceptual switch is reported (“multistable condition”). When participants instead responded to actual stimulus changes that closely mimicked the multistable perceptual experience (“replay condition”), the pupil dilated more around such responses than in multistability. This still held when data were corrected for the pupil response to the stimulus change as such. Hence, active responses to an exogeneous stimulus change trigger a stronger or temporally more confined pupil dilation than responses to an endogenous perceptual switch. In another condition, participants randomly pressed the buttons used for reporting multistability. In Study 1, this “random condition” failed to sufficiently mimic the temporal pattern of multistability. By adapting the instructions, in Study 2 we obtained a response pattern more similar to the multistable condition. In this case, the pupil dilated significantly around the random button presses. Albeit numerically smaller, this pupil response was not significantly different from the multistable condition. While there are several possible explanations–related, e.g., to the decision to respond–this underlines the difficulty to isolate a purely perceptual effect in multistability. Our data extend previous findings from visual to auditory multistability. They highlight methodological challenges in interpreting such data and suggest possible approaches to meet them, including a novel stimulus to simulate the experience of perceptual switches in auditory streaming.

Resetting of Auditory and Visual Segregation Occurs After Transient Stimuli of the Same Modality

In the presence of a continually changing sensory environment, maintaining stable but flexible awareness is paramount, and requires continual organization of information. Determining which stimulus features belong together, and which are separate is therefore one of the primary tasks of the sensory systems. Unknown is whether there is a global or sensory-specific mechanism that regulates the final perceptual outcome of this streaming process. To test the extent of modality independence in perceptual control, an auditory streaming experiment, and a visual moving-plaid experiment were performed. Both were designed to evoke alternating perception of an integrated or segregated percept. In both experiments, transient auditory and visual distractor stimuli were presented in separate blocks, such that the distractors did not overlap in frequency or space with the streaming or plaid stimuli, respectively, thus preventing peripheral interference. When a distractor was presented in the opposite modality as the bistable stimulus (visual distractors during auditory streaming or auditory distractors during visual streaming), the rate of percept switching was not significantly different than when no distractor was presented. Conversely, significant differences in switch rate were observed following within-modality distractors, but only when the pre-distractor percept was segregated. Due to the modality-specificity of the distractor-induced resetting, the results suggest that conscious perception is at least partially controlled by modality-specific processing. The fact that the distractors did not have peripheral overlap with the bistable stimuli indicates that the perceptual reset is due to interference at a locus in which stimuli of different frequencies and spatial locations are integrated.

Auditory streaming emerges from fast excitation and slow delayed inhibition

In the auditory streaming paradigm, alternating sequences of pure tones can be perceived as a single galloping rhythm (integration) or as two sequences with separated low and high tones (segregation). Although studied for decades, the neural mechanisms underlining this perceptual grouping of sound remains a mystery. With the aim of identifying a plausible minimal neural circuit that captures this phenomenon, we propose a firing rate model with two periodically forced neural populations coupled by fast direct excitation and slow delayed inhibition. By analyzing the model in a non-smooth, slow-fast regime we analytically prove the existence of a rich repertoire of dynamical states and of their parameter dependent transitions. We impose plausible parameter restrictions and link all states with perceptual interpretations. Regions of stimulus parameters occupied by states linked with each percept match those found in behavioural experiments. Our model suggests that slow inhibition masks the perception of subsequent tones during segregation (forward masking), whereas fast excitation enables integration for large pitch differences between the two tones.

Behavioral Outcomes and Neural Network Modeling of a Novel, Putative, Recategorization Sound Therapy

The mechanisms underlying sound’s effect on tinnitus perception are unclear. Tinnitus activity appears to conflict with perceptual expectations of “real” sound, resulting in it being a salient signal. Attention diverted towards tinnitus during the later stages of object processing potentially disrupts high-order auditory streaming, and its uncertain nature results in negative psychological responses. This study investigated the benefits and neurophysiological basis of passive perceptual training and informational counseling to recategorize phantom perception as a more real auditory object. Specifically, it examined underlying psychoacoustic correlates of tinnitus and the neural activities associated with tinnitus auditory streaming and how malleable these are to change with targeted intervention. Eighteen participants (8 females, 10 males, mean age = 61.6 years) completed the study. The study consisted of 2 parts: (1) An acute exposure over 30 min to a sound that matched the person’s tinnitus (Tinnitus Avatar) that was cross-faded to a selected nature sound (Cicadas, Fan, Water Sound/Rain, Birds, Water and Bird). (2) A chronic exposure for 3 months to the same “morphed” sound. A brain-inspired spiking neural network (SNN) architecture was used to model and compare differences between electroencephalography (EEG) patterns recorded prior to morphing sound presentation, during, after (3-month), and post-follow-up. Results showed that the tinnitus avatar generated was a good match to an individual’s tinnitus as rated on likeness scales and was not rated as unpleasant. The five environmental sounds selected for this study were also rated as being appropriate matches to individuals’ tinnitus and largely pleasant to listen to. There was a significant reduction in the Tinnitus Functional Index score and subscales of intrusiveness of the tinnitus signal and ability to concentrate with the tinnitus trial end compared to baseline. There was a significant decrease in how strong the tinnitus signal was rated as well as ratings of how easy it was to ignore the tinnitus signal on severity rating scales. Qualitative analysis found that the environmental sound interacted with the tinnitus in a positive way, but participants did not experience change in severity, however, characteristics of tinnitus, including pitch and uniformity of sound, were reported to change. The results indicate the feasibility of the computational SNN method and preliminary evidence that the sound exposure may change activation of neural tinnitus networks and greater bilateral hemispheric involvement as the sound morphs over time into natural environmental sound; particularly relating to attention and discriminatory judgments (dorsal attention network, precentral gyrus, ventral anterior network). This is the first study that attempts to recategorize tinnitus using passive auditory training to a sound that morphs from resembling the person’s tinnitus to a natural sound. These findings will be used to design future-controlled trials to elucidate whether the approach used differs in effect and mechanism from conventional Broadband Noise (BBN) sound therapy.