Crossmodal Correspondence Between Auditory Timbre and Visual Shape

Crossmodal correspondences are defined as associations between crossmodal stimuli based on seemingly irrelevant stimulus features (i.e., bright shapes being associated with high-pitched sounds). There is a large body of research describing auditory crossmodal correspondences involving pitch and volume, but not so much involving auditory timbre, the character or quality of a sound. Adeli and colleagues (2014, Front. Hum. Neurosci. 8, 352) found evidence of correspondences between timbre and visual shape. The present study aimed to replicate Adeli et al.’s findings, as well as identify novel timbre–shape correspondences. Participants were tested using two computerized tasks: an association task, which involved matching shapes to presented sounds based on best perceived fit, and a semantic task, which involved rating shapes and sounds on a number of scales. The analysis of association matches reveals nonrandom selection, with certain stimulus pairs being selected at a much higher frequency. The harsh/jagged and smooth/soft correspondences observed by Adeli et al. were found to be associated with a high level of consistency. Additionally, high matching frequency of sounds with unstudied timbre characteristics suggests the existence of novel correspondences. Finally, the ability of the semantic task to supplement existing crossmodal correspondence assessments was demonstrated. Convergent analysis of the semantic and association data demonstrates that the two datasets are significantly correlated (−0.36) meaning stimulus pairs associated with a high level of consensus were more likely to hold similar perceived meaning. The results of this study are discussed in both theoretical and applied contexts.

The Effects of Mandarin Chinese Lexical Tones in Sound–Shape and Sound–Size Correspondences

Crossmodal correspondences refer to when specific domains of features in different sensory modalities are mapped. We investigated how vowels and lexical tones drive sound–shape (rounded or angular) and sound–size (large or small) mappings among native Mandarin Chinese speakers. We used three vowels (/i/, /u/, and /a/), and each vowel was articulated in four lexical tones. In the sound–shape matching, the tendency to match the rounded shape was decreased in the following order: /u/, /i/, and /a/. Tone 2 was more likely to be matched to the rounded pattern, whereas Tone 4 was more likely to be matched to the angular pattern. In the sound–size matching, /a/ was matched to the larger object more than /u/ and /i/, and Tone 2 and Tone 4 correspond to the large–small contrast. The results demonstrated that both vowels and tones play prominent roles in crossmodal correspondences, and sound–shape and sound–size mappings are heterogeneous phenomena.

Commercializing Sonic Seasoning in Multisensory Offline Experiential Events and Online Tasting Experiences

The term “sonic seasoning” refers to the deliberate pairing of sound/music with taste/flavour in order to enhance, or modify, the multisensory tasting experience. Although the recognition that people experience a multitude of crossmodal correspondences between stimuli in the auditory and chemical senses originally emerged from the psychophysics laboratory, the last decade has seen an explosion of interest in the use and application of sonic seasoning research findings, in a range of multisensory experiential events and online offerings. These marketing-led activations have included a variety of different approaches, from curating pre-composed music selections that have the appropriate sonic qualities (such as pitch or timbre), to the composition of bespoke music/soundscapes that match the specific taste/flavour of particular food or beverage products. Moreover, given that our experience of flavour often changes over time and frequently contains multiple distinct elements, there is also scope to more closely match the sonic seasoning to the temporal evolution of the various components (or notes) of the flavour experience. We review a number of case studies of the use of sonic seasoning, highlighting some of the challenges and opportunities associated with the various approaches, and consider the intriguing interplay between physical and digital (online) experiences. Taken together, the various examples reviewed here help to illustrate the growing commercial relevance of sonic seasoning research.

People With High Autistic Traits Show Fewer Consensual Crossmodal Correspondences Between Visual Features and Tastes

Crossmodal correspondences between visual features (e.g., color/shape) and tastes have been extensively documented in recent years. Visual colors and shapes have been shown to consensually match to specific tastes. Meanwhile, individuals with autism spectrum disorder are reported to have atypical sensory processing and deficits in multisensory integration. However, the influence of autistic traits on the formation of such correspondences is relatively unknown. Here, we examined whether autistic traits could influence visual–taste associations using an online questionnaire survey among Japanese participants. The results showed that the participants exhibited strong color–taste, shape–taste, and shape–color associations, and the proportions of choosing the consensual color–taste/shape–color associations were significantly associated with autistic traits. The participants with higher autistic quotient scores chose fewer of the consensual color–taste/shape–color associations while there was no difference in choosing shape–taste associations. We interpreted the results as statistical learning with a reduced prior knowledge effect in participants with higher autistic quotient scores.

Musical Scents: On the Surprising Absence of Scented Musical/Auditory Events, Entertainments, and Experiences

The matching of scents with music is both one of the most natural (or intuitive) of crossmodal correspondences and, at the same time, one of the least frequently explored combinations of senses in an entertainment and multisensory experiential design context. This narrative review highlights the various occasions over the last century or two when scents and sounds have coincided, and the various motivations behind those who have chosen to bring these senses together: This has included everything from the masking of malodour to the matching of the semantic meaning or arousal potential of the two senses, through to the longstanding and recently-reemerging interest in the crossmodal correspondences (now that they have been distinguished from the superficially similar phenomenon of synaesthesia, with which they were previously often confused). As such, there exist a number of ways in which these two senses can be incorporated into meaningful multisensory experiences that can potentially resonate with the public. Having explored the deliberate combination of scent and music (or sound) in everything from “scent-sory” marketing through to fragrant discos and olfactory storytelling, I end by summarizing some of the opportunities around translating such unusual multisensory experiences from the public to the private sphere. This will likely be via the widespread dissemination of sensory apps that promise to convert (or translate) from one sense (likely scent) to another (e.g., music), as has, for example already started to occur in the world of music selections to match the flavour of specific wines.