Mid-Air Tactile Sensations Evoked by Laser-Induced Plasma: A Neurophysiological Study

This study demonstrates the feasibility of a mid-air means of haptic stimulation at a long distance using the plasma effect induced by laser. We hypothesize that the stress wave generated by laser-induced plasma in the air can propagate through the air to reach the nearby human skin and evoke tactile sensation. To validate this hypothesis, we investigated somatosensory responses in the human brain to laser plasma stimuli by analyzing electroencephalography (EEG) in 14 participants. Three types of stimuli were provided to the index finger: a plasma stimulus induced from the laser, a mechanical stimulus transferred through Styrofoam stick, and a sham stimulus providing only the sound of the plasma and mechanical stimuli at the same time. The event-related desynchronization/synchronization (ERD/S) of sensorimotor rhythms (SMRs) in EEG was analyzed. Every participant verbally reported that they could feel a soft tap on the finger in response to the laser stimulus, but not to the sham stimulus. The spectrogram of EEG evoked by laser stimulation was similar to that evoked by mechanical stimulation; alpha ERD and beta ERS were present over the sensorimotor area in response to laser as well as mechanical stimuli. A decoding analysis revealed that classification error increased when discriminating ERD/S patterns between laser and mechanical stimuli, compared to the case of discriminating between laser and sham, or mechanical and sham stimuli. Our neurophysiological results confirm that tactile sensation can be evoked by the plasma effect induced by laser in the air, which may provide a mid-air haptic stimulation method.

Normal and Tangential Forces Combine to Convey Contact Pressure during Dynamic Tactile Stimulation

Humans need to accurately process the contact forces that arise as they perform everyday haptic interactions, but the mechanisms by which the forces on the skin are represented and integrated remain little understood. In this study, we used a force-controlled robotic platform and simultaneous ultrasonic modulation of the finger-surface friction to briefly and independently manipulate the normal and tangential forces during passive haptic stimulation by a flat surface. When participants were asked whether the contact pressure on their finger had briefly increased or decreased, they could not distinguish the normal force from the tangential force. Instead, they integrated the normal and tangential components of the force vector into a multidimensional computation of the contact force. We additionally investigated whether participants relied on three common contact-force metrics. Interestingly, the change in the amplitude of the force vector predicted participants’ responses better than the change of the coefficient of dynamic friction and the change of the angle of the contact force vector. Thus, intensive cues related to the amplitude of the applied force may be meaningful for the sensing of contact pressure during haptic stimulation by a moving surface.

Can Haptic Stimulation Enhance Music Perception in Hearing-Impaired Listeners?

Cochlear implants (CIs) have been remarkably successful at restoring hearing in severely-to-profoundly hearing-impaired individuals. However, users often struggle to deconstruct complex auditory scenes with multiple simultaneous sounds, which can result in reduced music enjoyment and impaired speech understanding in background noise. Hearing aid users often have similar issues, though these are typically less acute. Several recent studies have shown that haptic stimulation can enhance CI listening by giving access to sound features that are poorly transmitted through the electrical CI signal. This “electro-haptic stimulation” improves melody recognition and pitch discrimination, as well as speech-in-noise performance and sound localization. The success of this approach suggests it could also enhance auditory perception in hearing-aid users and other hearing-impaired listeners. This review focuses on the use of haptic stimulation to enhance music perception in hearing-impaired listeners. Music is prevalent throughout everyday life, being critical to media such as film and video games, and often being central to events such as weddings and funerals. It represents the biggest challenge for signal processing, as it is typically an extremely complex acoustic signal, containing multiple simultaneous harmonic and inharmonic sounds. Signal-processing approaches developed for enhancing music perception could therefore have significant utility for other key issues faced by hearing-impaired listeners, such as understanding speech in noisy environments. This review first discusses the limits of music perception in hearing-impaired listeners and the limits of the tactile system. It then discusses the evidence around integration of audio and haptic stimulation in the brain. Next, the features, suitability, and success of current haptic devices for enhancing music perception are reviewed, as well as the signal-processing approaches that could be deployed in future haptic devices. Finally, the cutting-edge technologies that could be exploited for enhancing music perception with haptics are discussed. These include the latest micro motor and driver technology, low-power wireless technology, machine learning, big data, and cloud computing. New approaches for enhancing music perception in hearing-impaired listeners could substantially improve quality of life. Furthermore, effective haptic techniques for providing complex sound information could offer a non-invasive, affordable means for enhancing listening more broadly in hearing-impaired individuals.

May I Smell Your Attention: Exploration of Smell and Sound for Visuospatial Attention in Virtual Reality

When interacting with technology, attention is mainly driven by audiovisual and increasingly haptic stimulation. Olfactory stimuli are widely neglected, although the sense of smell influences many of our daily life choices, affects our behavior, and can catch and direct our attention. In this study, we investigated the effect of smell and sound on visuospatial attention in a virtual environment. We implemented the Bells Test, an established neuropsychological test to assess attentional and visuospatial disorders, in virtual reality (VR). We conducted an experiment with 24 participants comparing the performance of users under three experimental conditions (smell, sound, and smell and sound). The results show that multisensory stimuli play a key role in driving the attention of the participants and highlight asymmetries in directing spatial attention. We discuss the relevance of the results within and beyond human-computer interaction (HCI), particularly with regard to the opportunity of using VR for rehabilitation and assessment procedures for patients with spatial attention deficits.

Electro-Haptic Stimulation: A New Approach for Improving Cochlear-Implant Listening

Cochlear implants (CIs) have been remarkably successful at restoring speech perception for severely to profoundly deaf individuals. Despite their success, several limitations remain, particularly in CI users’ ability to understand speech in noisy environments, locate sound sources, and enjoy music. A new multimodal approach has been proposed that uses haptic stimulation to provide sound information that is poorly transmitted by the implant. This augmenting of the electrical CI signal with haptic stimulation (electro-haptic stimulation; EHS) has been shown to improve speech-in-noise performance and sound localization in CI users. There is also evidence that it could enhance music perception. We review the evidence of EHS enhancement of CI listening and discuss key areas where further research is required. These include understanding the neural basis of EHS enhancement, understanding the effectiveness of EHS across different clinical populations, and the optimization of signal-processing strategies. We also discuss the significant potential for a new generation of haptic neuroprosthetic devices to aid those who cannot access hearing-assistive technology, either because of biomedical or healthcare-access issues. While significant further research and development is required, we conclude that EHS represents a promising new approach that could, in the near future, offer a non-invasive, inexpensive means of substantially improving clinical outcomes for hearing-impaired individuals.

Sensitivity to Haptic Sound-Localization Cues at Different Body Locations

Cochlear implants (CIs) recover hearing in severely to profoundly hearing-impaired people by electrically stimulating the cochlea. While they are extremely effective, spatial hearing is typically severely limited. Recent studies have shown that haptic stimulation can supplement the electrical CI signal (electro-haptic stimulation) and substantially improve sound localization. In haptic sound-localization studies, the signal is extracted from the audio received by behind-the-ear devices and delivered to each wrist. Localization is achieved using tactile intensity differences (TIDs) across the wrists, which match sound intensity differences across the ears (a key sound localization cue). The current study established sensitivity to across-limb TIDs at three candidate locations for a wearable haptic device, namely: the lower tricep and the palmar and dorsal wrist. At all locations, TID sensitivity was similar to the sensitivity to across-ear intensity differences for normal-hearing listeners. This suggests that greater haptic sound-localization accuracy than previously shown can be achieved. The dynamic range was also measured and far exceeded that available through electrical CI stimulation for all of the locations, suggesting that haptic stimulation could provide additional sound-intensity information. These results indicate that an effective haptic aid could be deployed for any of the candidate locations, and could offer a low-cost, non-invasive means of improving outcomes for hearing-impaired listeners.

Attention Enhancement for Exoskeleton-Assisted Hand Rehabilitation Using Fingertip Haptic Stimulation

Active enrollment in rehabilitation training yields better treatment outcomes. This paper introduces an exoskeleton-assisted hand rehabilitation system. It is the first attempt to combine fingertip cutaneous haptic stimulation with exoskeleton-assisted hand rehabilitation for training participation enhancement. For the first time, soft material 3D printing techniques are adopted to make soft pneumatic fingertip haptic feedback actuators to achieve cheaper and faster iterations of prototype designs with consistent quality. The fingertip haptic stimulation is synchronized with the motion of our hand exoskeleton. The contact force of the fingertips resulted from a virtual interaction with a glass of water was based on data collected from normal hand motions to grasp a glass of water. System characterization experiments were conducted and exoskeleton-assisted hand motion with and without the fingertip cutaneous haptic stimulation were compared in an experiment involving healthy human subjects. Users’ attention levels were monitored in the motion control process using a Brainlink EEG-recording device and software. The results of characterization experiments show that our created haptic actuators are lightweight (6.8 ± 0.23 g each with a PLA fixture and Velcro) and their performance is consistent and stable with small hysteresis. The user study experimental results show that participants had significantly higher attention levels with additional haptic stimulations compared to when only the exoskeleton was deployed; heavier stimulated grasping weight (a 300 g glass) was associated with significantly higher attention levels of the participants compared to when lighter stimulated grasping weight (a 150 g glass) was applied. We conclude that haptic stimulations increase the involvement level of human subjects during exoskeleton-assisted hand exercises. Potentially, the proposed exoskeleton-assisted hand rehabilitation with fingertip stimulation may better attract user’s attention during treatment.