Dissecting self-voice perception: From bone conduction to robotically-induced self-other voice misattribution in healthy listeners

Inspired by the prominent theory suggesting that auditory-verbal hallucinations (AVH) - the sensation of hearing voices without present speakers - arise as misattribution of inner speech towards external agents, my thesis revolved around self-voice perception and the experimental attempts of self-other voice misattribution in a healthy, non-hallucinating population, potentially mimicking the AVH phenomenology.

In the first part of my thesis, I investigated behavioral (Study 1) and neural (Study 2) underpinnings of self-other voice discrimination (SOVD). Compared to other self-related processes, self-voice perception has been investigated to a surprisingly lesser extent. Namely, self-voice research has been thwarted by the inability to experimentally match self-voice recordings to the natural sound of our voice that is altered by bone conduction. In a series of five experiments, I showed that this discrepancy can be reduced by presenting self-voice stimuli through a commercial bone conduction headset, thereby rendering self-voice as an essentially multi-modal construct. My data further shows that self-voice recognition differs from the recognition of familiar voices, however, that it still involves some familiarity processing. In addition, I explored the roles other-voice familiarity and acoustic similarity to other voice play in SOVD. Finally, I identified a self-voice specific EEG pattern, around 345 milliseconds after stimulus onset, that followed the initial auditory cortex activation, discriminated the self-voice from the voice of another unfamiliar person, and activated an extended network involving the cingulate cortex, insula, and medial temporal lobe structures. Moreover, this network was recruited less frequently with self-voices presented through bone conduction, and the occurrence of the network negatively correlated with SOVD task performance.

Based on these methodological, behavioral, and neural findings in SOVD, in the second part of my thesis, I tried to experimentally induce specific misattributions of self-towards-other voices, following the prominent theory linking AVH with self-monitoring deficits. In order to do so, I combined the aforementioned SOVD task with a robotic procedure able to engender mild hallucinations in healthy individuals by perturbing bodily self-monitoring mechanisms. With such a procedure, I managed to alter self-voice perception of healthy participants (Study 3), what I further related to breathing (Study 4). Finally, I managed to induce identity-specific AVH in healthy individuals (Study 5), as quantified by the false alarm rate in a voice detection task, thereby contributing to the understanding of the mechanisms underlying AVH and relating them to impairments in bodily self-monitoring.

Laying at the intersection of three sub-fields of neuroscience - voice perception, self-processing, and psychiatry - my thesis made a contribution by identifying neural correlates of SOVD, by proposing a new, multimodal perspective on self-voice-related research questions, and by demonstrating an experimental method of inducing AVH in a controlled laboratory environment. Together, my findings shed new light on the interactions between self-voice perception, sensorimotor processing, and interoception.

EEG Spatiotemporal Patterns Underlying Self-other Voice Discrimination

Sixto Luis Alcoba Banqueri, Olaf Blanke, Giannina Rita Iannotti, Thomas Koenig, Pavo Orepic

There is growing evidence showing that the representation of the human “self” recruits special systems across different functions and modalities. Compared to self-face and self-body representations, few studies have investigated neural underpinnings specific to self-voice. Moreover, self-voice stimuli in those studies were consistently presented through air and lacking bone conduction, rendering the sound of self-voice stimuli different to the self-voice heard during natural speech. Here, we combined psychophysics, voice-morphing technology, and high-density EEG in order to identify the spatiotemporal patterns underlying self-other voice discrimination (SOVD) in a population of 26 healthy participants, both with air- and bone-conducted stimuli. We identified a self-voice-specific EEG topographic map occurring around 345 ms post-stimulus and activating a network involving insula, cingulate cortex, and medial temporal lobe structures. Occurrence of this map was modulated both with SOVD task performance and bone conduction. Specifically, the better participants performed at SOVD task, the less frequently they activated this network. In addition, the same network was recruited less frequently with bone conduction, which, accordingly, increased the SOVD task performance. This work could have an important clinical impact. Indeed, it reveals neural correlates of SOVD impairments, believed to account for auditory-verbal hallucinations, a common and highly distressing psychiatric symptom.

2021

Breathing affects self-other voice discrimination in a bodily state associated with somatic passivity

Olaf Blanke, Nathan Quentin Faivre, Pavo Orepic, Hyeongdong Park, Giulio Rognini

A growing number of studies have focused on identifying cognitive processes that are modulated by interoceptive signals, particularly in relation to the respiratory or cardiac cycle. Considering the fundamental role of interoception in bodily self-consciousness, we here investigated whether interoceptive signals also impact self-voice perception. We applied an interactive, robotic paradigm associated with somatic passivity (a bodily state characterized by illusory misattribution of self-generated touches to someone else) to investigate whether somatic passivity impacts self-voice perception as a function of concurrent interoceptive signals. Participants' breathing and heartbeat signals were recorded while they performed two self-voice tasks (self-other voice discrimination and loudness perception) and while simultaneously experiencing two robotic conditions (somatic passivity condition; control condition). Our data reveal that respiration, but not cardiac activity, affects self-voice perception: participants were better at discriminating self-voice from another person's voice during the inspiration phase of the respiration cycle. Moreover, breathing effects were prominent in participants experiencing somatic passivity and a different task with the same stimuli (i.e., judging the loudness and not identity of the voices) was unaffected by breathing. Combining interoception and voice perception with self-monitoring framework, these data extend findings on breathing-dependent changes in perception and cognition to self-related processing.

WILEY2022

Analysis of Anticipation Related Potentials for Brain Computer Interaction

Gangadhar Garipelli

Anticipation is a mental process during which a person actively engages in a phase required for the sensory perception and execution of the optimal actions at the arrival of the relevant future events. Since this process occurs before the execution of an intended action, it may be used as a control signal for Brain Computer Interface (BCI) applications. Recognition of neural correlates of this process can enhance the performance of a BCI and in turn reduce mental workload of its users. To this end, it is vital to understand the neural correlates involved in this process and to design robust methods for its recognition in single trials. The analysis of these correlates may also contribute to the basic knowledge of the mechanisms underlying this behavior. The thesis provides three major contributions: (i) it reports methods for the robust recognition of anticipation related Electroencephalogram (EEG) potentials (ii) it provides insights into the selection of appropriate preprocessing steps required for enhancing the Signal-to-Noise Ratio (SNR) of anticipatory slow cortical potentials (SCPs) and (iii) it identifies scalp area specific oscillatory activity related to different aspects of anticipatory behavior. First, we focus on methods for the single trial recognition of anticipatory SCPs using the widely known classical contingent negative variation (CNV) paradigm. Using this paradigm, we demonstrate the feasibility of recognizing the anticipatory SCPs (CNV potentials) using features thatmodel its temporal pattern. We propose a Bayesian approach that exploits temporal evolution and redundancy to quickly classify (e.g., within half of the anticipatory period), without compromising classification accuracy. We then improve upon these recognition rates by using a source localization technique based on the biophysical model of the human head. We further validate the feasibility of recognizing CNV potentials in an online experiment, and report for the first time that, under controlled conditions, these potentials can be reproduced and recognized in realistic interaction scenarios (assistive technology web-browsing) with high accuracies. Second, the thesis provides insights into the selection of appropriate preprocessing stages required for improving the SNR of SCPs. The CNV potentials are characterized by low frequencies that are usually recorded with full-DC, and hence suffer from task-irrelevant high amplitude fluctuations and spatial noise. To account for this, we identified appropriate spectral and spatial filters to improve the SNR. We demonstrate the potential of these preprocessing stages by using fusing multiple electrode specific linear classifiers, which achieve recognition performances of 90±2% (area under curve of receiver operating characteristic), where the classifiers are trained using recordings from one day and tested on the recordings from several days apart. Finally, the thesis identifies different facets of anticipatory behavior. Apart from the widely known CNV potentials, it is not clear which other spectral bands could be related to anticipatory behavior. Using recordings from an experiment (i.e., the assistive technology web browser) where multiple warning stimuli predicted an imperative stimulus, we explored the phase and amplitude response of various oscillatory sub-bands for the identification of markers that could be associated with different aspects of anticipation. From this study we report that: (i) there are duration (4-10 seconds) specific changes in Electroencephalography (EEG) activity in the range 0-1 Hz in the central electrodes correlate with reaction time, (ii) there exist slow oscillations (0.1-1 Hz) in the central electrodes that exhibit phase tuning up to 4 seconds before the onset of a target cue, (iii) there are delta oscillations (1.5-3 Hz), which are entrained to predictive rhythmic warning cues, (iv) there is a selective modulation (increase or decrease) in the amplitude of occipital alpha band (8-12 Hz) based on the relevance of forthcoming visual cue, and (v) there exist a reduction in the beta band (14-30 Hz) amplitude in the sensory motor and association areas lasting up to 10 seconds. The phase tuning and entrainment resulted in a low variance of phase values at the arrival of the imperative stimulus, which may be required for its optimal processing. The amplitude modulation of alpha band activity is likely to be a resultant of sensory suppression and attention. The reduction of beta-band activity over long periods of time suggests holding of sensory-motor association areas until the execution of a planned action. We believe that these observations are the consequence of the endogenous drive on the ongoing oscillations to enhance the processing of the forthcoming stimuli and preparation for an intended action. In summary, the thesis provides methods for the recognition of anticipatory SCPs by exploiting spectral and spatio-temporal characteristics with high performances. By exploring various other oscillatory sub-bands, the spectral characteristics of different aspects of anticipatory behavior are also identified. Further, methods modeling these characteristics can bring forth more robust and faster techniques for BCI systems.

EPFL2011

Analysis of Anticipation Related Potentials for Brain Computer Interaction

Gangadhar Garipelli

EPFL2011