Looming Signals Reveal Synergistic Principles of Multisensory Integration

Multisensory interactions are a fundamental feature of brain organization. Principles governing multisensory processing have been established by varying stimulus location, timing and efficacy independently. Determining whether and how such principles operate when stimuli vary dynamically in their perceived distance (as when looming/receding) provides an assay for synergy among the above principles and also means for linking multisensory interactions between rudimentary stimuli with higher-order signals used for communication and motor planning. Human participants indicated movement of looming or receding versus static stimuli that were visual, auditory, or multisensory combinations while 160-channel EEG was recorded. Multivariate EEG analyses and distributed source estimations were performed. Nonlinear interactions between looming signals were observed at early poststimulus latencies (∼75 ms) in analyses of voltage waveforms, global field power, and source estimations. These looming-specific interactions positively correlated with reaction time facilitation, providing direct links between neural and performance metrics of multisensory integration. Statistical analyses of source estimations identified looming-specific interactions within the right claustrum/insula extending inferiorly into the amygdala and also within the bilateral cuneus extending into the inferior and lateral occipital cortices. Multisensory effects common to all conditions, regardless of perceived distance and congruity, followed (∼115 ms) and manifested as faster transition between temporally stable brain networks (vs summed responses to unisensory conditions). We demonstrate the early-latency, synergistic interplay between existing principles of multisensory interactions. Such findings change the manner in which to model multisensory interactions at neural and behavioral/perceptual levels. We also provide neurophysiologic backing for the notion that looming signals receive preferential treatment during perception.

Spatio-temporal coupling of the electric and hemodynamic brain responses in humans

Roberto Martuzzi

Our comprehension of human brain functions and their dynamics has been dramatically improved by recent developments in non-invasive imaging techniques. These methods can be divided into two different categories, according to the nature of the measured signal: hemodynamic techniques, such as functional magnetic resonance imaging (fMRI) and positiron emission tomography (PET), and electromagnetic techniques, such as electroencephalography (EEG) and magnetoencephalography (MEG). These two categories are have complementary characteristics: hemodynamic techniques have a good spatial resolution (on a millimeter spatial scale) but have a poor temporal resolution, which is inherently limited by the rate changes in blood flow and oxygenation. Electromagnetic techniques have sub-millisecond temporal resolution but have a poor spatial resolution, since the analysis of intracranial generators requires the solution of an underdetermined inverse problem (i.e. there are infinite solutions that can explain equally well the same scalp-recorded distribution). The complementarity of the characteristics of these two families of methods allowed researchers to suppose that the understanding of spatio-temporal brain dynamics can be drastically improved by their combination (so-called multimodal imaging). Unfortunately some caveats hinder such combination. First, the nature of neurovascular coupling is still poorly understood. Second, analytical methods for multimodal imaging are largely in their infancy. The first part of this thesis focuses on the analysis of the temporal characteristics of the blood oxygenation level dependent (BOLD) signal and on how they are modulated by stimulus conditions. To analyze the BOLD dynamics, a novel method for synchronizing stimulus delivery and volume acquisition was developed. This method allows for estimating the BOLD signal with a high temporal resolution (in this thesis up to 125 ms) and for studying how the temporal characteristics (in this thesis mainly the BOLD peak latency and slope) are modulated by stimulus conditions (with an approach similar to that used in the analysis of the EEG evoked potentials). We applied this novel technique to a simple reaction time task to lateralized visual stimuli (the so-called Poffenberger paradigm) as well as to a multisensory auditory-visual reaction time task. In the first study (the Poffenberger paradigm) the analysis of BOLD dynamics supported the theory of a bilateral visuo-motor pathway even in the case of a visual stimulus ipsilateral to the responding hand. In the second study, (the auditory-visual multisensory reaction-time task), the analysis showed auditory-visual interactions within both primary auditory and visual cortices that could not be otherwise revealed by traditional fMRI analysis methods since it does not involve changes in signal amplitude. The second part of this thesis focuses on the comparison of the statistical results obtained by the analyses of fMRI and of the intracranial local field potentials (LFPs), estimated by the ELECTRA inverse solution. We first developed a new method for the analysis of EEG data. This method is based on the statistical comparison of the spectral characteristics of the estimated intracranial LFPs of the pre- and post- stimulus onset periods. Each single trial is analyzed independently, without including an averaging step, so that the information carried by high frequencies is preserved. We also propose a new metric, called resemblance, to investigate the relationship between fMRI and the estimated intracranial LFPs. Single-trial analysis and the resemblance metric were applied in an experiment involving separate EEG and fMRI acquisitions during the same passive visual stimulation protocol. This experiment revealed that only a limited set of LFP frequencies shows a spatial correlation with fMRI. This set of frequencies changes across brain areas, such that progression from lower to higher cortical levels of visual processing incorporates at each step new frequencies. In conclusion, in this thesis we show that the estimation and the analysis of the BOLD time course can give an important contribution to better understanding brain functions and brain organization. To fully understand the meaning of changes in BOLD dynamics, we need a better knowledge of the neuro-vascular coupling. To do that, we introduced a new method for evaluating the relationship between EEG and fMRI across frequencies and anatomical regions.

EPFL2006

Multisensory Spatial Mechanisms of the Bodily Self and Social Cognition - A Commentary on Vittorio Gallese and Valentina Cuccio

Christian Pfeiffer

This commentary aims to find the right description of the pre-reflective brain mechanisms underlying our phenomenal experience of being a subject bound to a physical body (bodily self) and basic cognitive, perceptual, and subjective aspects related to interaction with other individuals (social cognition). I will focus on the proposal by Gallese and Cuccio that embodied simulation, in terms of motor resonance, is the primary brain mechanism underlying the pre-reflective aspects of social cognition and the bodily self. I will argue that this proposal is too narrow to serve a unified theory of the neurobiological mechanisms of both target phenomena. I support this criticism with theoretical considerations and empirical evidence suggesting that multisensory spatial processing, which is distinct from but a pre-requisite of motor resonance, substantially contributes to the bodily self and social cognition. My commentary is structured in three sections. The first section addresses social cognition and compares embodied simulation to an alternative account, namely the attention schema theory. According to this theory we pre-reflectively empathize with others by predicting their current state of attention which involves predicting the spatial focus of attention. Thereby we derive a representational model of their state of mind. On this account, spatial coding of attention, rather than motor resonance, is the primary mechanism underlying social cognition. I take this as a theoretical alternative complementing motor resonance mechanisms. The second section focuses on the bodily self. Comparison of the brain net- works of the bodily self and social cognition reveals strong overlap, suggesting that both phenomena depend on shared multisensory and sensorimotor mechanisms. I will review recent empirical data about altered states of the bodily self in terms of self-location and the first-person perspective. These spatial aspects of the bodily self are encoded in brain regions distinct from the brain network of em- bodied simulation. I argue that while motor resonance might contribute to body ownership and agency, it does not account for spatial aspects of the bodily self. Thus, embodied simulation appears to be a necessary but insufficiently “primary” brain mechanism of the bodily self and social cognition. The third section discusses the contributions of the vestibular system, i.e., the sensory system encoding head motion and gravity, to the bodily self and social cognition. Vestibular cortical processing seems relevant to both processes, be- cause it directly encodes the world-centered direction of gravity and allows us to distinguish between motions of the own body and motions of other individuals and the external world. Furthermore, the vestibular cortical network largely overlaps with those neural networks relevant to the bodily self and social cognition. Thus, the vestibular system may play a crucial role in multisensory spatial coding relating the bodily self to other individuals in the external world.

2015

Spatio-temporal coupling of the electric and hemodynamic brain responses in humans

Roberto Martuzzi

EPFL2006

Multisensory Spatial Mechanisms of the Bodily Self and Social Cognition - A Commentary on Vittorio Gallese and Valentina Cuccio

Christian Pfeiffer

2015