Sharpening your senses: mechanisms for multisensory integration
/My baseball coach always told me to pay attention to the crack of the bat because it would make it easier to find the ball from the outfield. He was right! This combined processing of visual and auditory information is called multisensory integration and has been shown to enhance perception.
Lyle Overbay integrates multiple sources of sensory information. Source: Graham Richter, Wikimedia.
Neuroscientists typically think about this integration happening in late stages of neural processing, after information from a single sense has been processed. However, as early as the 1970s, physiologists found that regions of the cortex thought to process single sensory modalities actually responded to information from multiple senses. Primary visual cortex is one such example. This region, located at the very back of the brain, contains neurons that respond to light hitting the eyes by firing action potentials—electrical impulses that make up the language neurons use to speak to each other. While this region is canonically thought to solely process visual information at its early stages, up to forty percent of the neurons in this region can also be driven by sound. Responses to visual stimuli are even enhanced by the presence of a sound (Morrell, 1972). While this result has been lying around for over forty years, the mechanisms by which auditory information arrives in visual cortex so quickly and the identity of the cells that are responsible for this sharpening of spatial acuity remained elusive.
Leena Ibrahim and colleagues made major progress towards understanding this phenomenon in a recent paper in the journal Neuron titled “Cross-Modality Sharpening of Visual Cortical Processing through Layer-1-Mediated Inhibition and Disinhibition”. Ibrahim and colleagues systematically interrogated the roles of different neuron types in this response and identified a subset of neurons that directly connect auditory cortex to visual cortex. They recorded from a specific type of neuron (Layer 2/3 pyramidal cells) in the visual cortex of awake mice as they received visual and auditory stimuli. These neurons are excitatory, meaning that they increase the probability that the neurons they connect to will also fire action potentials. It is well-known that this type of neuron in visual cortex preferentially fires more action potentials when object edges with particular orientations enter their spatial “territory” (e.g neuron A may like to respond to vertical edges in the lower left quadrant of the animal’s vision but neuron B prefers horizontal edges in the upper left quadrant.)
When the mouse simultaneously received visual and auditory stimuli, the cells’ tuning to preferred orientations sharpened. Not only did the response at the preferred orientation of the cell increase, but the response to other orientations decreased. This simultaneous increase to the preferred orientation and decrease to the non-preferred orientations cannot result from simple direct input onto these cells from excitatory cells in the auditory cortex. This simple type of mechanism could increase the overall response rate to any orientation, but it could not simultaneously increase and decrease responses to different orientations.
Using anatomical tracing techniques, the authors found that a subset of neurons in the auditory cortex (Layer 5 pyramidal cells) send some of their axons to specific inhibitory neurons (cells that decrease the probability of their neighbors firing action potentials) in primary visual cortex (Layer 1 VIP- interneurons). These inhibitory neurons in visual cortex responded to sound alone and to simple stimulation of the auditory cortex neurons. Inhibitory neurons responded to the presence of sound even before the excitatory neurons could respond to the visual input, meaning that auditory input primes the visual cortex to better respond to visual stimuli.
Ibrahim and colleagues used state-of-the-art techniques to pinpoint specific circuit mechanisms that allow for multisensory integration and enhancement of perception at the earliest stages of sensory processing. Their experiments allow for several possible interpretations for how specific inhibitory neuron types wire together to accomplish this task. In future work it will be important for them to test these alternatives and directly show which neurons are connected.
