Active learning of novel sound-producing objects: motor reactivation and enhancement of visuo-motor connectivity.

Our experience with the world commonly involves physical interaction with objects enabling us to learn associations between multisensory information perceived during an event and our actions that create an event. The interplay among active interactions during learning and multisensory integration of object properties is not well understood. To better understand how action might enhance multisensory associative recognition, we investigated the interplay among motor and perceptual systems after active learning. Fifteen participants were included in an fMRI study during which they learned visuo-auditory-motor associations between novel objects and the sounds they produce, either through self-generated actions on the objects (active learning) or by observing an experimenter produce the actions (passive learning). Immediately after learning, behavioral and BOLD fMRI measures were collected while perceiving the objects used during unisensory and multisensory training in associative perception and recognition tasks. Active learning was faster and led to more accurate recognition of audiovisual associations than passive learning. Functional ROI analyses showed that in motor, somatosensory, and cerebellar regions there was greater activation during both the perception and recognition of actively learned associations. Finally, functional connectivity between visual- and motor-related processing regions was enhanced during the presentation of actively learned audiovisual associations. Overall, the results of the current study clarify and extend our own previous work [Butler, A. J., James, T. W., & Harman James, K. Enhanced multisensory integration and motor reactivation after active motor learning of audiovisual associations. Journal of Cognitive Neuroscience, 23, 3515-3528, 2011] by providing several novel findings and highlighting the task-based nature of motor reactivation and retrieval after active learning.

Expert individuation of objects increases activation in the fusiform face area of children.

The role of experience in the development of brain mechanisms for face recognition is intensely debated. Experience with subordinate- and individual-level classification of faces is thought, by some, to be foundational in the development of the specialization of face recognition. Studying children with extremely intense interests (EII) provides an opportunity to examine experience-related changes in non-face object recognition in a population where face expertise is not fully developed. Here, two groups of school-aged children -one group with an EII with Pokémon cards and another group of age-matched controls - underwent fMRI while viewing faces, Pokémon characters, Pokémon objects, and Digimon characters. Pokémon objects were non-character Pokémon cards that experts do not typically individuate during game play and trading. Neither experts nor controls had previous experience with Digimon characters. As expected, experts and controls showed equivalent activation in the fusiform face area (FFA) with face stimuli. As predicted by the expertise hypothesis, experts showed greater activation than controls with Pokémon characters, and showed greater activation with Pokémon characters than Pokémon objects. Experts and controls showed equivalent activation with Digimon characters. However, heightened activation with Digimon characters in both groups suggested that there are other strong influences on the activation of the FFA beyond stimulus characteristics, experience, and classification level. By demonstrating the important role of expertise, the findings are inconsistent with a purely face-specific account of FFA function. To our knowledge, this is the first demonstration of the effects of expertise and categorization level on activation in the FFA in a group of typically developing children.

Enhanced multisensory integration and motor reactivation after active motor learning of audiovisual associations.

Everyday experience affords us many opportunities to learn about objects through multiple senses using physical interaction. Previous work has shown that active motor learning of unisensory items enhances memory and leads to the involvement of motor systems during subsequent perception. However, the impact of active motor learning on subsequent perception and recognition of associations among multiple senses has not been investigated. Twenty participants were included in an fMRI study that explored the impact of active motor learning on subsequent processing of unisensory and multisensory stimuli. Participants were exposed to visuo-motor associations between novel objects and novel sounds either through self-generated actions on the objects or by observing an experimenter produce the actions. Immediately after exposure, accuracy, RT, and BOLD fMRI measures were collected with unisensory and multisensory stimuli in associative perception and recognition tasks. Response times during audiovisual associative and unisensory recognition were enhanced by active learning, as was accuracy during audiovisual associative recognition. The difference in motor cortex activation between old and new associations was greater for the active than the passive group. Furthermore, functional connectivity between visual and motor cortices was stronger after active learning than passive learning. Active learning also led to greater activation of the fusiform gyrus during subsequent unisensory visual perception. Finally, brain regions implicated in audiovisual integration (e.g., STS) showed greater multisensory gain after active learning than after passive learning. Overall, the results show that active motor learning modulates the processing of multisensory associations.

Only self-generated actions create sensori-motor systems in the developing brain.

Previous research shows that sensory and motor systems interact during perception, but how these connections among systems are created during development is unknown. The current work exposes young children to novel 'verbs' and objects through either (a) actively exploring the objects or (b) by seeing an experimenter interact with the objects. Results demonstrate that the motor system is recruited during auditory perception only after learning involved self-generated interactions with objects. Action observation itself led to above-baseline activation in one motor region during visual perception, but was still significantly less active than after self-generated action. Therefore, in the developing brain, associations are built upon real-world interactions of body and environment, leading to sensori-motor representations of both objects and words.

Sensori-motor experience leads to changes in visual processing in the developing brain.

Since Broca's studies on language processing, cortical functional specialization has been considered to be integral to efficient neural processing. A fundamental question in cognitive neuroscience concerns the type of learning that is required for functional specialization to develop. To address this issue with respect to the development of neural specialization for letters, we used functional magnetic resonance imaging (fMRI) to compare brain activation patterns in pre-school children before and after different letter-learning conditions: a sensori-motor group practised printing letters during the learning phase, while the control group practised visual recognition. Results demonstrated an overall left-hemisphere bias for processing letters in these pre-literate participants, but, more interestingly, showed enhanced blood oxygen-level-dependent activation in the visual association cortex during letter perception only after sensori-motor (printing) learning. It is concluded that sensori-motor experience augments processing in the visual system of pre-school children. The change of activation in these neural circuits provides important evidence that 'learning-by-doing' can lay the foundation for, and potentially strengthen, the neural systems used for visual letter recognition.

Auditory verb perception recruits motor systems in the developing brain: an fMRI investigation.

This study investigated neural activation patterns during verb processing in children, using fMRI (functional Magnetic Resonance Imaging). Preschool children (aged 4-6) passively listened to lists of verbs and adjectives while neural activation was measured. Findings indicated that verbs were processed differently than adjectives, as the verbs recruited motor systems in the frontal cortex during auditory perception, but the adjectives did not. Further evidence suggested that different types of verbs activated different regions in the motor cortex. The results demonstrate that the motor system is recruited during verb perception in the developing brain, reflecting the embodied nature of language learning and processing.

Crossmodal enhancement in the LOC for visuohaptic object recognition over development.

Research has provided strong evidence of multisensory convergence of visual and haptic information within the visual cortex. These studies implement crossmodal matching paradigms to examine how systems use information from different sensory modalities for object recognition. Developmentally, behavioral evidence of visuohaptic crossmodal processing has suggested that communication within sensory systems develops earlier than across systems; nonetheless, it is unknown how the neural mechanisms driving these behavioral effects develop. To address this gap in knowledge, BOLD functional Magnetic Resonance Imaging (fMRI) was measured during delayed match-to-sample tasks that examined intramodal (visual-to-visual, haptic-to-haptic) and crossmodal (visual-to-haptic, haptic-to-visual) novel object recognition in children aged 7-8.5 years and adults. Tasks were further divided into sample encoding and test matching phases to dissociate the relative contributions of each. Results of crossmodal and intramodal object recognition revealed the network of known visuohaptic multisensory substrates, including the lateral occipital complex (LOC) and the intraparietal sulcus (IPS). Critically, both adults and children showed crossmodal enhancement within the LOC, suggesting a sensitivity to changes in sensory modality during recognition. These groups showed similar regions of activation, although children generally exhibited more widespread activity during sample encoding and weaker BOLD signal change during test matching than adults. Results further provided evidence of a bilateral region in the occipitotemporal cortex that was haptic-preferring in both age groups. This region abutted the bimodal LOtv, and was consistent with a medial to lateral organization that transitioned from a visual to haptic bias within the LOC. These findings converge with existing evidence of visuohaptic processing in the LOC in adults, and extend our knowledge of crossmodal processing in adults and children.

SELF-GENERATED ACTIONS DURING LEARNING OBJECTS AND SOUNDS CREATE SENSORI-MOTOR SYSTEMS IN THE DEVELOPING BRAIN.

Previous research shows that sensory and motor systems interact during verb perception, and that these interactions are formed through self-generated actions that refer to verb labels during development. Here we expand on these findings by investigating whether self-generated actions lead to sensori-motor interaction during sound perception and visual perception. The current research exposes young children to novel sounds that are produced by object movement through either a) actively exploring the objects and producing the sounds or b) by seeing and hearing an experimenter interact with the objects. Results demonstrate that the motor system was recruited during auditory perception only after learning involved self-generated interactions with objects. Interestingly, visual association regions were also active during both sound perception and visual perception after active exploratory learning, but not after passive observation. Therefore, in the developing brain, associations are built upon real-world interactions of body and environment, leading to sensori-motor representations of both objects and sounds.

Multisensory perception of action in posterior temporal and parietal cortices.

Environmental events produce many sensory cues for identifying the action that evoked the event, the agent that performed the action, and the object targeted by the action. The cues for identifying environmental events are usually distributed across multiple sensory systems. Thus, to understand how environmental events are recognized requires an understanding of the fundamental cognitive and neural processes involved in multisensory object and action recognition. Here, we investigated the neural substrates involved in auditory and visual recognition of object-directed actions. Consistent with previous work on visual recognition of isolated objects, visual recognition of actions, and recognition of environmental sounds, we found evidence for multisensory audiovisual event-selective activation bilaterally at the junction of the posterior middle temporal gyrus and the lateral occipital cortex, the left superior temporal sulcus, and bilaterally in the intraparietal sulcus. The results suggest that recognition of events through convergence of visual and auditory cues is accomplished through a network of brain regions that was previously implicated only in visual recognition of action.

Shape from sound: evidence for a shape operator in the lateral occipital cortex.

A recent view of cortical functional specialization suggests that the primary organizing principle of the cortex is based on task requirements, rather than sensory modality. Consistent with this view, recent evidence suggests that a region of the lateral occipitotemporal cortex (LO) may process object shape information regardless of the modality of sensory input. There is considerable evidence that area LO is involved in processing visual and haptic shape information. However, sound can also carry acoustic cues to an object's shape, for example, when a sound is produced by an object's impact with a surface. Thus, the current study used auditory stimuli that were created from recordings of objects impacting a hard surface to test the hypothesis that area LO is also involved in auditory shape processing. The objects were of two shapes, rods and balls, and of two materials, metal and wood. Subjects were required to categorize the impact sounds in one of three tasks, (1) by the shape of the object while ignoring material, (2) by the material of the object while ignoring shape, or (3) by using all the information available. Area LO was more strongly recruited when subjects discriminated impact sounds based on the shape of the object that made them, compared to when subjects discriminated those same sounds based on material. The current findings suggest that activation in area LO is shape selective regardless of sensory input modality, and are consistent with an emerging theory of perceptual functional specialization of the brain that is task-based rather than sensory modality-based.