The importance of multisensory-motor learning on subsequent visual recognition.

Speed of visual object recognition is facilitated after active manual exploration of objects relative to passive visual processing alone. Manual exploration allows viewers to select important information about object structure that may facilitate recognition. Viewpoints where the objects' axis of elongation is perpendicular or parallel to the line of sight are selected more during exploration, recognized faster than other viewpoints, and afford the most information about structure when object movement is controlled by the viewer. Prior work used virtual object exploration in active and passive viewing conditions, limiting multisensory structural object information. Adding multisensory information to encoding may change accuracy of overall recognition, viewpoint selection, and viewpoint recognition. We tested whether the known active advantage for object recognition would change when real objects were studied, affording visual and haptic information. Participants interacted with 3D novel objects during manual exploration or passive viewing of another's object interactions. Object recognition was tested using several viewpoints of rendered objects. We found that manually explored objects were recognized more accurately than objects studied through passive exploration and that recognition of viewpoints differed from previous work.

Protracted Neural Development of Dorsal Motor Systems During Handwriting and the Relation to Early Literacy Skills.

Handwriting is a complex visual-motor skill that affects early reading development. A large body of work has demonstrated that handwriting is supported by a widespread neural system comprising ventral-temporal, parietal, and frontal motor regions in adults. Recent work has demonstrated that this neural system is largely established by 8 years of age, suggesting that the development of this system occurs in young children who are still learning to read and write. We made use of a novel MRI-compatible writing tablet that allowed us to measure brain activation in 5-8-year-old children during handwriting. We compared activation during handwriting in children and adults to provide information concerning the developmental trajectory of the neural system that supports handwriting. We found that parietal and frontal motor involvement during handwriting in children is different from adults, suggesting that the neural system that supports handwriting changes over the course of development. Furthermore, we found that parietal and frontal motor activation correlated with a literacy composite score in our child sample, suggesting that the individual differences in the dorsal response during handwriting are related to individual differences in emerging literacy skills. Our results suggest that components of the widespread neural system supporting handwriting develop at different rates and provide insight into the mechanisms underlying the contributions of handwriting to early literacy development.

The Effects of Frequency, Variability, and Co-occurrence on Category Formation in Neural Systems.

Objects are grouped into categories through a complex combination of statistical and structural regularities. We sought to better understand the neural responses to the structural features of object categories that result from implicit learning. Adult participants were exposed to 32 object categories that contained three structural properties: frequency, variability, and co-occurrences, during an implicit learning task. After this exposure, participants completed a recognition task and were then presented with blocks of learned object categories during fMRI sessions. Analyses were performed by extracting data from ROIs placed throughout the fusiform gyri and lateral occipital cortex and comparing the effects of the different structural properties throughout the ROIs. Behaviorally, we found that symbol category recognition was supported by frequency, but not variability. Neurally, we found that sensitivity to object categories was greater in the right hemisphere and increased as ROIs were moved posteriorly. Frequency and variability altered the brain activation while processing object categories, although the presence of learned co-occurrences did not. Moreover, variability and co-occurrence interacted as a function of ROI, with the posterior fusiform gyrus being most sensitive to this relationship. This result suggests that variability may guide the learner to relevant co-occurrences and this is supported by the posterior ventral temporal cortex. Broadly, our results suggest that the internal features of the categories themselves are key factors in the category learning system.

Category structure guides the formation of neural representations.

Perceptual variability is often viewed as having multiple benefits in object learning and categorization. Despite the abundant results demonstrating benefits such as increased transfer of knowledge, the neural mechanisms underlying variability as well as the developmental trajectories of how variability precipitates changes to category boundaries are unknown. By manipulating an individual's exposure to variability of novel, metrically organized categories during an fMRI-adaptation paradigm, we were able to assess the functional differences between similarity and variability in category learning and generalization across two time-points in development: adulthood (n = 14) and late childhood (n = 13). During this study, participants were repeatedly exposed to category members from different distributions. After a period of adaptation, a deviant stimulus that differed from the expected distribution was then presented. This deviant differed in either an invariant dimension (a feature that remained consistent throughout presentation was altered) or a similarity dimension (a feature that changed throughout exposure was changed in a new dimension). Our results can be summarized in three main findings: (1) Variability during exposure recruited the right fusiform gyrus to a greater extent than tight exposure. (2) Deviant items were generalized based on the exemplar distributions during exposure, although children only generalized items if provided variable exposure. (3) Variability influenced release to a greater extent in children than adults. These results are discussed in relation to the variability and category learning literature more broadly.

"Neural differences in expert guitarists during over-learned non-standard visuomotor mapping of abstract versus concrete information".

Using visual information to perform actions is a fundamental aspect of human behavior. Musicians commonly translate visual information into action using both concrete and abstract visual information. We exposed expert guitarists to four types of familiar visual depictions of action instruction including musical notation (very abstract), tablature (abstract), chord diagrams (more concrete), and actual pictures of guitars chords being formed (very concrete). These were shown during fMRI scanning as the guitarists formed the appropriate chords (as visually depicted) on a magnet safe guitar fret board with strings, or where they simply viewed the visual stimuli without an action. Whole brain contrasts revealed that the right precuneus was more active for abstract instruction while an occipito-insular circuit was more active for concrete instruction. The current findings highlight that the degree of over-learned visual abstraction is an important factor modulating visual-motor processing.

Constraining Stroke Order During Manual Symbol Learning Hinders Subsequent Recognition in Children Under 4 1/2 Years.

In the age of technology, writing by hand has become less common than texting and keyboarding. Learning letters by hand, however, has been shown to have profound developmental importance. One aspect of writing by hand that has been understudied is the effect of learning symbols stroke-by-stroke, a dynamic action that does not occur with keyboarding. We trained children to draw novel symbols in either an instructed stroke order or in a self-directed stroke order and tested: (1) whether learning novel symbols in a self-directed stroke order benefits subsequent recognition more than learning in a specified stroke order, (2) whether seeing novel symbols unfold in the stroke order that was taught would aid in recognition, and (3) whether any effects are age-dependent. Our results demonstrate that producing a symbol with a self-directed stroke order provides more benefit to symbol recognition than instructed stroke orders in 4.0-4.5-year-old children but not in 4.5-5.0-year-old children. We found, further, that the observed recognition benefits were not affected by seeing the symbol unfold in the same stroke order it was learned during testing, suggesting that the learning was not reliant upon the exact stroke order experienced during learning. These results stress the importance of allowing children to produce symbols in a self-directed manner and, by extension, that constraining how a child learns to write can adversely affect subsequent recognition.

Ecological validity of experimental set-up affects parietal involvement during letter production.

Studies of symbol production using fMRI often use techniques that introduce an artificial pairing between motor production and visual perception. These techniques allow participants to see their own output by recording their pen trajectories using a touchscreen-only tablet and displaying these productions on a mirror placed above their head. We recently developed an MR-safe writing tablet with video display that allows participants to see their own hand and their own productions while producing symbols in real time on the surface where they are producing them-allowing for more ecologically valid fMRI studies of production. We conducted a study to determine whether the participation of posterior parietal cortex during symbol production was affected by the pairing of motor production and visual feedback associated with the two types of tablets. We performed ROI analyses in intraparietal sulcus while adult participants produced letters to dictation using either a touchscreen-only tablet (no visual guidance of the hand) (n = 14) or using a touchscreen-and-video-display tablet (visual guidance of the hand) (n = 14). We found that left posterior intraparietal sulcus was more active during production with the touchscreen-only tablet than during production with the touchscreen-and-video-display tablet. These results suggest that posterior parietal involvement during production tasks is associated with the somewhat artificial visual-motor pairing that is introduced by the techniques used in some studies of symbol production.

Visual experiences during letter production contribute to the development of the neural systems supporting letter perception.

Letter production through handwriting creates visual experiences that may be important for the development of visual letter perception. We sought to better understand the neural responses to different visual percepts created during handwriting at different levels of experience. Three groups of participants, younger children, older children, and adults, ranging in age from 4.5 to 22 years old, were presented with dynamic and static presentations of their own handwritten letters, static presentations of an age-matched control's handwritten letters, and typeface letters during fMRI. First, data from each group were analyzed through a series of contrasts designed to highlight neural systems that were most sensitive to each visual experience in each age group. We found that younger children recruited ventral-temporal cortex during perception and this response was associated with the variability present in handwritten forms. Older children and adults also recruited ventral-temporal cortex; this response, however, was significant for typed letter forms but not variability. The adult response to typed letters was more distributed than in the children, including ventral-temporal, parietal, and frontal motor cortices. The adult response was also significant for one's own handwritten letters in left parietal cortex. Second, we compared responses among age groups. Compared to older children, younger children demonstrated a greater fusiform response associated with handwritten form variability. When compared to adults, younger children demonstrated a greater response to this variability in left parietal cortex. Our results suggest that the visual perception of the variability present in handwritten forms that occurs during handwriting may contribute to developmental changes in the neural systems that support letter perception.

Learning math by hand: The neural effects of gesture-based instruction in 8-year-old children.

Producing gesture can be a powerful tool for facilitating learning. This effect has been replicated across a variety of academic domains, including algebra, chemistry, geometry, and word learning. Yet the mechanisms underlying the effect are poorly understood. Here we address this gap using functional magnetic resonance imaging (fMRI). We examine the neural correlates underlying how children solve mathematical equivalence problems learned with the help of either a speech + gesture strategy, or a speech-alone strategy. Children who learned through a speech + gesture were more likely to recruit motor regions when subsequently solving problems during a scan than children who learned through speech alone. This suggests that gesture promotes learning, at least in part, because it is a type of action. In an exploratory analysis, we also found that children who learned through speech + gesture showed subthreshold activation in regions outside the typical action-learning network, corroborating behavioral findings suggesting that the mechanisms supporting learning through gesture and action are not identical. This study is one of the first to explore the neural mechanisms of learning through gesture.

An Analysis of the Brain Systems Involved with Producing Letters by Hand.

Complex visual-motor behaviors dominate human-environment interactions. Letter production, writing individual letters by hand, is an example of a complex visual-motor behavior composed of numerous behavioral components, including the required motor movements and the percepts that those motor movements create. By manipulating and isolating components of letter production, we provide experimental evidence that this complex visual-motor behavior is supported by a widespread neural system that is composed of smaller subsystems related to different sensorimotor components. Adult participants hand-printed letters with and without "ink" on an MR-safe digital writing tablet, perceived static and dynamic representations of their own handwritten letters, and perceived typeface letters during fMRI scanning. Our results can be summarized by three main findings: (1) Frontoparietal systems were associated with the motor component of letter production, whereas temporo-parietal systems were more associated with the visual component. (2) The more anterior regions of the left intraparietal sulcus were more associated with the motor component, whereas the more posterior regions were more associated with the visual component, with an area of visual-motor overlap in the posterior intraparietal sulcus. (3) The left posterior intraparietal sulcus and right fusiform gyrus responded similarly to both visual and motor components, and both regions also responded more during the perception of one's own handwritten letters compared with perceiving typed letters. These findings suggest that the neural systems recruited during complex visual-motor behaviors are composed of a set of interrelated sensorimotor subsystems that support the full behavior in different ways and, furthermore, that some of these subsystems can be rerecruited during passive perception in the absence of the full visual-motor behavior.

Functional fixedness in tool use: Learning modality, limitations and individual differences.

Functional fixedness is a cognitive bias that describes how previous knowledge of a tool's function can negatively impact the use of this tool in novel contexts. As such, functional fixedness disturbs the use of tools during mechanical problem solving. Little is known about whether this bias emerges from different experiences with tools, whether it occurs regardless of problem difficulty, or whether there are protective factors against it. To resolve the first issue, we created five experimental groups: Reading (R), Video (V), Manual (M), No Functional Fixedness (NFF), and No Training (NT). The R group learned to use tools by reading a description of their use, the V group by watching an instructional video, and the M group through direct instruction and active manipulation of the tools. To resolve the remaining two issues, we created mechanical puzzles of distinct difficulty and used tests of intuitive physics, fine motor skills, and creativity. Results showed that misleading functional knowledge is at the core of functional fixedness, and that this bias generates cognitive impasses in simple puzzles, but it does not play a role in higher difficulty problems. Additionally, intuitive physics and motor skills were protective factors against its emergence, but creativity did not influence it. Although functional fixedness leads to inaccurate problem solving, our results suggest that its effects are more limited than previously assumed.

The MRItab: A MR-compatible touchscreen with video-display.

BACKGROUND: A touchscreen interface permits rich user interactions for research in many fields, but is rarely found within a Magnetic Resonance Imaging (MRI) environment due to difficulties adapting conventional technologies to the strong electromagnetic fields. Conventional MR-compatible video display technology uses either large-screen displays that are placed outside of the bore of the MRI itself, or projectors located beyond the participant's reach, making touch interfaces impossible. NEW METHOD: Here, we describe the MR-compatibility of the 'MRItab' in terms of MR safety and image quality. The MRItab adapts inexpensive off-the-shelf components with special signal-driver circuitry and shielding to bring the touchscreen interface into the MR environment, without adversely affecting MRI image quality, thereby making touch interfaces possible. RESULTS: Our testing demonstrated that the functioning of the MRItab was not affected by the functioning of the MRI scanner and that the MRItab did not adversely affect the image data acquired. Participants were able to interact naturally with the MRItab during MRI scanning. COMPARISON WITH OTHER METHOD (S): The MRItab is the first MR-compatible touchscreen device with video-display screen capabilities designed for use in the MRI environment. This interactive digital device is the first to allow participants to see their hands directly as they interact with a touch-sensitive display screen, resulting in high ecological validity. CONCLUSIONS: The MRItab provides a methodological advantage for research in many fields, given the realistic human-computer interaction it supports.

Gesture for generalization: gesture facilitates flexible learning of words for actions on objects.

Verb learning is difficult for children (Gentner, ), partially because children have a bias to associate a novel verb not only with the action it represents, but also with the object on which it is learned (Kersten & Smith, ). Here we investigate how well 4- and 5-year-old children (N = 48) generalize novel verbs for actions on objects after doing or seeing the action (e.g., twisting a knob on an object) or after doing or seeing a gesture for the action (e.g., twisting in the air near an object). We find not only that children generalize more effectively through gesture experience, but also that this ability to generalize persists after a 24-hour delay.

Dorsal stream function in the young child: an fMRI investigation of visually guided action.

Visually guided action is a ubiquitous component of human behavior, but the neural substrates that support the development of this behavior are unknown. Here we take an initial step in documenting visual-motor system development in the young (4- to 7-year-old) child. Through functional MRI and by using a new technique to measure the mechanisms underlying real-time visually guided action in the MRI environment, we demonstrate that children rely primarily on the IPS and cerebellum for this complex behavior. This pattern is consistent across three different visually guided actions, suggesting generalizability of these neural substrates across such tasks. However, minor differences in neural processing across tasks were also demonstrated. Overall, results are interpreted as demonstrating that the functions of the dorsal stream can be viewed as fairly mature in the young child. These results provide a benchmark for future studies that aim to understand the development of the neural circuitry for visually guided action.

A new statistical model for analyzing rating scale data pertaining to word meaning.

The concrete-abstract categorization scheme has guided several research programs. A popular way to classify words into one of these categories is to calculate a word's mean value in a Concreteness or Imageability rating scale. However, this procedure has several limitations. For instance, results can be highly distorted by outliers, ascribe differences among words when none may exist, and neglect rating trends in participants. We suggest using an alternative procedure to analyze rating scale data called median polish analysis (MPA). MPA is tolerant to outliers and accounts for information in multiple dimensions, including trends among participants. MPA performance can be readily evaluated using an effect size measure called analog R 2 and be integrated with bootstrap 95% confidence intervals, which can prevent assigning inexistent differences among words. To compare these analysis procedures, we asked 80 participants to rate a set of nouns and verbs using four different rating scales: Action, Concreteness, Imageability, and Multisensory. We analyzed the data using both two-way and three-way MPA models. We also calculated 95% CIs for the two-way models. Categorizing words with the Action scale revealed a continuum of word meaning for both nouns and verbs. The remaining scales produced dichotomous or stratified results for nouns, and continuous results for verbs. While the sample mean analysis generated continua irrespective of the rating scale, MPA differentiated among dichotomies and continua. We conclude that MPA allowed us to better classify words by discarding outliers, focusing on main trends, and considering the differences in rating criteria among participants.

Handwriting generates variable visual output to facilitate symbol learning.

Recent research has demonstrated that handwriting practice facilitates letter categorization in young children. The present experiments investigated why handwriting practice facilitates visual categorization by comparing 2 hypotheses: that handwriting exerts its facilitative effect because of the visual-motor production of forms, resulting in a direct link between motor and perceptual systems, or because handwriting produces variable visual instances of a named category in the environment that then changes neural systems. We addressed these issues by measuring performance of 5-year-old children on a categorization task involving novel, Greek symbols across 6 different types of learning conditions: 3 involving visual-motor practice (copying typed symbols independently, tracing typed symbols, tracing handwritten symbols) and 3 involving visual-auditory practice (seeing and saying typed symbols of a single typed font, of variable typed fonts, and of handwritten examples). We could therefore compare visual-motor production with visual perception both of variable and similar forms. Comparisons across the 6 conditions (N = 72) demonstrated that all conditions that involved studying highly variable instances of a symbol facilitated symbol categorization relative to conditions where similar instances of a symbol were learned, regardless of visual-motor production. Therefore, learning perceptually variable instances of a category enhanced performance, suggesting that handwriting facilitates symbol understanding by virtue of its environmental output: supporting the notion of developmental change though brain-body-environment interactions. (PsycINFO Database Record

Neural substrates of sensorimotor processes: letter writing and letter perception.

Writing and perceiving letters are thought to share similar neural substrates; however, what constitutes a neural representation for letters is currently debated. One hypothesis is that letter representation develops from sensorimotor experience resulting in an integrated set of modality-specific regions, whereas an alternative account suggests that letter representations may be abstract, independent of modality. Studies reviewed suggest that letter representation consists of a network of modality-responsive brain regions that may include an abstract component.

Crossmodal enhancement in the LOC for visuohaptic object recognition over development.

Research has provided strong evidence of multisensory convergence of visual and haptic information within the visual cortex. These studies implement crossmodal matching paradigms to examine how systems use information from different sensory modalities for object recognition. Developmentally, behavioral evidence of visuohaptic crossmodal processing has suggested that communication within sensory systems develops earlier than across systems; nonetheless, it is unknown how the neural mechanisms driving these behavioral effects develop. To address this gap in knowledge, BOLD functional Magnetic Resonance Imaging (fMRI) was measured during delayed match-to-sample tasks that examined intramodal (visual-to-visual, haptic-to-haptic) and crossmodal (visual-to-haptic, haptic-to-visual) novel object recognition in children aged 7-8.5 years and adults. Tasks were further divided into sample encoding and test matching phases to dissociate the relative contributions of each. Results of crossmodal and intramodal object recognition revealed the network of known visuohaptic multisensory substrates, including the lateral occipital complex (LOC) and the intraparietal sulcus (IPS). Critically, both adults and children showed crossmodal enhancement within the LOC, suggesting a sensitivity to changes in sensory modality during recognition. These groups showed similar regions of activation, although children generally exhibited more widespread activity during sample encoding and weaker BOLD signal change during test matching than adults. Results further provided evidence of a bilateral region in the occipitotemporal cortex that was haptic-preferring in both age groups. This region abutted the bimodal LOtv, and was consistent with a medial to lateral organization that transitioned from a visual to haptic bias within the LOC. These findings converge with existing evidence of visuohaptic processing in the LOC in adults, and extend our knowledge of crossmodal processing in adults and children.

Effects of learning with gesture on children's understanding of a new language concept.

Asking children to gesture while being taught a concept facilitates their learning. Here, we investigated whether children benefitted equally from producing gestures that reflected speech (speech-gesture matches) versus gestures that complemented speech (speech-gesture mismatches), when learning the concept of palindromes. As in previous studies, we compared the utility of each gesture strategy to a speech alone strategy. Because our task was heavily based on language ability, we also considered children's phonological competency as a predictor of success at posttest. Across conditions, children who had low phonological competence were equally likely to perform well at posttest. However, gesture use was predictive of learning for children with high phonological competence: Those who produced either gesture strategy during training were more likely to learn than children who used a speech alone strategy. These results suggest that educators should be encouraged to use either speech-gesture match or mismatch strategies to aid learners, but that gesture may be especially beneficial to children who possess basic skills related to the new concept, in this case, phonological competency. Results also suggest that there are differences between the cognitive effects of naturally produced speech-gesture matches and mismatches, and those that are scripted and taught to children.

Social decisions affect neural activity to perceived dynamic gaze.

Gaze direction, a cue of both social and spatial attention, is known to modulate early neural responses to faces e.g. N170. However, findings in the literature have been inconsistent, likely reflecting differences in stimulus characteristics and task requirements. Here, we investigated the effect of task on neural responses to dynamic gaze changes: away and toward transitions (resulting or not in eye contact). Subjects performed, in random order, social (away/toward them) and non-social (left/right) judgment tasks on these stimuli. Overall, in the non-social task, results showed a larger N170 to gaze aversion than gaze motion toward the observer. In the social task, however, this difference was no longer present in the right hemisphere, likely reflecting an enhanced N170 to gaze motion toward the observer. Our behavioral and event-related potential data indicate that performing social judgments enhances saliency of gaze motion toward the observer, even those that did not result in gaze contact. These data and that of previous studies suggest two modes of processing visual information: a 'default mode' that may focus on spatial information; a 'socially aware mode' that might be activated when subjects are required to make social judgments. The exact mechanism that allows switching from one mode to the other remains to be clarified.