White matter plasticity following cataract surgery in congenitally blind patients.

The visual system develops abnormally when visual input is absent or degraded during a critical period early in life. Restoration of the visual input later in life is generally thought to have limited benefit because the visual system will lack sufficient plasticity to adapt to and utilize the information from the eyes. Recent evidence, however, shows that congenitally blind adolescents can recover both low-level and higher-level visual function following surgery. In this study, we assessed behavioral performance in both a visual acuity and a face perception task alongside longitudinal structural white matter changes in terms of fractional anisotropy (FA) and mean diffusivity (MD). We studied congenitally blind patients with dense bilateral cataracts, who received cataract surgery at different stages of adolescence. Our goal was to differentiate between age- and surgery-related changes in both behavioral performance and structural measures to identify neural correlates which might contribute to recovery of visual function. We observed surgery-related long-term increases of structural integrity of late-visual pathways connecting the occipital regions with ipsilateral fronto-parieto-temporal regions or homotopic contralateral areas. Comparison to a group of age-matched healthy participants indicated that these improvements went beyond the expected changes in FA and MD based on maturation alone. Finally, we found that the extent of behavioral improvement in face perception was mediated by changes in structural integrity in late visual pathways. Our results suggest that sufficient plasticity remains in adolescence to partially overcome abnormal visual development and help localize the sites of neural change underlying sight recovery.

Robustness to Transformations Across Categories: Is Robustness Driven by Invariant Neural Representations?

Deep convolutional neural networks (DCNNs) have demonstrated impressive robustness to recognize objects under transformations (e.g., blur or noise) when these transformations are included in the training set. A hypothesis to explain such robustness is that DCNNs develop invariant neural representations that remain unaltered when the image is transformed. However, to what extent this hypothesis holds true is an outstanding question, as robustness to transformations could be achieved with properties different from invariance; for example, parts of the network could be specialized to recognize either transformed or nontransformed images. This article investigates the conditions under which invariant neural representations emerge by leveraging that they facilitate robustness to transformations beyond the training distribution. Concretely, we analyze a training paradigm in which only some object categories are seen transformed during training and evaluate whether the DCNN is robust to transformations across categories not seen transformed. Our results with state-of-the-art DCNNs indicate that invariant neural representations do not always drive robustness to transformations, as networks show robustness for categories seen transformed during training even in the absence of invariant neural representations. Invariance emerges only as the number of transformed categories in the training set is increased. This phenomenon is much more prominent with local transformations such as blurring and high-pass filtering than geometric transformations such as rotation and thinning, which entail changes in the spatial arrangement of the object. Our results contribute to a better understanding of invariant neural representations in deep learning and the conditions under which it spontaneously emerges.

Prenatal auditory experience and its sequelae.

Towards the end of the second trimester of gestation, a human fetus is able to register environmental sounds. This in utero auditory experience is characterized by comprising strongly low-pass-filtered versions of sounds from the external world. Here, we present computational tests of the hypothesis that this early exposure to severely degraded auditory inputs serves an adaptive purpose-it may induce the neural development of extended temporal integration. Such integration can facilitate the detection of information carried by low-frequency variations in the auditory signal, including emotional or other prosodic content. To test this prediction, we characterized the impact of several training regimens, biomimetic and otherwise, on a computational model system trained and tested on the task of emotion recognition. We find that training with an auditory trajectory recapitulating that of a neurotypical infant in the pre-to-postnatal period results in temporally extended receptive field structures and yields the best subsequent accuracy and generalization performance on the task of emotion recognition. This strongly suggests that the progression from low-pass-filtered to full-frequency inputs is likely to be an adaptive feature of our development, conferring significant benefits to later auditory processing abilities relying on temporally extended analyses. Additionally, this finding can help explain some of the auditory impairments associated with preterm births, suggests guidelines for the design of auditory environments in neonatal care units, and points to enhanced training procedures for computational models.

Vulnerability of facial attractiveness perception to early and multi-year visual deprivation.

Judgments of facial attractiveness invariably accompany our perception of faces. Even neonates appear to be capable of making such judgments in a manner consistent with adults. This suggests that the processes supporting facial attractiveness require little, if any, visual experience to manifest. Here we investigate the resilience of these processes to several years of early-onset visual deprivation. Specifically, we study whether congenitally blind children treated several years after birth possess the ability to rate facial attractiveness in a manner congruent to normally sighted individuals. The data reveal significant individual variability in the way each newly sighted child perceives attractiveness. This is in marked contrast to data from normally sighted controls who exhibit strong across-subject agreement in facial attractiveness ratings. This variability may be attributable, in part, to atypical facial encoding strategies used by the newly sighted children. Overall, our results suggest that the development of facial attractiveness perception is likely to be vulnerable to early visual deprivation, pointing to the existence of a possible sensitive period early in the developmental trajectory.

Development of Visual Memory Capacity Following Early-Onset and Extended Blindness.

It is unknown whether visual memory capacity can develop if onset of pattern vision is delayed for several years following birth. We had an opportunity to address this question through our work with an unusual population of 12 congenitally blind individuals ranging in age from 8 to 22 years. After providing them with sight surgery, we longitudinally evaluated their visual memory capacity using an image-memorization task. Our findings revealed poor visual memory capacity soon after surgery but significant improvement in subsequent months. Although there may be limits to this improvement, performance 1 year after surgery was found to be comparable with that of control participants with matched visual acuity. These findings provide evidence for plasticity of visual memory mechanisms into late childhood but do not rule out vulnerability to early deprivation. Our computational simulations suggest that a potential mechanism to account for changes in memory performance may be progressive representational elaboration in image encoding.

Potential downside of high initial visual acuity.

Children who are treated for congenital cataracts later exhibit impairments in configural face analysis. This has been explained in terms of a critical period for the acquisition of normal face processing. Here, we consider a more parsimonious account according to which deficits in configural analysis result from the abnormally high initial retinal acuity that children treated for cataracts experience, relative to typical newborns. According to this proposal, the initial period of low retinal acuity characteristic of normal visual development induces extended spatial processing in the cortex that is important for configural face judgments. As a computational test of this hypothesis, we examined the effects of training with high-resolution or blurred images, and staged combinations, on the receptive fields and performance of a convolutional neural network. The results show that commencing training with blurred images creates receptive fields that integrate information across larger image areas and leads to improved performance and better generalization across a range of resolutions. These findings offer an explanation for the observed face recognition impairments after late treatment of congenital blindness, suggest an adaptive function for the acuity trajectory in normal development, and provide a scheme for improving the performance of computational face recognition systems.

Emergence of categorical face perception after extended early-onset blindness.

It is unknown whether the ability to visually distinguish between faces and nonfaces is subject to a critical period during development. Would a congenitally blind child who gains sight several years after birth be able to acquire this skill? This question has remained unanswered because of the rarity of cases of late sight onset. We had the opportunity to work with five early-blind individuals who gained sight late in childhood after treatment for dense bilateral cataracts. We tested their ability to categorize patterns as faces, using natural images that spanned a spectrum of face semblance. The results show that newly sighted individuals are unable to distinguish between faces and nonfaces immediately after sight onset, but improve markedly in the following months. These results demonstrate preserved plasticity for acquiring face/nonface categorization ability even late in life, and set the stage for investigating the informational and neural basis of this skill acquisition.

Recognizing Facial Slivers.

We report here an unexpectedly robust ability of healthy human individuals ( n = 40) to recognize extremely distorted needle-like facial images, challenging the well-entrenched notion that veridical spatial configuration is necessary for extracting facial identity. In face identification tasks of parametrically compressed internal and external features, we found that the sum of performances on each cue falls significantly short of performance on full faces, despite the equal visual information available from both measures (with full faces essentially being a superposition of internal and external features). We hypothesize that this large deficit stems from the use of positional information about how the internal features are positioned relative to the external features. To test this, we systematically changed the relations between internal and external features and found preferential encoding of vertical but not horizontal spatial relationships in facial representations ( n = 20). Finally, we employ magnetoencephalography imaging ( n = 20) to demonstrate a close mapping between the behavioral psychometric curve and the amplitude of the M250 face familiarity, but not M170 face-sensitive evoked response field component, providing evidence that the M250 can be modulated by faces that are perceptually identifiable, irrespective of extreme distortions to the face's veridical configuration. We theorize that the tolerance to compressive distortions has evolved from the need to recognize faces across varying viewpoints. Our findings help clarify the important, but poorly defined, concept of facial configuration and also enable an association between behavioral performance and previously reported neural correlates of face perception.

Development of pattern vision following early and extended blindness.

Visual plasticity peaks during early critical periods of normal visual development. Studies in animals and humans provide converging evidence that gains in visual function are minimal and deficits are most severe when visual deprivation persists beyond the critical period. Here we demonstrate visual development in a unique sample of patients who experienced extended early-onset blindness (beginning before 1 y of age and lasting 8-17 y) before removal of bilateral cataracts. These patients show surprising improvements in contrast sensitivity, an assay of basic spatial vision. We find that contrast sensitivity development is independent of the age of sight onset and that individual rates of improvement can exceed those exhibited by normally developing infants. These results reveal that the visual system can retain considerable plasticity, even after early blindness that extends beyond critical periods.

Autism as a disorder of prediction.

A rich collection of empirical findings accumulated over the past three decades attests to the diversity of traits that constitute the autism phenotypes. It is unclear whether subsets of these traits share any underlying causality. This lack of a cohesive conceptualization of the disorder has complicated the search for broadly effective therapies, diagnostic markers, and neural/genetic correlates. In this paper, we describe how theoretical considerations and a review of empirical data lead to the hypothesis that some salient aspects of the autism phenotype may be manifestations of an underlying impairment in predictive abilities. With compromised prediction skills, an individual with autism inhabits a seemingly "magical" world wherein events occur unexpectedly and without cause. Immersion in such a capricious environment can prove overwhelming and compromise one's ability to effectively interact with it. If validated, this hypothesis has the potential of providing unifying insights into multiple aspects of autism, with attendant benefits for improving diagnosis and therapy.

Imaging prior information in the brain.

In making sense of the visual world, the brain's processing is driven by two factors: the physical information provided by the eyes ("bottom-up" data) and the expectancies driven by past experience ("top-down" influences). We use degraded stimuli to tease apart the effects of bottom-up and top-down processes because they are easier to recognize with prior knowledge of undegraded images. Using machine learning algorithms, we quantify the amount of information that brain regions contain about stimuli as the subject learns the coherent images. Our results show that several distinct regions, including high-level visual areas and the retinotopic cortex, contain more information about degraded stimuli with prior knowledge. Critically, these regions are separate from those that exhibit classical priming, indicating that top-down influences are more than feature-based attention. Together, our results show how the neural processing of complex imagery is rapidly influenced by fleeting experiences.

Motion sequence analysis in the presence of figural cues.

The perception of 3D structure in dynamic sequences is believed to be subserved primarily through the use of motion cues. However, real-world sequences contain many figural shape cues besides the dynamic ones. We hypothesize that if figural cues are perceptually significant during sequence analysis, then inconsistencies in these cues over time would lead to percepts of non-rigidity in sequences showing physically rigid objects in motion. We develop an experimental paradigm to test this hypothesis and present results with two patients with impairments in motion perception due to focal neurological damage, as well as two control subjects. Consistent with our hypothesis, the data suggest that figural cues strongly influence the perception of structure in motion sequences, even to the extent of inducing non-rigid percepts in sequences where motion information alone would yield rigid structures. Beyond helping to probe the issue of shape perception, our experimental paradigm might also serve as a possible perceptual assessment tool in a clinical setting.

Improvement in spatial imagery following sight onset late in childhood.

The factors contributing to the development of spatial imagery skills are not well understood. Here, we consider whether visual experience shapes these skills. Although differences in spatial imagery between sighted and blind individuals have been reported, it is unclear whether these differences are truly due to visual deprivation or instead are due to extraneous factors, such as reduced opportunities for the blind to interact with their environment. A direct way of assessing vision's contribution to the development of spatial imagery is to determine whether spatial imagery skills change soon after the onset of sight in congenitally blind individuals. We tested 10 children who gained sight after several years of congenital blindness and found significant improvements in their spatial imagery skills following sight-restoring surgeries. These results provide evidence of vision's contribution to spatial imagery and also have implications for the nature of internal spatial representations.

The influence of semantics on long-term visual memory capacity in children and adults.

Human visual memory capacity has a rapid developmental progression. Here we examine whether image semantics modulate this progression. We assessed the performance of children (6-14 years) and young adults (19-36 years) on a visual memory task using real-world (or meaningful) as well as abstract image sets, which were matched in low-level image attributes. For real images, we find comparable performance across the two age groups, consistent with previously reported results. However, for abstract images, we find a clear age-related difference indicating greater reliance of children's memory processes on semantics, suggesting that strategies for encoding abstract patterns keep improving even into late childhood. We complemented these studies with computational experiments designed to examine the role of increasing experience with real-world images on real and abstract image encoding, to examine whether the observed age-related differences, as well as the general privilege of real over abstract images, can emerge directly through experience with meaningful images. Our results provide support for this possibility and set the stage for a finer-grained investigation of the timeline along which children's memory capacity for abstract images reaches adult levels.

Autistic Adults Show Intact Learning on a Visuospatial Serial Reaction Time Task.

Some theories have proposed that autistic individuals have difficulty learning predictive relationships. We tested this hypothesis using a serial reaction time task in which participants learned to predict the locations of a repeating sequence of target locations. We conducted a large-sample online study with 61 autistic and 71 neurotypical adults. The autistic group had slower overall reaction times, but demonstrated sequence-specific learning equivalent to the neurotypical group, consistent with other findings of typical procedural memory in autism. The neurotypical group, however, made significantly more prediction-related errors early in the experiment when the stimuli changed from repeated sequences to random locations, suggesting certain limited behavioural differences in the learning or utilization of predictive relationships for autistic adults.

Impact of early visual experience on later usage of color cues.

Human visual recognition is remarkably robust to chromatic changes. In this work, we provide a potential account of the roots of this resilience based on observations with 10 congenitally blind children who gained sight late in life. Several months or years following their sight-restoring surgeries, the removal of color cues markedly reduced their recognition performance, whereas age-matched normally sighted children showed no such decrement. This finding may be explained by the greater-than-neonatal maturity of the late-sighted children's color system at sight onset, inducing overly strong reliance on chromatic cues. Simulations with deep neural networks corroborate this hypothesis. These findings highlight the adaptive significance of typical developmental trajectories and provide guidelines for enhancing machine vision systems.

Development of a Self-Report Measure of Prediction in Daily Life: The Prediction-Related Experiences Questionnaire.

PURPOSE: Predictions are complex, multisensory, and dynamic processes involving real-time adjustments based on environmental inputs. Disruptions to prediction abilities have been proposed to underlie characteristics associated with autism. While there is substantial empirical literature related to prediction, the field lacks a self-assessment measure of prediction skills related to daily tasks. Such a measure would be useful to better understand the nature of day-to-day prediction-related activities and characterize these abilities in individuals who struggle with prediction. METHODS: An interdisciplinary mixed-methods approach was utilized to develop and validate a self-report questionnaire of prediction skills for adults, the Prediction-Related Experiences Questionnaire (PRE-Q). Two rounds of online field testing were completed in samples of autistic and neurotypical (NT) adults. Qualitative feedback from a subset of these participants regarding question content and quality was integrated and Rasch modeling of the item responses was applied. RESULTS: The final PRE-Q includes 19 items across 3 domains (Sensory, Motor, Social), with evidence supporting the validity of the measure's 4-point response categories, internal structure, and relationship to other outcome measures associated with prediction. Consistent with models of prediction challenges in autism, autistic participants indicated more prediction-related difficulties than the NT group. CONCLUSIONS: This study provides evidence for the validity of a novel self-report questionnaire designed to measure the day-to-day prediction skills of autistic and non-autistic adults. Future research should focus on characterizing the relationship between the PRE-Q and lab-based measures of prediction, and understanding how the PRE-Q may be used to identify potential areas for clinical supports for individuals with prediction-related challenges.

Developmental change in predictive motor abilities.

Prediction is critical for successful interactions with a dynamic environment. To test the development of predictive processes over the life span, we designed a suite of interceptive tasks implemented as interactive video games. Four tasks involving interactions with a flying ball with titrated challenge quantified spatiotemporal aspects of prediction. For comparison, reaction time was assessed in a matching task. The experiments were conducted in a museum, where over 400 visitors across all ages participated, and in a laboratory with a focused age group. Results consistently showed that predictive ability improved with age to reach adult level by age 12. In contrast, reaction time continued to decrease into late adolescence. Inter-task correlations revealed that the tasks tested different aspects of predictive processes. This developmental progression complements recent findings on cerebellar and cortical maturation. Additionally, these results can serve as normative data to study predictive processes in individuals with neurodevelopmental conditions.

Rhythmic and interval-based temporal orienting in autism.

Individuals with autism spectrum disorder (ASD) may show secondary sensory and cognitive characteristics, including differences in auditory processing, attention, and, according to a prominent hypothesis, the formulation and utilization of predictions. We explored the overlap of audition, attention, and prediction with an online auditory "temporal orienting" task in which participants utilized predictive timing cues (both rhythmic and interval-based) to improve their detection of faint sounds. We compared an autistic (n = 78) with a nonautistic (n = 83) group, controlling for nonverbal IQ, and used signal detection measures and reaction times to evaluate the effect of valid temporally predictive cues. We hypothesized that temporal orienting would be compromised in autism, but this was not supported by the data: the boost in performance induced by predictability was practically identical for the two groups, except for the small subset of the ASD group with co-occurring attention deficit hyperactivity disorder, who received less benefit from interval-based cueing. However, we found that the presence of a rhythm induced a significantly stronger bias toward reporting target detections in the ASD group at large, suggesting weakened response inhibition during rhythmic entrainment.