White matter plasticity following cataract surgery in congenitally blind patients.

The visual system develops abnormally when visual input is absent or degraded during a critical period early in life. Restoration of the visual input later in life is generally thought to have limited benefit because the visual system will lack sufficient plasticity to adapt to and utilize the information from the eyes. Recent evidence, however, shows that congenitally blind adolescents can recover both low-level and higher-level visual function following surgery. In this study, we assessed behavioral performance in both a visual acuity and a face perception task alongside longitudinal structural white matter changes in terms of fractional anisotropy (FA) and mean diffusivity (MD). We studied congenitally blind patients with dense bilateral cataracts, who received cataract surgery at different stages of adolescence. Our goal was to differentiate between age- and surgery-related changes in both behavioral performance and structural measures to identify neural correlates which might contribute to recovery of visual function. We observed surgery-related long-term increases of structural integrity of late-visual pathways connecting the occipital regions with ipsilateral fronto-parieto-temporal regions or homotopic contralateral areas. Comparison to a group of age-matched healthy participants indicated that these improvements went beyond the expected changes in FA and MD based on maturation alone. Finally, we found that the extent of behavioral improvement in face perception was mediated by changes in structural integrity in late visual pathways. Our results suggest that sufficient plasticity remains in adolescence to partially overcome abnormal visual development and help localize the sites of neural change underlying sight recovery.

Robustness to Transformations Across Categories: Is Robustness Driven by Invariant Neural Representations?

Deep convolutional neural networks (DCNNs) have demonstrated impressive robustness to recognize objects under transformations (e.g., blur or noise) when these transformations are included in the training set. A hypothesis to explain such robustness is that DCNNs develop invariant neural representations that remain unaltered when the image is transformed. However, to what extent this hypothesis holds true is an outstanding question, as robustness to transformations could be achieved with properties different from invariance; for example, parts of the network could be specialized to recognize either transformed or nontransformed images. This article investigates the conditions under which invariant neural representations emerge by leveraging that they facilitate robustness to transformations beyond the training distribution. Concretely, we analyze a training paradigm in which only some object categories are seen transformed during training and evaluate whether the DCNN is robust to transformations across categories not seen transformed. Our results with state-of-the-art DCNNs indicate that invariant neural representations do not always drive robustness to transformations, as networks show robustness for categories seen transformed during training even in the absence of invariant neural representations. Invariance emerges only as the number of transformed categories in the training set is increased. This phenomenon is much more prominent with local transformations such as blurring and high-pass filtering than geometric transformations such as rotation and thinning, which entail changes in the spatial arrangement of the object. Our results contribute to a better understanding of invariant neural representations in deep learning and the conditions under which it spontaneously emerges.

Vulnerability of facial attractiveness perception to early and multi-year visual deprivation.

Judgments of facial attractiveness invariably accompany our perception of faces. Even neonates appear to be capable of making such judgments in a manner consistent with adults. This suggests that the processes supporting facial attractiveness require little, if any, visual experience to manifest. Here we investigate the resilience of these processes to several years of early-onset visual deprivation. Specifically, we study whether congenitally blind children treated several years after birth possess the ability to rate facial attractiveness in a manner congruent to normally sighted individuals. The data reveal significant individual variability in the way each newly sighted child perceives attractiveness. This is in marked contrast to data from normally sighted controls who exhibit strong across-subject agreement in facial attractiveness ratings. This variability may be attributable, in part, to atypical facial encoding strategies used by the newly sighted children. Overall, our results suggest that the development of facial attractiveness perception is likely to be vulnerable to early visual deprivation, pointing to the existence of a possible sensitive period early in the developmental trajectory.

Potential downside of high initial visual acuity.

Children who are treated for congenital cataracts later exhibit impairments in configural face analysis. This has been explained in terms of a critical period for the acquisition of normal face processing. Here, we consider a more parsimonious account according to which deficits in configural analysis result from the abnormally high initial retinal acuity that children treated for cataracts experience, relative to typical newborns. According to this proposal, the initial period of low retinal acuity characteristic of normal visual development induces extended spatial processing in the cortex that is important for configural face judgments. As a computational test of this hypothesis, we examined the effects of training with high-resolution or blurred images, and staged combinations, on the receptive fields and performance of a convolutional neural network. The results show that commencing training with blurred images creates receptive fields that integrate information across larger image areas and leads to improved performance and better generalization across a range of resolutions. These findings offer an explanation for the observed face recognition impairments after late treatment of congenital blindness, suggest an adaptive function for the acuity trajectory in normal development, and provide a scheme for improving the performance of computational face recognition systems.

Recognizing Facial Slivers.

We report here an unexpectedly robust ability of healthy human individuals ( n = 40) to recognize extremely distorted needle-like facial images, challenging the well-entrenched notion that veridical spatial configuration is necessary for extracting facial identity. In face identification tasks of parametrically compressed internal and external features, we found that the sum of performances on each cue falls significantly short of performance on full faces, despite the equal visual information available from both measures (with full faces essentially being a superposition of internal and external features). We hypothesize that this large deficit stems from the use of positional information about how the internal features are positioned relative to the external features. To test this, we systematically changed the relations between internal and external features and found preferential encoding of vertical but not horizontal spatial relationships in facial representations ( n = 20). Finally, we employ magnetoencephalography imaging ( n = 20) to demonstrate a close mapping between the behavioral psychometric curve and the amplitude of the M250 face familiarity, but not M170 face-sensitive evoked response field component, providing evidence that the M250 can be modulated by faces that are perceptually identifiable, irrespective of extreme distortions to the face's veridical configuration. We theorize that the tolerance to compressive distortions has evolved from the need to recognize faces across varying viewpoints. Our findings help clarify the important, but poorly defined, concept of facial configuration and also enable an association between behavioral performance and previously reported neural correlates of face perception.

The influence of semantics on long-term visual memory capacity in children and adults.

Human visual memory capacity has a rapid developmental progression. Here we examine whether image semantics modulate this progression. We assessed the performance of children (6-14 years) and young adults (19-36 years) on a visual memory task using real-world (or meaningful) as well as abstract image sets, which were matched in low-level image attributes. For real images, we find comparable performance across the two age groups, consistent with previously reported results. However, for abstract images, we find a clear age-related difference indicating greater reliance of children's memory processes on semantics, suggesting that strategies for encoding abstract patterns keep improving even into late childhood. We complemented these studies with computational experiments designed to examine the role of increasing experience with real-world images on real and abstract image encoding, to examine whether the observed age-related differences, as well as the general privilege of real over abstract images, can emerge directly through experience with meaningful images. Our results provide support for this possibility and set the stage for a finer-grained investigation of the timeline along which children's memory capacity for abstract images reaches adult levels.

Impact of early visual experience on later usage of color cues.

Human visual recognition is remarkably robust to chromatic changes. In this work, we provide a potential account of the roots of this resilience based on observations with 10 congenitally blind children who gained sight late in life. Several months or years following their sight-restoring surgeries, the removal of color cues markedly reduced their recognition performance, whereas age-matched normally sighted children showed no such decrement. This finding may be explained by the greater-than-neonatal maturity of the late-sighted children's color system at sight onset, inducing overly strong reliance on chromatic cues. Simulations with deep neural networks corroborate this hypothesis. These findings highlight the adaptive significance of typical developmental trajectories and provide guidelines for enhancing machine vision systems.

Motion's privilege in recognizing facial expressions following treatment for blindness.

In his 1872 monograph, Charles Darwin posited that "… the habit of expressing our feelings by certain movements, though now rendered innate, had been in some manner gradually acquired."1 Nearly 150 years later, researchers are still teasing apart innate versus experience-dependent contributions to expression recognition. Indeed, studies have shown that face detection is surprisingly resilient to early visual deprivation,2,3,4,5 pointing to plasticity that extends beyond dogmatic critical periods.6,7,8 However, it remains unclear whether such resilience extends to downstream processing, such as the ability to recognize facial expressions. The extent to which innate versus experience-dependent mechanisms contribute to this ability has yet to be fully explored.9,10,11,12,13 To investigate the impact of early visual experience on facial-expression recognition, we studied children with congenital cataracts who have undergone sight-correcting treatment14,15 and tracked their longitudinal skill acquisition as they gain sight late in life. We introduce and explore two potential facilitators of late-life plasticity: the availability of newborn-like coarse visual acuity prior to treatment16 and the privileged role of motion following treatment.4,17,18 We find that early visual deprivation does not preclude partial acquisition of facial-expression recognition. While rudimentary pretreatment vision is sufficient to allow a low level of expression recognition, it does not facilitate post-treatment improvements. Additionally, only children commencing vision with high visual acuity privilege the use of dynamic cues. We conclude that skipping typical visual experience early in development and introducing high-resolution imagery late in development restricts, but does not preclude, facial-expression skill acquisition and that the representational mechanisms driving this learning differ from those that emerge during typical visual development.

Recognizing distant faces.

As an 'early alerting' sense, one of the primary tasks for the human visual system is to recognize distant objects. In the specific context of facial identification, this ecologically important task has received surprisingly little attention. Most studies have investigated facial recognition at short, fixed distances. Under these conditions, the photometric and configural information related to the eyes, nose and mouth are typically found to be primary determinants of facial identity. Here we characterize face recognition performance as a function of viewing distance and investigate whether the primacy of the internal features continues to hold across increasing viewing distances. We find that exploring the distance dimension reveals a qualitatively different salience distribution across a face. Observers' recognition performance significantly exceeds that obtained with the internal facial physiognomy, and also exceeds the computed union of performances with internal and external features alone, suggesting that in addition to the mutual configuration of the eyes, nose and mouth, it is the relationships between these features and external head contours that are crucial for recognition. We have also conducted computational studies with convolutional neural networks trained on the task of face recognition to examine whether this representational bias could emerge spontaneously through exposure to faces. The results provide partial support for this possibility while also highlighting important differences between the human and artificial system. These findings have implications for the nature of facial representations useful for a visual system, whether human or machine, for recognition over large and varying distances.