Exploration of factors affecting webcam-based automated gaze coding.

Online experiments have been transforming the field of behavioral research, enabling researchers to increase sample sizes, access diverse populations, lower the costs of data collection, and promote reproducibility. The field of developmental psychology increasingly exploits such online testing approaches. Since infants cannot give explicit behavioral responses, one key outcome measure is infants' gaze behavior. In the absence of automated eyetrackers in participants' homes, automatic gaze classification from webcam data would make it possible to avoid painstaking manual coding. However, the lack of a controlled experimental environment may lead to various noise factors impeding automatic face detection or gaze classification. We created an adult webcam dataset that systematically reproduced noise factors from infant webcam studies which might affect automated gaze coding accuracy. We varied participants' left-right offset, distance to the camera, facial rotation, and the direction of the lighting source. Running two state-of-the-art classification algorithms (iCatcher+ and OWLET) revealed that facial detection performance was particularly affected by the lighting source, while gaze coding accuracy was consistently affected by the distance to the camera and lighting source. Morphing participants' faces to be unidentifiable did not generally affect the results, suggesting facial anonymization could be used when making online video data publicly available, for purposes of further study and transparency. Our findings will guide improving study design for infant and adult participants during online experiments. Moreover, training algorithms using our dataset will allow researchers to improve robustness and allow developmental psychologists to leverage online testing more efficiently.

The development of intrinsic timescales: A comparison between the neonate and adult brain.

In human adults and other mammals, different brain regions have distinct intrinsic timescales over which they integrate information, from shorter in unimodal sensory-motor regions to longer in transmodal higher-order regions. These have been related to cognitive performance and clinical symptoms, but it remains unclear how they develop. We asked if there are regional differences in timescales at birth that could shape learning by acting as an inductive bias, or if they develop later as the temporal statistics of the environment are learned. We used resting-state fMRI to characterise timescales in human neonates and adults. They were highly consistent across two independent neonatal groups, but in both sensory-motor and higher order areas, timescales were longer in infants compared to adults, as might be expected from their less developed myelination, and recent evidence of longer neural segments in infants watching naturalistic stimuli. In adults, we replicated the finding that transmodal areas have longer timescales than sensory-motor areas, but in infants the opposite pattern was found, driven by long infant timescales in the somotomotor network. Across regions within single brain networks, both positive (limbic) and negative (visual) correlations were found between neonates and adults. In conclusion, neonatal timescales were found to be highly structured, but distinct from adults, suggesting they act as an inductive bias that favours learning on longer timescales, particularly in unimodal regions and then develop with experience or maturation. This "take it slow" initial approach might help human infants to create more regularised, holistic representations of the input less bound to fleeting details, which would favour the development of abstract and contextual representations.

The confound of head position in within-session connectome fingerprinting in infants.

Individuals differ in their functional connectome, which can be demonstrated using a "fingerprinting" analysis in which the connectome from an individual in one dataset is used to identify the same person from an independent dataset. Recently, the origin of these fingerprints has been studied by examining if they are present in infants. The results have varied considerably, with identification rates from 10 to 90%. When fingerprinting has been performed by splitting a single imaging session into two split-sessions (within session), identification rates were higher than when two full-sessions (between sessions) were compared. This study examined whether a methodological difference could account for this variation. It was hypothesized that the infant's exact head position in the head coil may affect the measured connectome, due to the gradual inhomogeneity of signal-to-noise in phased-array coils and the breadth of possible positions for a small infant head in a head coil. This study examined the impact of this using resting state functional MRI data from the Developing Human Connectome Project second release. Using functional timeseries, fingerprinting identification was high (84-91%) within a session while between sessions it was low (7%).Using N = 416 infants' head positions, a map of the average signal-to-noise across the physical volume of the head coil was calculated and was used (independent group of 44 infants with two scan sessions) to demonstrate a significant relationship between head position in the head coil and functional connectivity. Using only the head positions (signal-to-noise values extrapolated from the group average map) of the independent group of 44 infants, high identification success was achieved across split-sessions (within session) but not full-sessions (between sessions). Using a model examining factors influencing the stability of the functional connectome, head position was seen as the strongest of the explanatory variables. We conclude within-session fingerprinting is affected by head position and future infant functional fingerprint analyses must use a different strategy or account for this impact.

Responsiveness variability during anaesthesia relates to inherent differences in brain structure and function of the frontoparietal networks.

Anaesthesia combined with functional neuroimaging provides a powerful approach for understanding the brain mechanisms of consciousness. Although propofol is used ubiquitously in clinical interventions that reversibly suppress consciousness, it shows large inter-individual variability, and the brain bases of this variability remain poorly understood. We asked whether three networks key to conscious cognition-the dorsal attention (DAN), executive control (ECN), and default mode (DMN)-underlie responsiveness variability under anaesthesia. Healthy participants (N = 17) were moderately anaesthetized during narrative understanding and resting-state conditions inside the Magnetic Resonance Imaging scanner. A target detection task measured behavioural responsiveness. An independent behavioural study (N = 25) qualified the attention demands of narrative understanding. Then, 30% of participants were unaffected in their response times, thus thwarting a key aim of anaesthesia-the suppression of behavioural responsiveness. Individuals with stronger functional connectivity within the DAN and ECN, between them, and to the DMN, and with larger grey matter volume in frontal regions were more resilient to anaesthesia. For the first time, we show that responsiveness variability during propofol anaesthesia relates to inherent differences in brain structure and function of the frontoparietal networks, which can be predicted prior to sedation. Results highlight novel markers for improving awareness monitoring during clinical anaesthesia.

The fronto-parietal network is not a flexible hub during naturalistic cognition.

The fronto-parietal network (FPN) is crucial for cognitively demanding tasks as it selectively represents task-relevant information and controls other brain regions. To implement these functions, it has been argued that it is a flexible hub that reconfigures its functional connectivity with other networks. This was supported by a study in which a set of demanding tasks were presented, that varied in their sensory features, comparison rules, and response mappings, and the FPN showed greater reconfiguration of functional connectivity between tasks than any other network. However, this task set was designed to engage the FPN, and therefore it remains an open question whether the FPN is in a flexible hub in general or only for such task sets. Using two freely available datasets (Experiment 1, N = 15, Experiment 2, N = 644), we examined dynamic functional connectivity during naturalistic cognition, while participants watched a movie. Many differences in the flexibility were found across networks but the FPN was not the most flexible hub in the brain, during either movie for any of two measures, using a regression model or a correlation model and across five timescales. We, therefore, conclude that the FPN does not have the trait of being a flexible hub, although it may adopt this state for particular task sets.

Experience Transforms Conjunctive Object Representations: Neural Evidence for Unitization After Visual Expertise.

Certain transformations must occur within the brain to allow rapid processing of familiar experiences. Complex objects are thought to become unitized, whereby multifeature conjunctions are retrieved as rapidly as a single feature. Behavioral studies strongly support unitization theory, but a compelling neural mechanism is lacking. Here, we examined how unitization transforms conjunctive representations to become more "feature-like" by recruiting posterior regions of the ventral visual stream (VVS) whose architecture is specialized for processing single features. We used functional magnetic resonance imaging to scan humans before and after visual training with novel objects. We implemented a novel multivoxel pattern analysis to measure a conjunctive code, which represented a conjunction of object features above and beyond the sum of the parts. Importantly, a multivoxel searchlight showed that the strength of conjunctive coding in posterior VVS increased posttraining. Furthermore, multidimensional scaling revealed representational separation at the level of individual features in parallel to the changes at the level of feature conjunctions. Finally, functional connectivity between anterior and posterior VVS was higher for novel objects than for trained objects, consistent with early involvement of anterior VVS in unitizing feature conjunctions in response to novelty. These data demonstrate that the brain implements unitization as a mechanism to refine complex object representations over the course of multiple learning experiences.

Naturalistic Audio-Movies and Narrative Synchronize "Visual" Cortices across Congenitally Blind But Not Sighted Individuals.

How does developmental experience, as opposed to intrinsic physiology, shape cortical function? Naturalistic stimuli were used to elicit neural synchrony in individuals blind from birth (n = 18) and those who grew up with sight (n = 18). Blind and blindfolded sighted participants passively listened to three audio-movie clips, an auditory narrative, a sentence shuffled version of the narrative (maintaining language but lacking a plotline), and a version of the narrative backward (lacking both language and plot). For both groups, early auditory cortices were synchronized to a similar degree across stimulus types, whereas higher-cognitive temporoparietal and prefrontal areas were more synchronized by meaningful, temporally extended stimuli (i.e., audio movies and narrative). "Visual" cortices were more synchronized across blind than sighted individuals, but only for audio-movies and narrative. In the blind group, visual cortex synchrony was low for backward speech and intermediate for sentence shuffle. Meaningful auditory stimuli synchronize visual cortices of people born blind.SIGNIFICANCE STATEMENT Naturalistic stimuli engage cognitive processing at many levels. Here, we harnessed this richness to investigate the effect of experience on cortical function. We find that listening to naturalistic audio movies and narrative drives synchronized activity across "visual" cortices of blind, more so than sighted, individuals. Visual cortex synchronization varies with meaningfulness and cognitive complexity. Higher synchrony is observed for temporally extended meaningful stimuli (e.g., movies/narrative), intermediate for shuffled sentences, lowest for time varying complex noise. By contrast, auditory cortex was synchronized equally by meaningful and meaningless stimuli. In congenitally blind individuals most of visual cortex is engaged by meaningful naturalistic stimuli.

Animacy and real-world size shape object representations in the human medial temporal lobes.

Identifying what an object is, and whether an object has been encountered before, is a crucial aspect of human behavior. Despite this importance, we do not yet have a complete understanding of the neural basis of these abilities. Investigations into the neural organization of human object representations have revealed category specific organization in the ventral visual stream in perceptual tasks. Interestingly, these categories fall within broader domains of organization, with reported distinctions between animate, inanimate large, and inanimate small objects. While there is some evidence for category specific effects in the medial temporal lobe (MTL), in particular in perirhinal and parahippocampal cortex, it is currently unclear whether domain level organization is also present across these structures. To this end, we used fMRI with a continuous recognition memory task. Stimuli were images of objects from several different categories, which were either animate or inanimate, or large or small within the inanimate domain. We employed representational similarity analysis (RSA) to test the hypothesis that object-evoked responses in MTL structures during recognition-memory judgments also show evidence for domain-level organization along both dimensions. Our data support this hypothesis. Specifically, object representations were shaped by either animacy, real-world size, or both, in perirhinal and parahippocampal cortex, and the hippocampus. While sensitivity to these dimensions differed across structures when probed individually, hinting at interesting links to functional differentiation, similarities in organization across MTL structures were more prominent overall. These results argue for continuity in the organization of object representations in the ventral visual stream and the MTL.

Auditory structural connectivity in preterm and healthy term infants during the first postnatal year.

Assessing language development in the first postnatal year is difficult, as receptive and expressive skills are rudimentary. Although outward manifestations of change are limited, the auditory language system is thought to undergo critical development at this age, as the foundations are laid for the rapid onset of spoken language in the second and third years. We recruited 11 infants, 7 healthy controls (gestational age = 40.69 ± 0.56; range from 40 to 41.43) and preterm babies (gestational age = 28.04 ± 0.95; range from 27.43 to 29.43) who underwent a Magnetic Resonance Imaging study during the first postnatal year (age at scan = 194.18 ± 97.98). We assessed white matter tracts using diffusion-weighted magnetic resonance imaging with probabilistic tractography. Fractional anisotropy was found to be largely mature even at one month, although there was a little further increase during the first postnatal year in both the acoustic radiation and the direct brainstem-Heschl's pathway.

Disentangling Representations of Object and Grasp Properties in the Human Brain.

UNLABELLED: The properties of objects, such as shape, influence the way we grasp them. To quantify the role of different brain regions during grasping, it is necessary to disentangle the processing of visual dimensions related to object properties from the motor aspects related to the specific hand configuration. We orthogonally varied object properties (shape, size, and elongation) and task (passive viewing, precision grip with two or five digits, or coarse grip with five digits) and used representational similarity analysis of functional magnetic resonance imaging data to infer the representation of object properties and hand configuration in the human brain. We found that object elongation is the most strongly represented object feature during grasping and is coded preferentially in the primary visual cortex as well as the anterior and posterior superior-parieto-occipital cortex. By contrast, primary somatosensory, motor, and ventral premotor cortices coded preferentially the number of digits while ventral-stream and dorsal-stream regions coded a mix of visual and motor dimensions. The representation of object features varied with task modality, as object elongation was less relevant during passive viewing than grasping. To summarize, this study shows that elongation is a particularly relevant property of the object to grasp, which along with the number of digits used, is represented within both ventral-stream and parietal regions, suggesting that communication between the two streams about these specific visual and motor dimensions might be relevant to the execution of efficient grasping actions. SIGNIFICANCE STATEMENT: To grasp something, the visual properties of an object guide preshaping of the hand into the appropriate configuration. Different grips can be used, and different objects require different hand configurations. However, in natural actions, grip and object type are often confounded, and the few experiments that have attempted to separate them have produced conflicting results. As such, it is unclear how visual and motor properties are represented across brain regions during grasping. Here we orthogonally manipulated object properties and grip, and revealed the visual dimension (object elongation) and the motor dimension (number of digits) that are more strongly coded in ventral and dorsal streams. These results suggest that both streams play a role in the visuomotor coding essential for grasping.

The neural basis of precise visual short-term memory for complex recognisable objects.

Recent evidence suggests that visual short-term memory (VSTM) capacity estimated using simple objects, such as colours and oriented bars, may not generalise well to more naturalistic stimuli. More visual detail can be stored in VSTM when complex, recognisable objects are maintained compared to simple objects. It is not yet known if it is recognisability that enhances memory precision, nor whether maintenance of recognisable objects is achieved with the same network of brain regions supporting maintenance of simple objects. We used a novel stimulus generation method to parametrically warp photographic images along a continuum, allowing separate estimation of the precision of memory representations and the number of items retained. The stimulus generation method was also designed to create unrecognisable, though perceptually matched, stimuli, to investigate the impact of recognisability on VSTM. We adapted the widely-used change detection and continuous report paradigms for use with complex, photographic images. Across three functional magnetic resonance imaging (fMRI) experiments, we demonstrated greater precision for recognisable objects in VSTM compared to unrecognisable objects. This clear behavioural advantage was not the result of recruitment of additional brain regions, or of stronger mean activity within the core network. Representational similarity analysis revealed greater variability across item repetitions in the representations of recognisable, compared to unrecognisable complex objects. We therefore propose that a richer range of neural representations support VSTM for complex recognisable objects.

The Cambridge Centre for Ageing and Neuroscience (Cam-CAN) data repository: Structural and functional MRI, MEG, and cognitive data from a cross-sectional adult lifespan sample.

This paper describes the data repository for the Cambridge Centre for Ageing and Neuroscience (Cam-CAN) initial study cohort. The Cam-CAN Stage 2 repository contains multi-modal (MRI, MEG, and cognitive-behavioural) data from a large (approximately N=700), cross-sectional adult lifespan (18-87years old) population-based sample. The study is designed to characterise age-related changes in cognition and brain structure and function, and to uncover the neurocognitive mechanisms that support healthy cognitive ageing. The database contains raw and preprocessed structural MRI, functional MRI (active tasks and resting state), and MEG data (active tasks and resting state), as well as derived scores from cognitive behavioural experiments spanning five broad domains (attention, emotion, action, language, and memory), and demographic and neuropsychological data. The dataset thus provides a depth of neurocognitive phenotyping that is currently unparalleled, enabling integrative analyses of age-related changes in brain structure, brain function, and cognition, and providing a testbed for novel analyses of multi-modal neuroimaging data.

Adult-like processing of naturalistic sounds in auditory cortex by 3- and 9-month old infants.

Functional neuroimaging has been used to show that the developing auditory cortex of very young human infants responds, in some way, to sound. However, impoverished stimuli and uncontrolled designs have made it difficult to attribute brain responses to specific auditory features, and thus made it difficult to assess the maturity of feature tuning in auditory cortex. To address this, we used functional magnetic resonance imaging (fMRI) to measure the brain activity evoked by naturalistic sounds (a series of sung lullabies) in two groups of infants (3 and 9 months) and adults. We developed a novel analysis method - inter-subject regression (ISR) - to quantify the similarity of cortical responses between infants and adults, and to decompose components of the response due to different auditory features. We found that the temporal pattern of activity in infant auditory cortex shared similarity with adults. Some of this shared response could be attributed to simple acoustic features, such as frequency, pitch, envelope, but other parts were not, suggesting that even more complex adult-like features are represented in auditory cortex in early infancy.

Using Functional Magnetic Resonance Imaging to Detect Preserved Function in a Preterm Infant with Brain Injury.

We studied developmental plasticity using functional magnetic resonance imaging (fMRI) in a preterm infant with brain injury on structural MRI. fMRI showed preserved brain function and subsequent neurodevelopment was within the normal range. Multimodal neuroimaging including fMRI can improve understanding of neural plasticity after preterm birth and brain injury.

Conjunctive Coding of Complex Object Features.

Critical to perceiving an object is the ability to bind its constituent features into a cohesive representation, yet the manner by which the visual system integrates object features to yield a unified percept remains unknown. Here, we present a novel application of multivoxel pattern analysis of neuroimaging data that allows a direct investigation of whether neural representations integrate object features into a whole that is different from the sum of its parts. We found that patterns of activity throughout the ventral visual stream (VVS), extending anteriorly into the perirhinal cortex (PRC), discriminated between the same features combined into different objects. Despite this sensitivity to the unique conjunctions of features comprising objects, activity in regions of the VVS, again extending into the PRC, was invariant to the viewpoints from which the conjunctions were presented. These results suggest that the manner in which our visual system processes complex objects depends on the explicit coding of the conjunctions of features comprising them.

Online recruitment and testing of infants with Mechanical Turk.

Testing infants in the laboratory is expensive in time and money; consequently, many studies are underpowered, reducing their reproducibility. We investigated whether the online platform, Amazon Mechanical Turk (MTurk), could be used as a resource to more easily recruit and measure the behavior of infant populations. Using a looking time paradigm, with users' webcams we recorded how long infants aged 5 to 8months attended while viewing children's television programs. We found that infants (N=57) were more reliably engaged by some movies than by others and that the most engaging movies could maintain attention for approximately 70% of a 10- to 13-min period. We then identified the cinematic features within the movies. Faces, singing-and-rhyming, and camera zooms were found to increase infant attention. Together, we established that MTurk can be used as a rapid tool for effectively recruiting and testing infants.

A common neural code for similar conscious experiences in different individuals.

The interpretation of human consciousness from brain activity, without recourse to speech or action, is one of the most provoking and challenging frontiers of modern neuroscience. We asked whether there is a common neural code that underpins similar conscious experiences, which could be used to decode these experiences in the absence of behavior. To this end, we used richly evocative stimulation (an engaging movie) portraying real-world events to elicit a similar conscious experience in different people. Common neural correlates of conscious experience were quantified and related to measurable, quantitative and qualitative, executive components of the movie through two additional behavioral investigations. The movie's executive demands drove synchronized brain activity across healthy participants' frontal and parietal cortices in regions known to support executive function. Moreover, the timing of activity in these regions was predicted by participants' highly similar qualitative experience of the movie's moment-to-moment executive demands, suggesting that synchronization of activity across participants underpinned their similar experience. Thus we demonstrate, for the first time to our knowledge, that a neural index based on executive function reliably predicted every healthy individual's similar conscious experience in response to real-world events unfolding over time. This approach provided strong evidence for the conscious experience of a brain-injured patient, who had remained entirely behaviorally nonresponsive for 16 y. The patient's executive engagement and moment-to-moment perception of the movie content were highly similar to that of every healthy participant. These findings shed light on the common basis of human consciousness and enable the interpretation of conscious experience in the absence of behavior.

Flexible information coding in human auditory cortex during perception, imagery, and STM of complex sounds.

Auditory cortex is the first cortical region of the human brain to process sounds. However, it has recently been shown that its neurons also fire in the absence of direct sensory input, during memory maintenance and imagery. This has commonly been taken to reflect neural coding of the same acoustic information as during the perception of sound. However, the results of the current study suggest that the type of information encoded in auditory cortex is highly flexible. During perception and memory maintenance, neural activity patterns are stimulus specific, reflecting individual sound properties. Auditory imagery of the same sounds evokes similar overall activity in auditory cortex as perception. However, during imagery abstracted, categorical information is encoded in the neural patterns, particularly when individuals are experiencing more vivid imagery. This highlights the necessity to move beyond traditional "brain mapping" inference in human neuroimaging, which assumes common regional activation implies similar mental representations.

Strength of retinotopic representation of visual memories is modulated by strategy.

Human visual cortex shows retinotopic organization during both perception and attention, but whether this remains true for visual short-term memory (VSTM) is uncertain. In 2 functional magnetic resonance imaging experiments, we separated retinotopic activation during perception, attention, and VSTM maintenance. The 2 experiments differed in whether spatial encoding of the VSTM stimuli and prospective attention to the locations of the remembered items was encouraged or discouraged. Using multivoxel pattern analysis to extract a measure of spatial coding in early visual cortex, we saw sensory and attentional retinotopic coding in both experiments. However, significant spatial coding during memory maintenance was only seen where a spatial strategy was encouraged. Furthermore, individual differences in attentional spatial coding predicted performance in both experiments, while individual differences in maintenance spatial coding predicted performance in neither. We conclude that retinotopic coding in the early visual cortex during VSTM maintenance is not obligatory, that attentional processes during stimulus perception modulate memory performance, and that different attentional strategies are used depending on the task in hand.

Tunes stuck in your brain: The frequency and affective evaluation of involuntary musical imagery correlate with cortical structure.

Recent years have seen a growing interest in the neuroscience of spontaneous cognition. One form of such cognition is involuntary musical imagery (INMI), the non-pathological and everyday experience of having music in one's head, in the absence of an external stimulus. In this study, aspects of INMI, including frequency and affective evaluation, were measured by self-report in 44 subjects and related to variation in brain structure in these individuals. Frequency of INMI was related to cortical thickness in regions of right frontal and temporal cortices as well as the anterior cingulate and left angular gyrus. Affective aspects of INMI, namely the extent to which subjects wished to suppress INMI or considered them helpful, were related to gray matter volume in right temporopolar and parahippocampal cortices respectively. These results provide the first evidence that INMI is a common internal experience recruiting brain networks involved in perception, emotions, memory and spontaneous thoughts.