Evolving Images for Visual Neurons Using a Deep Generative Network Reveals Coding Principles and Neuronal Preferences.

What specific features should visual neurons encode, given the infinity of real-world images and the limited number of neurons available to represent them? We investigated neuronal selectivity in monkey inferotemporal cortex via the vast hypothesis space of a generative deep neural network, avoiding assumptions about features or semantic categories. A genetic algorithm searched this space for stimuli that maximized neuronal firing. This led to the evolution of rich synthetic images of objects with complex combinations of shapes, colors, and textures, sometimes resembling animals or familiar people, other times revealing novel patterns that did not map to any clear semantic category. These results expand our conception of the dictionary of features encoded in the cortex, and the approach can potentially reveal the internal representations of any system whose input can be captured by a generative model.

A Whole-Brain Topographic Ontology.

It is a common view that the intricate array of specialized domains in the ventral visual pathway is innately prespecified. What this review postulates is that it is not. We explore the origins of domain specificity, hypothesizing that the adult brain emerges from an interplay between a domain-general map-based architecture, shaped by intrinsic mechanisms, and experience. We argue that the most fundamental innate organization of cortex in general, and not just the visual pathway, is a map-based topography that governs how the environment maps onto the brain, how brain areas interconnect, and ultimately, how the brain processes information.

On the relationship between maps and domains in inferotemporal cortex.

How does the brain encode information about the environment? Decades of research have led to the pervasive notion that the object-processing pathway in primate cortex consists of multiple areas that are each specialized to process different object categories (such as faces, bodies, hands, non-face objects and scenes). The anatomical consistency and modularity of these regions have been interpreted as evidence that these regions are innately specialized. Here, we propose that ventral-stream modules do not represent clusters of circuits that each evolved to process some specific object category particularly important for survival, but instead reflect the effects of experience on a domain-general architecture that evolved to be able to adapt, within a lifetime, to its particular environment. Furthermore, we propose that the mechanisms underlying the development of domains are both evolutionarily old and universal across cortex. Topographic maps are fundamental, governing the development of specializations across systems, providing a framework for brain organization.

Triggers for mother love.

Previous studies showed that baby monkeys separated from their mothers develop strong and lasting attachments to inanimate surrogate mothers, but only if the surrogate has a soft texture; soft texture is more important for the infant's attachment than is the provision of milk. Here I report that postpartum female monkeys also form strong and persistent attachments to inanimate surrogate infants, that the template for triggering maternal attachment is also tactile, and that even a brief period of attachment formation can dominate visual and auditory cues indicating a more appropriate target.

Face neurons encode nonsemantic features.

The primate inferior temporal cortex contains neurons that respond more strongly to faces than to other objects. Termed "face neurons," these neurons are thought to be selective for faces as a semantic category. However, face neurons also partly respond to clocks, fruits, and single eyes, raising the question of whether face neurons are better described as selective for visual features related to faces but dissociable from them. We used a recently described algorithm, XDream, to evolve stimuli that strongly activated face neurons. XDream leverages a generative neural network that is not limited to realistic objects. Human participants assessed images evolved for face neurons and for nonface neurons and natural images depicting faces, cars, fruits, etc. Evolved images were consistently judged to be distinct from real faces. Images evolved for face neurons were rated as slightly more similar to faces than images evolved for nonface neurons. There was a correlation among natural images between face neuron activity and subjective "faceness" ratings, but this relationship did not hold for face neuron­evolved images, which triggered high activity but were rated low in faceness. Our results suggest that so-called face neurons are better described as tuned to visual features rather than semantic categories.

Look twice: A generalist computational model predicts return fixations across tasks and species.

Primates constantly explore their surroundings via saccadic eye movements that bring different parts of an image into high resolution. In addition to exploring new regions in the visual field, primates also make frequent return fixations, revisiting previously foveated locations. We systematically studied a total of 44,328 return fixations out of 217,440 fixations. Return fixations were ubiquitous across different behavioral tasks, in monkeys and humans, both when subjects viewed static images and when subjects performed natural behaviors. Return fixations locations were consistent across subjects, tended to occur within short temporal offsets, and typically followed a 180-degree turn in saccadic direction. To understand the origin of return fixations, we propose a proof-of-principle, biologically-inspired and image-computable neural network model. The model combines five key modules: an image feature extractor, bottom-up saliency cues, task-relevant visual features, finite inhibition-of-return, and saccade size constraints. Even though there are no free parameters that are fine-tuned for each specific task, species, or condition, the model produces fixation sequences resembling the universal properties of return fixations. These results provide initial steps towards a mechanistic understanding of the trade-off between rapid foveal recognition and the need to scrutinize previous fixation locations.

Anatomical correlates of face patches in macaque inferotemporal cortex.

Primate brains typically have regions within the ventral visual stream that are selectively responsive to faces. In macaques, these face patches are located in similar parts of inferotemporal cortex across individuals although correspondence with particular anatomical features has not been reported previously. Here, using high-resolution functional and anatomical imaging, we show that small "bumps," or buried gyri, along the lower bank of the superior temporal sulcus are predictive of the location of face-selective regions. Recordings from implanted multielectrode arrays verified that these bumps contain face-selective neurons. These bumps were present in monkeys raised without seeing faces and that lack face patches, indicating that these anatomical landmarks are predictive of, but not sufficient for, the presence of face selectivity. These bumps are found across primate species that span taxonomy lines, indicating common evolutionary developmental mechanisms. The bumps emerge during fetal development in macaques, indicating that they arise from general developmental mechanisms that result in the regularity of cortical folding of the entire brain.

Evolution of Osteocrin as an activity-regulated factor in the primate brain.

Sensory stimuli drive the maturation and function of the mammalian nervous system in part through the activation of gene expression networks that regulate synapse development and plasticity. These networks have primarily been studied in mice, and it is not known whether there are species- or clade-specific activity-regulated genes that control features of brain development and function. Here we use transcriptional profiling of human fetal brain cultures to identify an activity-dependent secreted factor, Osteocrin (OSTN), that is induced by membrane depolarization of human but not mouse neurons. We find that OSTN has been repurposed in primates through the evolutionary acquisition of DNA regulatory elements that bind the activity-regulated transcription factor MEF2. In addition, we demonstrate that OSTN is expressed in primate neocortex and restricts activity-dependent dendritic growth in human neurons. These findings suggest that, in response to sensory input, OSTN regulates features of neuronal structure and function that are unique to primates.

Sulcal Depth in the Medial Ventral Temporal Cortex Predicts the Location of a Place-Selective Region in Macaques, Children, and Adults.

The evolution and development of anatomical-functional relationships in the cerebral cortex is of major interest in neuroscience. Here, we leveraged the fact that a functional region selective for visual scenes is located within a sulcus in the medial ventral temporal cortex (VTC) in both humans and macaques to examine the relationship between sulcal depth and place selectivity in the medial VTC across species and age groups. To do so, we acquired anatomical and functional magnetic resonance imaging scans in 9 macaques, 26 human children, and 28 human adults. Our results revealed a strong structural-functional coupling between sulcal depth and place selectivity across age groups and species in which selectivity was strongest near the deepest sulcal point (the sulcal pit). Interestingly, this coupling between sulcal depth and place selectivity strengthens from childhood to adulthood in humans. Morphological analyses suggest that the stabilization of sulcal-functional coupling in adulthood may be due to sulcal deepening and areal expansion with age as well as developmental differences in cortical curvature at the pial, but not the white matter surfaces. Our results implicate sulcal features as functional landmarks in high-level visual cortex and highlight that sulcal-functional relationships in the medial VTC are preserved between macaques and humans despite differences in cortical folding.

Body map proto-organization in newborn macaques.

Topographic sensory maps are a prominent feature of the adult primate brain. Here, we asked whether topographic representations of the body are present at birth. Using functional MRI (fMRI), we find that the newborn somatomotor system, spanning frontoparietal cortex and subcortex, comprises multiple topographic representations of the body. The organization of these large-scale body maps was indistinguishable from those in older monkeys. Finer-scale differentiation of individual fingers increased over the first 2 y, suggesting that topographic representations are refined during early development. Last, we found that somatomotor representations were unchanged in 2 visually impaired monkeys who relied on touch for interacting with their environment, demonstrating that massive shifts in early sensory experience in an otherwise anatomically intact brain are insufficient for driving cross-modal plasticity. We propose that a topographic scaffolding is present at birth that both directs and constrains experience-driven modifications throughout somatosensory and motor systems.

The neurovascular response is attenuated by focused ultrasound-mediated disruption of the blood-brain barrier.

Focused ultrasound (FUS)-induced disruption of the blood-brain barrier (BBB) is a non-invasive method to target drug delivery to specific brain areas that is now entering into the clinic. Recent studies have shown that the method has several secondary effects on local physiology and brain function beyond making the vasculature permeable to normally non-BBB penetrant molecules. This study uses functional MRI methods to investigate how FUS BBB opening alters the neurovascular response in the rat brain. Nine rats underwent actual and sham FUS induced BBB opening targeted to the right somatosensory cortex (SI) followed by four runs of bilateral electrical hind paw stimulus-evoked fMRI. The neurovascular response was quantified using measurements of the blood oxygen level dependent (BOLD) signal and cerebral blood flow (CBF). An additional three rats underwent the same FUS-BBB opening followed by stimulus-evoked fMRI with high resolution BOLD imaging and BOLD imaging of a carbogen-breathing gas challenge. BOLD and CBF measurements at two different stimulus durations demonstrate that the neurovascular response to the stimulus is attenuated in both amplitude and duration in the region targeted for FUS-BBB opening. The carbogen results show that the attenuation in response amplitude, but not duration, is still present when the signaling mechanism originates from changes in blood oxygenation instead of stimulus-induced neuronal activity. There is some evidence of non-local effects, including a possible global decrease in baseline CBF. All effects are resolved by 24 h after FUS-BBB opening. Taken together, these results suggest that FUS-BBB opening alters that state of local brain neurovascular physiology in such a way that hinders its ability to respond to demands for increased blood flow to the region. The mechanisms for this effect need to be elucidated.

Modulation of brain function by targeted delivery of GABA through the disrupted blood-brain barrier.

The technology of transcranial focused ultrasound (FUS) enables a novel approach to neuromodulation, a tool for selective manipulation of brain function to be used in neurobiology research and with potential applications in clinical treatment. The method uses transcranial focused ultrasound to non-invasively open the blood-brain barrier (BBB) in a localized region such that a systemically injected neurotransmitter chemical can be delivered to the targeted brain site. The approach modulates the chemical signaling that occurs in and between neurons, making it complimentary to most other neuromodulation techniques that affect the electrical properties of neuronal activity. Here, we report delivering the inhibitory neurotransmitter GABA to the right somatosensory cortex of the rat brain during bilateral hind paw electrical stimulation and measure the inhibition of activation using functional MRI (fMRI). In a 2 × 2 factorial design, we evaluated conditions of BBB Closed vs BBB Open and No GABA vs GABA. Results from fMRI measurements of the blood oxygen level-dependent (BOLD) signal show: 1) intravenous GABA injection without FUS-mediated BBB opening does not have an effect on the BOLD response; 2) FUS-mediated BBB opening alone significantly alters the BOLD signal response to the stimulus, both in amplitude and shape of the time course; 3) the combination of FUS-mediated BBB opening and GABA injection further reduces the peak amplitude and spatial extent of the BOLD signal response to the stimulus. The data support the thesis that FUS-mediated opening of the BBB can be used to achieve non-invasive delivery of neuroactive substances for targeted manipulation of brain function.

Retinotopic Organization of Scene Areas in Macaque Inferior Temporal Cortex.

Primates have specialized domains in inferior temporal (IT) cortex that are responsive to particular image categories. Though IT traditionally has been regarded as lacking retinotopy, several recent studies in monkeys have shown that retinotopic maps extend to face patches along the lower bank of the superior temporal sulcus (STS) and neighboring regions of IT cortex. Here, we used fMRI to map the retinotopic organization of medial ventral temporal cortex in four monkeys (2 male and 2 female). We confirm the presence of visual field maps within and around the lower bank of the STS and extend these prior findings to scene-selective cortex in the ventral-most regions of IT. Within the occipitotemporal sulcus (OTS), we identified two retinotopic areas, OTS1 and OTS2. The polar angle representation of OTS2 was a mirror reversal of the OTS1 representation. These regions contained representations of the contralateral periphery and were selectively active for scene versus face, body, or object images. The extent of this retinotopy parallels that in humans and shows that the organization of the scene network is preserved across primate species. In addition retinotopic maps were identified in dorsal extrastriate, posterior parietal, and frontal cortex as well as the thalamus, including both the lateral geniculate nucleus and pulvinar. Together, it appears that most, if not all, of the macaque visual system contains organized representations of visual space.SIGNIFICANCE STATEMENT Primates have specialized domains in inferior temporal (IT) cortex that are responsive to particular image categories. Though retinotopic maps are considered a fundamental organizing principle of posterior visual cortex, IT traditionally has been regarded as lacking retinotopy. Recent imaging studies have demonstrated the presence of several visual field maps within the lateral IT. Using neuroimaging, we found multiple representations of visual space within ventral IT cortex of macaques that included scene-selective cortex. Scene domains were biased toward the peripheral visual field. These data demonstrate the prevalence of visual field maps throughout the primate visual system, including late stages in the ventral visual hierarchy, and support the idea that domains representing different categories are biased toward different parts of the visual field.

Posterior Inferotemporal Cortex Cells Use Multiple Input Pathways for Shape Encoding.

In the macaque monkey brain, posterior inferior temporal (PIT) cortex cells contribute to visual object recognition. They receive concurrent inputs from visual areas V4, V3, and V2. We asked how these different anatomical pathways shape PIT response properties by deactivating them while monitoring PIT activity in two male macaques. We found that cooling of V4 or V2|3 did not lead to consistent changes in population excitatory drive; however, population pattern analyses showed that V4-based pathways were more important than V2|3-based pathways. We did not find any image features that predicted decoding accuracy differences between both interventions. Using the HMAX hierarchical model of visual recognition, we found that different groups of simulated "PIT" units with different input histories (lacking "V2|3" or "V4" input) allowed for comparable levels of object-decoding performance and that removing a large fraction of "PIT" activity resulted in similar drops in performance as in the cooling experiments. We conclude that distinct input pathways to PIT relay similar types of shape information, with V1-dependent V4 cells providing more quantitatively useful information for overall encoding than cells in V2 projecting directly to PIT.SIGNIFICANCE STATEMENT Convolutional neural networks are the best models of the visual system, but most emphasize input transformations across a serial hierarchy akin to the primary "ventral stream" (V1 → V2 → V4 → IT). However, the ventral stream also comprises parallel "bypass" pathways: V1 also connects to V4, and V2 to IT. To explore the advantages of mixing long and short pathways in the macaque brain, we used cortical cooling to silence inputs to posterior IT and compared the findings with an HMAX model with parallel pathways.

End-Stopping Predicts Curvature Tuning along the Ventral Stream.

Neurons in primate inferotemporal cortex (IT) are clustered into patches of shared image preferences. Functional imaging has shown that these patches are activated by natural categories (e.g., faces, body parts, and places), artificial categories (numerals, words) and geometric features (curvature and real-world size). These domains develop in the same cortical locations across monkeys and humans, which raises the possibility of common innate mechanisms. Although these commonalities could be high-level template-based categories, it is alternatively possible that the domain locations are constrained by low-level properties such as end-stopping, eccentricity, and the shape of the preferred images. To explore this, we looked for correlations among curvature preference, receptive field (RF) end-stopping, and RF eccentricity in the ventral stream. We recorded from sites in V1, V4, and posterior IT (PIT) from six monkeys using microelectrode arrays. Across all visual areas, we found a tendency for end-stopped sites to prefer curved over straight contours. Further, we found a progression in population curvature preferences along the visual hierarchy, where, on average, V1 sites preferred straight Gabors, V4 sites preferred curved stimuli, and many PIT sites showed a preference for curvature that was concave relative to fixation. Our results provide evidence that high-level functional domains may be mapped according to early rudimentary properties of the visual system. SIGNIFICANCE STATEMENT: The macaque occipitotemporal cortex contains clusters of neurons with preferences for categories such as faces, body parts, and places. One common question is how these clusters (or "domains") acquire their cortical position along the ventral stream. We and other investigators previously established an fMRI-level correlation among these category domains, retinotopy, and curvature preferences: for example, in inferotemporal cortex, face- and curvature-preferring domains show a central visual field bias whereas place- and rectilinear-preferring domains show a more peripheral visual field bias. Here, we have found an electrophysiological-level explanation for the correlation among domain preference, curvature, and retinotopy based on neuronal preference for short over long contours, also called end-stopping.

Focused ultrasound induced opening of the blood-brain barrier disrupts inter-hemispheric resting state functional connectivity in the rat brain.

Focused ultrasound (FUS) is a technology capable of delivering therapeutic levels of energy through the intact skull to a tightly localized brain region. Combining the FUS pressure wave with intravenously injected microbubbles creates forces on blood vessel walls that open the blood-brain barrier (BBB). This noninvasive and localized opening of the BBB allows for targeted delivery of pharmacological agents into the brain for use in therapeutic development. It is possible to use FUS power levels such that the BBB is opened without damaging local tissues. However, open questions remain related to the effects that FUS-induced BBB opening has on brain function including local physiology and vascular hemodynamics. We evaluated the effects that FUS-induced BBB opening has on resting state functional magnetic resonance imaging (rs-fMRI) metrics. Data from rs-fMRI was acquired in rats that underwent sham FUS BBB vs. FUS BBB opening targeted to the right primary somatosensory cortex hindlimb region (S1HL). FUS BBB opening reduced the functional connectivity between the right S1HL and other sensorimotor regions, including statistically significant reduction of connectivity to the homologous region in the left hemisphere (left S1HL). The effect was observed in all three metrics analyzed: functional connectivity between anatomically defined regions, whole brain voxel-wise correlation maps based on anatomical seeds, and spatial patterns from independent component analysis. Connectivity metrics for other regions where the BBB was not perturbed were not affected. While it is not clear whether the effect is vascular or neuronal in origin, these results suggest that even safe levels of FUS BBB opening have an effect on the physiological processes that drive the signals measured by BOLD fMRI. As such these effects must be accounted for when carrying out studies using fMRI to evaluate the effects of pharmacological agents delivered via FUS-induced BBB opening.

Codeine use among children in the United States: a nationally representative study from 1996 to 2013.

BACKGROUND: Concerns regarding the safety of codeine have been raised. Cases of life-threatening respiratory depression and death in children have been attributed to codeine's polymorphic metabolic pathway. International health agencies recommend restricted use of codeine in children. Despite these recommendations, the epidemiology of codeine use among children remains unknown. AIMS: Our objective was to examine patterns of codeine use in the US among children. METHODS: A cross-sectional analysis of children of age 0-17 years from 1996 to 2013 in the US was performed. Data were extracted from MEPS, a nationally representative set of health care surveys. Prevalence rates of codeine use between 1996 and 2013 were examined. Multivariable logistic regression examined relationships between codeine use and patient demographics. RESULTS: Codeine use remained largely unchanged from 1996 to 2013 (1.08 vs 1.03 million children, respectively). Odds of codeine use was higher in ages 12-17 (OR, 1.40; [1.21-1.61]), outside of the Northeastern US, and among those with poor physical health status (OR, 3.29 [1.79-6.03]). Codeine use was lower in children whose ethnicity was not white and those uninsured (OR, 0.47 [0.34-0.63]). Codeine was most frequently prescribed by emergency physicians (18%) and dentists (14%). The most common condition associated with codeine use was trauma-related pain. CONCLUSIONS: Pediatric codeine use has declined since 1996; however, more than 1 million children still used codeine in 2013. Health care providers must be made aware of guidelines advising against the use of codeine in children. Codeine is potentially hazardous and safer alternatives to treat children's pain are available.

Symbol addition by monkeys provides evidence for normalized quantity coding.

Weber's law can be explained either by a compressive scaling of sensory response with stimulus magnitude or by a proportional scaling of response variability. These two mechanisms can be distinguished by asking how quantities are added or subtracted. We trained Rhesus monkeys to associate 26 distinct symbols with 0-25 drops of reward, and then tested how they combine, or add, symbolically represented reward magnitude. We found that they could combine symbolically represented magnitudes, and they transferred this ability to a novel symbol set, indicating that they were performing a calculation, not just memorizing the value of each combination. The way they combined pairs of symbols indicated neither a linear nor a compressed scale, but rather a dynamically shifting, relative scaling.

Hemodynamic Stability During Pheochromocytoma Resection: Lessons Learned Over the Last Two Decades.

BACKGROUND: Ideal perioperative management of pheochromocytomas/paragangliomas (pheo) is a subject of debate and can be highly variable. The purpose of this study was to identify potential predictive factors of hemodynamic instability during pheo resection. METHODS: A retrospective review of pheo resections from 1992 to 2013 was undertaken. Intraoperative hemodynamics, patient demographics, tumor characteristics, and perioperative management were examined. Postoperative intensive-care admission, myocardial infarction, stroke, and 30-day mortality were reviewed. Linear regression was used to analyze factors influencing intraoperative hemodynamics. RESULTS: During the 20-year study period, 100 patients underwent pheo resection. Postoperative morbidity and mortality was significantly reduced (p = 0.003) in the last 10 years of practice, and there was a trend towards greater morbidity and mortality with intraoperative hemodynamic instability (p = 0.06). The preoperative dose of phenoxybenzamine and the number of laparoscopic procedures has increased in the last decade [59 mg (95 % CI 32-108) to 106 mg (95 % CI 91-124), p = 0.008, and 27 vs. 54 %, p = 0.05, respectively]. Increased preoperative phenoxybenzamine dose was a significant predictor of improved intraoperative hemodynamic stability (p = 0.01). Lack of intraoperative magnesium use resulted in greater hemodynamic instability as preoperative systolic blood pressure increased (p = 0.002). CONCLUSIONS: Postoperative outcomes following pheo resection have improved over the last two decades. Preoperative α-blockade plays a significant role in improving intraoperative hemodynamics and post-op outcomes. Increased doses of phenoxybenzamine and utilization of laparoscopic approaches have likely contributed to improved outcomes in the last decade. Intraoperative magnesium use may provide protection against hemodynamic instability and warrants further study.

Feature-selective responses in macaque visual cortex follow eye movements during natural vision.

In natural vision, primates actively move their eyes several times per second via saccades. It remains unclear whether, during this active looking, visual neurons exhibit classical retinotopic properties, anticipate gaze shifts or mirror the stable quality of perception, especially in complex natural scenes. Here, we let 13 monkeys freely view thousands of natural images across 4.6 million fixations, recorded 883 h of neuronal responses in six areas spanning primary visual to anterior inferior temporal cortex and analyzed spatial, temporal and featural selectivity in these responses. Face neurons tracked their receptive field contents, indicated by category-selective responses. Self-consistency analysis showed that general feature-selective responses also followed eye movements and remained gaze-dependent over seconds of viewing the same image. Computational models of feature-selective responses located retinotopic receptive fields during free viewing. We found limited evidence for feature-selective predictive remapping and no viewing-history integration. Thus, ventral visual neurons represent the world in a predominantly eye-centered reference frame during natural vision.