Going beyond still images to improve input variance resilience in multi-stream vision understanding models.

Traditionally, vision models have predominantly relied on spatial features extracted from static images, deviating from the continuous stream of spatiotemporal features processed by the brain in natural vision. While numerous video-understanding models have emerged, incorporating videos into image-understanding models with spatiotemporal features has been limited. Drawing inspiration from natural vision, which exhibits remarkable resilience to input changes, our research focuses on the development of a brain-inspired model for vision understanding trained with videos. Our findings demonstrate that models that train on videos instead of still images and include temporal features become more resilient to various alternations on input media.

A low-cost protocol for reconditioning of deep-brain neural microelectrodes with material failure for electrophysiology recording.

To date, a myriad of neural microelectrodes has been meticulously developed, but the focus of existing literature predominantly revolves around fabrication methodologies rather than delving into the reconditioning processes or strategies for salvaging electrodes exhibiting diminished performance due to material failure. This study aims to elucidate the underlying factors contributing to the degradation in performance of neural microelectrodes. Additionally, it introduces a comprehensive, cost-effective protocol for the reconditioning and repurposing of electrodes afflicted by material failure, tailored for a broad spectrum of electrode types. The efficacy of the proposed reconditioning protocol is substantiated through experimental validation on single-site tungsten microelectrodes. The results of neural signal recording unequivocally demonstrate the successful restoration of a substantial number of electrodes, underscoring the protocol's effectiveness.

A framework and resource for global collaboration in non-human primate neuroscience.

As science and technology evolve, there is an increasing need for promotion of international scientific exchange. Collaborations, while offering substantial opportunities for scientists and benefit to society, also present challenges for those working with animal models, such as non-human primates (NHPs). Diversity in regulation of animal research is sometimes mistaken for the absence of common international welfare standards. Here, the ethical and regulatory protocols for 13 countries that have guidelines in place for biomedical research involving NHPs were assessed with a focus on neuroscience. Review of the variability and similarity in trans-national NHP welfare regulations extended to countries in Asia, Europe and North America. A tabulated resource was established to advance solution-oriented discussions and scientific collaborations across borders. Our aim is to better inform the public and other stakeholders. Through cooperative efforts to identify and analyze information with reference to evidence-based discussion, the proposed key ingredients may help to shape and support a more informed, open framework. This framework and resource can be expanded further for biomedical research in other countries.

The variation of functional connectivity and activity before and after thalamotomy surgery (review).

Ablation surgeries are utilized to treat certain brain disorders. Recently, these surgeries have become more prevalent using techniques such as magnetic resonance guided focused ultrasound (MRgFUS) ablation and Gamma knife thalamotomy (GKT). However, as the thalamus plays a critical role in cognitive functions, the potential impact of these surgeries on functional connectivity and cognition is a matter of concern. Various approaches have been developed to locate the target for ablation and also investigate changes in functional connectivity before and after surgery. Functional magnetic resonance imaging (fMRI) and electroencephalogram (EEG) are widely used methods for assessing changes in functional connectivity and activity in clinical research. In this Review, we summarize the use of fMRI and EEG in thalamotomy surgeries. Our analysis shows that thalamotomy surgery can result in changes in functional connectivity in motor-related, visuomotor, and default-mode networks, as detected by fMRI. EEG data also indicate a reduction in over-activities observed in the preoperative state.

A multimodal imaging-guided software for access to primate brains.

Background: Imaging-guided access to the brain has become a routine procedure for various research and clinical applications, including drug administration, neurophysiological recording, and sampling tissue. Therefore, open-source software is required to handle such datasets in these specific applications. New methods: Here, we proposed an open-source tool utilizing different imaging modalities for automating the steps to access the brain. This tool provides means for easily calculating the coordination of the area of interest concerning a specific point of entry. The source and documentation are available at this link. Results: We have used this software for three different applications: electrophysiological recording, drug infusion in the nonhuman primate brain, and guided biopsy procedure in the human brain. We performed a neural recording of two monkeys' prefrontal cortex and inferior temporal cortex using this software in submillimeter resolution. We also applied our procedure for infusion in the putamen and caudate nuclei in both hemispheres of another group of rhesus monkeys with histological proof in one animal. More so, we validated this software in the human subjects that underwent biopsy surgery with the commercial software used in human biopsy surgery. Comparison with existing methods: Our software uses different imaging modalities by co-registering them. This will provide structural details of the skull and brain tissue. We can calculate each brain region's coordination at the point of entry by re-slicing the images. Atlas-based image segmentation were implemented in our software. Three mentioned applications of our software in neuroscience will be further discussed in this paper. Conclusion: In our procedure, working with different imaging modalities provides a precise estimation of the specific region in the brain related to the location of implants or stereotaxic frames. There is no limitation to using metal implants in this procedure.

Neural signature of the perceptual decision in the neural population responses of the inferior temporal cortex.

Rapid categorization of visual objects is critical for comprehending our complex visual world. The role of individual cortical neurons and neural populations in categorizing visual objects during passive vision has previously been studied. However, it is unclear whether and how perceptually guided behaviors affect the encoding of stimulus categories by neural population activity in the higher visual cortex. Here we studied the activity of the inferior temporal (IT) cortical neurons in macaque monkeys during both passive viewing and categorization of ambiguous body and object images. We found enhanced category information in the IT neural population activity during the correct, but not wrong, trials of the categorization task compared to the passive task. This encoding enhancement was task difficulty dependent with progressively larger values in trials with more ambiguous stimuli. Enhancement of IT neural population information for behaviorally relevant stimulus features suggests IT neural networks' involvement in perceptual decision-making behavior.

An automatic spike sorting algorithm based on adaptive spike detection and a mixture of skew-t distributions.

Developing high-density electrodes for recording large ensembles of neurons provides a unique opportunity for understanding the mechanism of the neuronal circuits. Nevertheless, the change of brain tissue around chronically implanted neural electrodes usually causes spike wave-shape distortion and raises the crucial issue of spike sorting with an unstable structure. The automatic spike sorting algorithms have been developed to extract spikes from these big extracellular data. However, due to the spike wave-shape instability, there have been a lack of robust spike detection procedures and clustering to overcome the spike loss problem. Here, we develop an automatic spike sorting algorithm based on adaptive spike detection and a mixture of skew-t distributions to address these distortions and instabilities. The adaptive detection procedure applies to the detected spikes, consists of multi-point alignment and statistical filtering for removing mistakenly detected spikes. The detected spikes are clustered based on the mixture of skew-t distributions to deal with non-symmetrical clusters and spike loss problems. The proposed algorithm improves the performance of the spike sorting in both terms of precision and recall, over a broad range of signal-to-noise ratios. Furthermore, the proposed algorithm has been validated on different datasets and demonstrates a general solution to precise spike sorting, in vitro and in vivo.

Frontotemporal coordination predicts working memory performance and its local neural signatures.

Neurons in some sensory areas reflect the content of working memory (WM) in their spiking activity. However, this spiking activity is seldom related to behavioral performance. We studied the responses of inferotemporal (IT) neurons, which exhibit object-selective activity, along with Frontal Eye Field (FEF) neurons, which exhibit spatially selective activity, during the delay period of an object WM task. Unlike the spiking activity and local field potentials (LFPs) within these areas, which were poor predictors of behavioral performance, the phase-locking of IT spikes and LFPs with the beta band of FEF LFPs robustly predicted successful WM maintenance. In addition, IT neurons exhibited greater object-selective persistent activity when their spikes were locked to the phase of FEF LFPs. These results reveal that the coordination between prefrontal and temporal cortex predicts the successful maintenance of visual information during WM.

Dependence of Working Memory on Coordinated Activity Across Brain Areas.

Neural signatures of working memory (WM) have been reported in numerous brain areas, suggesting a distributed neural substrate for memory maintenance. In the current manuscript we provide an updated review of the literature focusing on intracranial neurophysiological recordings during WM in primates. Such signatures of WM include changes in firing rate or local oscillatory power within an area, along with measures of coordinated activity between areas based on synchronization between oscillations. In comparing the ability of various neural signatures in any brain area to predict behavioral performance, we observe that synchrony between areas is more frequently and robustly correlated with WM performance than any of the within-area neural signatures. We further review the evidence for alteration of inter-areal synchrony in brain disorders, consistent with an important role for such synchrony during behavior. Additionally, results of causal studies indicate that manipulating synchrony across areas is especially effective at influencing WM task performance. Each of these lines of research supports the critical role of inter-areal synchrony in WM. Finally, we propose a framework for interactions between prefrontal and sensory areas during WM, incorporating a range of experimental findings and offering an explanation for the observed link between intra-areal measures and WM performance.

Selective Changes in Noise Correlations Contribute to an Enhanced Representation of Saccadic Targets in Prefrontal Neuronal Ensembles.

ABSTARCT: An ensemble of neurons can provide a dynamic representation of external stimuli, ongoing processes, or upcoming actions. This dynamic representation could be achieved by changes in the activity of individual neurons and/or their interactions. To investigate these possibilities, we simultaneously recorded from ensembles of prefrontal neurons in non-human primates during a memory-guided saccade task. Using both decoding and encoding methods, we examined changes in the information content of individual neurons and that of ensembles between visual encoding and saccadic target selection. We found that individual neurons maintained their limited spatial sensitivity between these cognitive states, whereas the ensemble selectively improved its encoding of spatial locations far from the neurons' preferred locations. This population-level "encoding expansion" was not due to the ceiling effect at the preferred locations and was accompanied by selective changes in noise correlations for non-preferred locations. Moreover, the encoding expansion was observed for ensembles of different types of neurons and could not be explained by shifts in the preferred location of individual neurons. Our results demonstrate that the representation of space by neuronal ensembles is dynamically enhanced prior to saccades, and this enhancement occurs alongside changes in noise correlations more than changes in the activity of individual neurons.

Rapid face adaptation distributes representation in inferior-temporal cortex across time and neuronal dimensions.

Neuronal networks of the brain adapt their information processing according to the history of stimuli. Whereas most studies have linked adaptation to repetition suppression, recurrent connections within a network and disinhibition due to adaptation predict more complex response patterns. The main questions of this study are as follows: what is the effect of the selectivity of neurons on suppression/enhancement of neural responses? What are the consequences of adaptation on information representation in neural population and the temporal structure of response patterns? We studied rapid face adaptation using spiking activities of neurons in the inferior-temporal (IT) cortex. Investigating the responses of neurons, within a wide range from negative to positive face selectivity, showed that despite the peak amplitude suppression in highly positive selective neurons, responses were enhanced in most other neurons. This enhancement can be attributed to disinhibition due to adaptation. Delayed and distributed responses were observed for positive selective neurons. Principal component analysis of the IT population responses over time revealed that repetition of face stimuli resulted in temporal decorrelation of the network activity. The contributions of the main and higher neuronal dimensions were changed under an adaptation condition, where more neuronal dimensions were used to encode repeated face stimuli.

Low dimensional representation of face space by face-selective inferior temporal neurons.

The representation of visual objects in primate brain is distributed and multiple neurons are involved in encoding each object. One way to understand the neural basis of object representation is to estimate the number of neural dimensions that are needed for veridical representation of object categories. In this study, the characteristics of the match between physical-shape and neural representational spaces in monkey inferior temporal (IT) cortex were evaluated. Specifically, we examined how the number of neural dimensions, stimulus behavioral saliency and stimulus category selectivity of neurons affected the correlation between shape and neural representational spaces in IT cortex. Single-unit recordings from monkey IT cortex revealed that there was a significant match between face space and its neural representation at lower neural dimensions, whereas the optimal match for the non-face objects was observed at higher neural dimensions. There was a statistically significant match between the face and neural spaces only in the face-selective neurons, whereas a significant match was observed for non-face objects in all neurons regardless of their category selectivity. Interestingly, the face neurons showed a higher match for the non-face objects than for the faces at higher neural dimensions. The optimal representation of face space in the responses of the face neurons was a low dimensional map that emerged early (~150 ms post-stimulus onset) and was followed by a high dimensional and relatively late (~300 ms) map for the non-face stimuli. These results support a multiplexing function for the face neurons in the representation of very similar shape spaces, but with different dimensionality and timing scales.

Impairment of perceptual closure in autism for vertex- but not edge-defined object images.

One of the characteristics of autism spectrum disorder (ASD) is atypical sensory processing and perceptual integration. Here, we used an object naming task to test the significance of deletion of vertices versus extended contours (edges) in naming fragmented line drawings of natural objects in typically developing and ASD children. The basic components of a fragmented image in perceptual closure need to be integrated to make a coherent visual perception. When vertices were missing and only edges were visible, typically developing and ASD subjects performed similarly. But typically developing children performed significantly better than ASD children when only vertex information was visible. These results indicate impairment of binding vertices but not edges to form a holistic representation of an object in children with ASD.

Temporal dynamics of visual category representation in the macaque inferior temporal cortex.

Object categories are recognized at multiple levels of hierarchical abstractions. Psychophysical studies have shown a more rapid perceptual access to the mid-level category information (e.g., human faces) than the higher (superordinate; e.g., animal) or the lower (subordinate; e.g., face identity) level. Mid-level category members share many features, whereas few features are shared among members of different mid-level categories. To understand better the neural basis of expedited access to mid-level category information, we examined neural responses of the inferior temporal (IT) cortex of macaque monkeys viewing a large number of object images. We found an earlier representation of mid-level categories in the IT population and single-unit responses compared with superordinate- and subordinate-level categories. The short-latency representation of mid-level category information shows that visual cortex first divides the category shape space at its sharpest boundaries, defined by high/low within/between-group similarity. This short-latency, mid-level category boundary map may be a prerequisite for representation of other categories at more global and finer scales.