Bidirectional Generative Adversarial Representation Learning for Natural Stimulus Synthesis.

Thousands of species use vocal signals to communicate with one another.Vocalisations carry rich information, yet characterising and analysing these high-dimensional signals is difficult and prone to human bias. Moreover, animal vocalisations are ethologically relevant stimuli whose representation by auditory neurons is an important subject of research in sensory neuroscience. A method that can efficiently generate naturalistic vocalisation waveforms would offer an unlimited supply of stimuli to probe neuronal computations. While unsupervised learning methods allow for the projection of vocalisations into low-dimensional latent spaces learned from the waveforms themselves, and generative modelling allows for the synthesis of novel vocalisations for use in downstream tasks, there is currently no method that would combine these tasks to produce naturalistic vocalisation waveforms for stimulus playback. Here, we demonstrate BiWaveGAN: a bidirectional Generative Adversarial Network (GAN) capable of learning a latent representation of ultrasonic vocalisations (USVs) from mice. We show that BiWaveGAN can be used to generate, and interpolate between, realistic vocalisation waveforms. We then use these synthesised stimuli along with natural USVs to probe the sensory input space of mouse auditory cortical neurons. We show that stimuli generated from our method evoke neuronal responses as effectively as real vocalisations, and produce receptive fields with the same predictive power. BiWaveGAN is not restricted to mouse USVs but can be used to synthesise naturalistic vocalisations of any animal species and interpolate between vocalisations of the same or different species, which could be useful for probing categorical boundaries in representations of ethologically relevant auditory signals.

Data-driven modelling of visual receptive fields: comparison between the generalized quadratic model and the nonlinear input model.

Objective: Neurons in primary visual cortex (V1) display a range of sensitivity in their response to translations of their preferred visual features within their receptive field: from high specificity to a precise position through to complete invariance. This visual feature selectivity and invariance is frequently modeled by applying a selection of linear spatial filters to the input image, that define the feature selectivity, followed by a nonlinear function that combines the filter outputs, that defines the invariance, to predict the neural response. We compare two such classes of model, that are both popular and parsimonious, the generalized quadratic model (GQM) and the nonlinear input model (NIM). These two classes of model differ primarily in that the NIM can accommodate a greater diversity in the form of nonlinearity that is applied to the outputs of the filters.Approach: We compare the two model types by applying them to data from multielectrode recordings from cat primary visual cortex in response to spatially white Gaussian noise After fitting both classes of model to a database of 342 single units (SUs), we analyze the qualitative and quantitative differences in the visual feature processing performed by the two models and their ability to predict neural response.Main results: We find that the NIM predicts response rates on a held-out data at least as well as the GQM for 95% of SUs. Superior performance occurs predominantly for those units with above average spike rates and is largely due to the NIMs ability to capture aspects of the model's nonlinear function cannot be captured with the GQM rather than differences in the visual features being processed by the two different models.Significance: These results can help guide model choice for data-driven receptive field modelling.

Expansive Receptive Field and Local Feature Extraction Network: Advancing Multiscale Feature Fusion for Breast Fibroadenoma Segmentation in Sonography.

Fibroadenoma is a common benign breast disease that affects women of all ages. Early diagnosis can greatly improve the treatment outcomes and reduce the associated pain. Computer-aided diagnosis (CAD) has great potential to improve diagnosis accuracy and efficiency. However, its application in sonography is limited. A network that utilizes expansive receptive fields and local information learning was proposed for the accurate segmentation of breast fibroadenomas in sonography. The architecture comprises the Hierarchical Attentive Fusion module, which conducts local information learning through channel-wise and pixel-wise perspectives, and the Residual Large-Kernel module, which utilizes multiscale large kernel convolution for global information learning. Additionally, multiscale feature fusion in both modules was included to enhance the stability of our network. Finally, an energy function and a data augmentation method were incorporated to fine-tune low-level features of medical images and improve data enhancement. The performance of our model is evaluated using both our local clinical dataset and a public dataset. Mean pixel accuracy (MPA) of 93.93% and 86.06% and mean intersection over union (MIOU) of 88.16% and 73.19% were achieved on the clinical and public datasets, respectively. They are significantly improved over state-of-the-art methods such as SegFormer (89.75% and 78.45% in MPA and 83.26% and 71.85% in MIOU, respectively). The proposed feature extraction strategy, combining local pixel-wise learning with an expansive receptive field for global information perception, demonstrates excellent feature learning capabilities. Due to this powerful and unique local-global feature extraction capability, our deep network achieves superior segmentation of breast fibroadenoma in sonography, which may be valuable in early diagnosis.

Understanding of Spinal Wide Dynamic Range Neurons and Their Modulation on Pathological Pain.

The spinal dorsal horn (SDH) transmits sensory information from the periphery to the brain. Wide dynamic range (WDR) neurons within this relay site play a critical role in modulating and integrating peripheral sensory inputs, as well as the process of central sensitization during pathological pain. This group of spinal multi-receptive neurons has attracted considerable attention in pain research due to their capabilities for encoding the location and intensity of nociception. Meanwhile, transmission, processing, and modulation of incoming afferent information in WDR neurons also establish the underlying basis for investigating the integration of acupuncture and pain signals. This review aims to provide a comprehensive examination of the distinctive features of WDR neurons and their involvement in pain. Specifically, we will examine the regulation of diverse supraspinal nuclei on these neurons and analyze their potential in elucidating the mechanisms of acupuncture analgesia.

Organization of pRF size along the AP axis of the hippocampus and adjacent medial temporal cortex is related to specialization for scenes versus faces.

The hippocampus is largely recognized for its integral contributions to memory processing. By contrast, its role in perceptual processing remains less clear. Hippocampal properties vary along the anterior-posterior (AP) axis. Based on past research suggesting a gradient in the scale of features processed along the AP extent of the hippocampus, the representations have been proposed to vary as a function of granularity along this axis. One way to quantify such granularity is with population receptive field (pRF) size measured during visual processing, which has so far received little attention. In this study, we compare the pRF sizes within the hippocampus to its activation for images of scenes versus faces. We also measure these functional properties in surrounding medial temporal lobe (MTL) structures. Consistent with past research, we find pRFs to be larger in the anterior than in the posterior hippocampus. Critically, our analysis of surrounding MTL regions, the perirhinal cortex, entorhinal cortex, and parahippocampal cortex shows a similar correlation between scene sensitivity and larger pRF size. These findings provide conclusive evidence for a tight relationship between the pRF size and the sensitivity to image content in the hippocampus and adjacent medial temporal cortex.

A neural model of modified excitation/inhibition and feedback levels in schizophrenia.

Introduction: The strength of certain visual illusions, including contrast-contrast and apparent motion, is weakened in individuals with schizophrenia. Such phenomena have been interpreted as the impaired integration of inhibitory and excitatory neural responses, and impaired top-down feedback mechanisms. Methods: To investigate whether and how these factors influence the perceived contrast-contrast and apparent motion illusions in individuals with schizophrenia, we propose a two-layer network, with top-down feedback from layer 2 to layer 1 that can model visual receptive fields (RFs) and their inhibitory and excitatory subfields. Results: Our neural model suggests that illusion perception changes in individuals with schizophrenia can be influenced by altered top-down mechanisms and the organization of the on-center off-surround receptive fields. Alteration of the RF inhibitory surround and/or the excitatory center can replicate the difference of illusion precepts between individuals with schizophrenia within certain clinical states and normal controls. The results show that the simulated top-down feedback modulation enlarges the difference of the model illusion representations, replicating the difference between the two groups. Discussion: We propose that the heterogeneity of visual and in general sensory processing in certain clinical states of schizophrenia can be largely explained by the degree of top-down feedback reduction, emphasizing the critical role of top-down feedback in illusion perception, and to a lesser extent on the imbalance of excitation/inhibition. Our neural model provides a mechanistic explanation for the modulated visual percepts of contrast-contrast and apparent motion in schizophrenia with findings that can explain a broad range of visual perceptual observations in previous studies. The two-layer motif of the current model provides a general framework that can be tailored to investigate subcortico-cortical (such as thalamocortical) and cortico-cortical networks, bridging neurobiological changes in schizophrenia and perceptual processing.

Hierarchical temporal prediction captures motion processing along the visual pathway.

Visual neurons respond selectively to features that become increasingly complex from the eyes to the cortex. Retinal neurons prefer flashing spots of light, primary visual cortical (V1) neurons prefer moving bars, and those in higher cortical areas favor complex features like moving textures. Previously, we showed that V1 simple cell tuning can be accounted for by a basic model implementing temporal prediction - representing features that predict future sensory input from past input (Singer et al., 2018). Here, we show that hierarchical application of temporal prediction can capture how tuning properties change across at least two levels of the visual system. This suggests that the brain does not efficiently represent all incoming information; instead, it selectively represents sensory inputs that help in predicting the future. When applied hierarchically, temporal prediction extracts time-varying features that depend on increasingly high-level statistics of the sensory input.

The dorsal lateral geniculate nucleus and the pulvinar as essential partners for visual cortical functions.

In most neuroscience textbooks, the thalamus is presented as a structure that relays sensory signals from visual, auditory, somatosensory, and gustatory receptors to the cerebral cortex. But the function of the thalamic nuclei goes beyond the simple transfer of information. This is especially true for the second-order nuclei, but also applies to first-order nuclei. First order thalamic nuclei receive information from the periphery, like the dorsal lateral geniculate nucleus (dLGN), which receives a direct input from the retina. In contrast, second order thalamic nuclei, like the pulvinar, receive minor or no input from the periphery, with the bulk of their input derived from cortical areas. The dLGN refines the information received from the retina by temporal decorrelation, thereby transmitting the most "relevant" signals to the visual cortex. The pulvinar is closely linked to virtually all visual cortical areas, and there is growing evidence that it is necessary for normal cortical processing and for aspects of visual cognition. In this article, we will discuss what we know and do not know about these structures and propose some thoughts based on the knowledge gained during the course of our careers. We hope that these thoughts will arouse curiosity about the visual thalamus and its important role, especially for the next generation of neuroscientists.

Characterization of extracellular spike waveforms recorded in wallaby primary visual cortex.

Extracellular recordings were made from 642 units in the primary visual cortex (V1) of a highly visual marsupial, the Tammar wallaby. The receptive field (RF) characteristics of the cells were objectively estimated using the non-linear input model (NIM), and these were correlated with spike shapes. We found that wallaby cortical units had 68% regular spiking (RS), 12% fast spiking (FS), 4% triphasic spiking (TS), 5% compound spiking (CS) and 11% positive spiking (PS). RS waveforms are most often associated with recordings from pyramidal or spiny stellate cell bodies, suggesting that recordings from these cell types dominate in the wallaby cortex. In wallaby, 70-80% of FS and RS cells had orientation selective RFs and had evenly distributed linear and nonlinear RFs. We found that 47% of wallaby PS units were non-orientation selective and they were dominated by linear RFs. Previous studies suggest that the PS units represent recordings from the axon terminals of non-orientation selective cells originating in the lateral geniculate nucleus (LGN). If this is also true in wallaby, as strongly suggested by their low response latencies and bursty spiking properties, the results suggest that significantly more neurons in wallaby LGN are already orientation selective. In wallaby, less than 10% of recorded spikes had triphasic (TS) or sluggish compound spiking (CS) waveforms. These units had a mixture of orientation selective and non-oriented properties, and their cellular origins remain difficult to classify.

Two-Dimensional Population Receptive Field Mapping of Human Primary Somatosensory Cortex.

Functional magnetic resonance imaging can provide detailed maps of how sensory space is mapped in the human brain. Here, we use a novel 16 stimulator setup (a 4 × 4 grid) to measure two-dimensional sensory maps of between and within-digit (D2-D4) space using high spatial-resolution (1.25 mm isotropic) imaging at 7 Tesla together with population receptive field (pRF) mapping in 10 participants. Using a 2D Gaussian pRF model, we capture maps of the coverage of digits D2-D5 across Brodmann areas and estimate pRF size and shape. In addition, we compare results to previous studies that used fewer stimulators by constraining pRF models to a 1D Gaussian Between Digit or 1D Gaussian Within Digit model. We show that pRFs across somatosensory areas tend to have a strong preference to cover the within-digit axis. We show an increase in pRF size moving from D2-D5. We quantify pRF shapes in Brodmann area (BA) 3b, 3a, 1, 2 and show differences in pRF size in Brodmann areas 3a-2, with larger estimates for BA2. Generally, the 2D Gaussian pRF model better represents pRF coverage maps generated by our data, which itself is produced from a 2D stimulation grid.

Retinotopic connectivity maps of human visual cortex with unconstrained eye movements.

Human visual cortex contains topographic visual field maps whose organization can be revealed with retinotopic mapping. Unfortunately, constraints posed by standard mapping hinder its use in patients, atypical subject groups, and individuals at either end of the lifespan. This severely limits the conclusions we can draw about visual processing in such individuals. Here, we present a novel data-driven method to estimate connective fields, resulting in fine-grained maps of the functional connectivity between brain areas. We find that inhibitory connectivity fields accompany, and often surround facilitatory fields. The visual field extent of these inhibitory subfields falls off with cortical magnification. We further show that our method is robust to large eye movements and myopic defocus. Importantly, freed from the controlled stimulus conditions in standard mapping experiments, using entertaining stimuli and unconstrained eye movements our approach can generate retinotopic maps, including the periphery visual field hitherto only possible to map with special stimulus displays. Generally, our results show that the connective field method can gain knowledge about retinotopic architecture of visual cortex in patients and participants where this is at best difficult and confounded, if not impossible, with current methods.

New biophysical rate-based modeling of long-term plasticity in mean-field neuronal population models.

The formation of customized neural networks as the basis of brain functions such as receptive field selectivity, learning or memory depends heavily on the long-term plasticity of synaptic connections. However, the current mean-field population models commonly used to simulate large-scale neural network dynamics lack explicit links to the underlying cellular mechanisms of long-term plasticity. In this study, we developed a new mean-field population model, the plastic density-based neural mass model (pdNMM), by incorporating a newly developed rate-based plasticity model based on the calcium control hypothesis into an existing density-based neural mass model. Derivation of the plasticity model was carried out using population density methods. Our results showed that the synaptic plasticity represented by the resulting rate-based plasticity model exhibited Bienenstock-Cooper-Munro-like learning rules. Furthermore, we demonstrated that the pdNMM accurately reproduced previous experimental observations of long-term plasticity, including characteristics of Hebbian plasticity such as longevity, associativity and input specificity, on hippocampal slices, and the formation of receptive field selectivity in the visual cortex. In conclusion, the pdNMM is a novel approach that can confer long-term plasticity to conventional mean-field neuronal population models.

Online spike-based recognition of digits with ultrafast microlaser neurons.

Classification and recognition tasks performed on photonic hardware-based neural networks often require at least one offline computational step, such as in the increasingly popular reservoir computing paradigm. Removing this offline step can significantly improve the response time and energy efficiency of such systems. We present numerical simulations of different algorithms that utilize ultrafast photonic spiking neurons as receptive fields to allow for image recognition without an offline computing step. In particular, we discuss the merits of event, spike-time and rank-order based algorithms adapted to this system. These techniques have the potential to significantly improve the efficiency and effectiveness of optical classification systems, minimizing the number of spiking nodes required for a given task and leveraging the parallelism offered by photonic hardware.

Partial inactivation of songbird auditory cortex impairs both tempo and pitch discrimination.

Neuronal tuning for spectral and temporal features has been studied extensively in the auditory system. In the auditory cortex, diverse combinations of spectral and temporal tuning have been found, but how specific feature tuning contributes to the perception of complex sounds remains unclear. Neurons in the avian auditory cortex are spatially organized in terms of spectral or temporal tuning widths, providing an opportunity for investigating the link between auditory tuning and perception. Here, using naturalistic conspecific vocalizations, we asked whether subregions of the auditory cortex that are tuned for broadband sounds are more important for discriminating tempo than pitch, due to the lower frequency selectivity. We found that bilateral inactivation of the broadband region impairs performance on both tempo and pitch discrimination. Our results do not support the hypothesis that the lateral, more broadband subregion of the songbird auditory cortex contributes more to processing temporal than spectral information.

Functional organization of visual responses in the octopus optic lobe.

Cephalopods are highly visual animals with camera-type eyes, large brains, and a rich repertoire of visually guided behaviors. However, the cephalopod brain evolved independently from those of other highly visual species, such as vertebrates; therefore, the neural circuits that process sensory information are profoundly different. It is largely unknown how their powerful but unique visual system functions, as there have been no direct neural measurements of visual responses in the cephalopod brain. In this study, we used two-photon calcium imaging to record visually evoked responses in the primary visual processing center of the octopus central brain, the optic lobe, to determine how basic features of the visual scene are represented and organized. We found spatially localized receptive fields for light (ON) and dark (OFF) stimuli, which were retinotopically organized across the optic lobe, demonstrating a hallmark of visual system organization shared across many species. An examination of these responses revealed transformations of the visual representation across the layers of the optic lobe, including the emergence of the OFF pathway and increased size selectivity. We also identified asymmetries in the spatial processing of ON and OFF stimuli, which suggest unique circuit mechanisms for form processing that may have evolved to suit the specific demands of processing an underwater visual scene. This study provides insight into the neural processing and functional organization of the octopus visual system, highlighting both shared and unique aspects, and lays a foundation for future studies of the neural circuits that mediate visual processing and behavior in cephalopods.

Mixed Receptive Fields Augmented YOLO with Multi-Path Spatial Pyramid Pooling for Steel Surface Defect Detection.

Aiming at the problems of low detection efficiency and poor detection accuracy caused by texture feature interference and dramatic changes in the scale of defect on steel surfaces, an improved YOLOv5s model is proposed. In this study, we propose a novel re-parameterized large kernel C3 module, which enables the model to obtain a larger effective receptive field and improve the ability of feature extraction under complex texture interference. Moreover, we construct a feature fusion structure with a multi-path spatial pyramid pooling module to adapt to the scale variation of steel surface defects. Finally, we propose a training strategy that applies different kernel sizes for feature maps of different scales so that the receptive field of the model can adapt to the scale changes of the feature maps to the greatest extent. The experiment on the NEU-DET dataset shows that our model improved the detection accuracy of crazing and rolled in-scale, which contain a large number of weak texture features and are densely distributed by 14.4% and 11.1%, respectively. Additionally, the detection accuracy of inclusion and scratched defects with prominent scale changes and significant shape features was improved by 10.5% and 6.6%, respectively. Meanwhile, the mean average precision value reaches 76.8%, compared with the YOLOv5s and YOLOv8s, which increased by 8.6% and 3.7%, respectively.

A neural model of modified excitation/inhibition and feedback levels in schizophrenia.

The strength of certain visual illusions is weakened in individuals with schizophrenia. Such phenomena have been interpreted as the impaired integration of inhibitory and excitatory neural responses, and impaired top-down feedback mechanisms. To investigate whether and how these factors influence the perceived illusions in individuals with schizophrenia, we propose a two-layer network that can model visual receptive fields (RFs), their inhibitory and excitatory subfields, and the top-down feedback. Our neural model suggests that illusion perception changes in individuals with schizophrenia can be influenced by altered top-down mechanisms and the organization of the on-center off-surround receptive fields. Alteration of the RF inhibitory surround and/or the excitatory center can replicate the difference of illusion precepts between individuals with schizophrenia and normal controls. The results show that the simulated top-down feedback modulation enlarges the difference of the model illusion representations, replicating the difference between the two groups. We propose that the heterogeneity of visual and in general sensory processing in schizophrenia can be largely explained by the degree of top-down feedback reduction, emphasizing the critical role of top-down feedback in illusion perception, and to a lesser extent on the imbalance of excitation/inhibition. Our neural model provides a mechanistic explanation for the modulated visual percepts in schizophrenia with findings that can explain a broad range of visual perceptual observations in previous studies. The two-layer motif of the current model provides a general framework that can be tailored to investigate subcortico-cortical (such as thalamocortical) and cortico-cortical networks, bridging neurobiological changes in schizophrenia and perceptual processing.

All-in-one aerial image enhancement network for forest scenes.

Drone monitoring plays an irreplaceable and significant role in forest firefighting due to its characteristics of wide-range observation and real-time messaging. However, aerial images are often susceptible to different degradation problems before performing high-level visual tasks including but not limited to smoke detection, fire classification, and regional localization. Recently, the majority of image enhancement methods are centered around particular types of degradation, necessitating the memory unit to accommodate different models for distinct scenarios in practical applications. Furthermore, such a paradigm requires wasted computational and storage resources to determine the type of degradation, making it difficult to meet the real-time and lightweight requirements of real-world scenarios. In this paper, we propose an All-in-one Image Enhancement Network (AIENet) that can restore various degraded images in one network. Specifically, we design a new multi-scale receptive field image enhancement block, which can better reconstruct high-resolution details of target regions of different sizes. In particular, this plug-and-play module enables it to be embedded in any learning-based model. And it has better flexibility and generalization in practical applications. This paper takes three challenging image enhancement tasks encountered in drone monitoring as examples, whereby we conduct task-specific and all-in-one image enhancement experiments on a synthetic forest dataset. The results show that the proposed AIENet outperforms the state-of-the-art image enhancement algorithms quantitatively and qualitatively. Furthermore, extra experiments on high-level vision detection also show the promising performance of our method compared with some recent baselines.

The Effect of Pre-Electroacupuncture on Nociceptive Discharges of Spinal Wide Dynamic Range Neurons in Rat.

Purpose: Spinal wide dynamic range (WDR) neurons are well studied in pain models and they play critical roles in regulating nociception. Evidence has started to accumulate that acupuncture produces a good analgesic effect via activating different primary fibers with distinct intensities. The purpose of the present study was to compare the distinct intensities of pre-electroacupuncture (pre-EA) at local muscular receptive fields (RFs), adjacent or contralateral non-RFs regulating the nociceptive discharges of spinal WDR neurons evoked by hypertonic saline (HS). Materials and Methods: Spinal segments of electrophysiological recording were identified by neural tracers applied at the left gastrocnemius muscle. The thresholds of Aß (TAß), Aδ (TAδ) and C (TC) components of WDR neurons were measured to determine the intensity of pre-EA by extracellular recording. The discharges of WDR neurons induced by distinct intensities of pre-EA and 200 µL HS (6%) injection in left gastrocnemius muscle of rats were observed by extracellular recording. Results: The spinal segments of WDR neurons were confirmed in lumbar (L)5-6 area according to the projective segments of dorsal root ganglion. TAß, TAδ and TC of WDR neurons was determined to be 0.5, 1, and 2 mA, respectively. The pre-EA with intensities of TAß (P < 0.05), TAδ (P < 0.05), TC (P < 0.05) or 2TC (P < 0.01) at ipsilateral adjacent non-RFs significantly reduced the discharges of WDR neurons, while at local RFs only pre-EA of TAδ (P < 0.05), TC (P < 0.05) and 2TC (P < 0.01) could inhibit the nociceptive discharges. In addition, intensity of pre-EA at contralateral non-RFs should reach at least TC to effectively inhibit the firing rates of WDR neurons (P < 0.01). Conclusion: Pre-EA could suppress nociceptive discharges of WDR neurons and the inhibitory effects were dependent on the distinct intensities and locations of stimulation.

Skin lesion segmentation using two-phase cross-domain transfer learning framework.

BACKGROUND AND OBJECTIVE: Deep learning (DL) models have been used for medical imaging for a long time but they did not achieve their full potential in the past because of insufficient computing power and scarcity of training data. In recent years, we have seen substantial growth in DL networks because of improved technology and an abundance of data. However, previous studies indicate that even a well-trained DL algorithm may struggle to generalize data from multiple sources because of domain shifts. Additionally, ineffectiveness of basic data fusion methods, complexity of segmentation target and low interpretability of current DL models limit their use in clinical decisions. To meet these challenges, we present a new two-phase cross-domain transfer learning system for effective skin lesion segmentation from dermoscopic images. METHODS: Our system is based on two significant technical inventions. We examine a two- phase cross-domain transfer learning approach, including model-level and data-level transfer learning, by fine-tuning the system on two datasets, MoleMap and ImageNet. We then present nSknRSUNet, a high-performing DL network, for skin lesion segmentation using broad receptive fields and spatial edge attention feature fusion. We examine the trained model's generalization capabilities on skin lesion segmentation to quantify these two inventions. We cross-examine the model using two skin lesion image datasets, MoleMap and HAM10000, obtained from varied clinical contexts. RESULTS: At data-level transfer learning for the HAM10000 dataset, the proposed model obtained 94.63% of DSC and 99.12% accuracy. In cross-examination at data-level transfer learning for the Molemap dataset, the proposed model obtained 93.63% of DSC and 97.01% of accuracy. CONCLUSION: Numerous experiments reveal that our system produces excellent performance and improves upon state-of-the-art methods on both qualitative and quantitative measures.