Grape clusters detection based on multi-scale feature fusion and augmentation.

This paper addresses the challenge of low detection accuracy of grape clusters caused by scale differences, illumination changes, and occlusion in realistic and complex scenes. We propose a multi-scale feature fusion and augmentation YOLOv7 network to enhance the detection accuracy of grape clusters across variable environments. First, we design a Multi-Scale Feature Extraction Module (MSFEM) to enhance feature extraction for small-scale targets. Second, we propose the Receptive Field Augmentation Module (RFAM), which uses dilated convolution to expand the receptive field and enhance the detection accuracy for objects of various scales. Third, we present the Spatial Pyramid Pooling Cross Stage Partial Concatenation Faster (SPPCSPCF) module to fuse multi-scale features, improving accuracy and speeding up model training. Finally, we integrate the Residual Global Attention Mechanism (ResGAM) into the network to better focus on crucial regions and features. Experimental results show that our proposed method achieves a mAP 0.5 of 93.29% on the GrappoliV2 dataset, an improvement of 5.39% over YOLOv7. Additionally, our method increases Precision, Recall, and F1 score by 2.83%, 3.49%, and 0.07, respectively. Compared to state-of-the-art detection methods, our approach demonstrates superior detection performance and adaptability to various environments for detecting grape clusters.

The Role of Population Receptive Field Sizes in Higher-Order Visual Dysfunction.

PURPOSE OF REVIEW: Population receptive field (pRF) modeling is an fMRI technique used to retinotopically map visual cortex, with pRF size characterizing the degree of spatial integration. In clinical populations, most pRF mapping research has focused on damage to visual system inputs. Herein, we highlight recent work using pRF modeling to study high-level visual dysfunctions. RECENT FINDINGS: Larger pRF sizes, indicating coarser spatial processing, were observed in homonymous visual field deficits, aging, and autism spectrum disorder. Smaller pRF sizes, indicating finer processing, were observed in Alzheimer's disease and schizophrenia. In posterior cortical atrophy, a unique pattern was found in which pRF size changes depended on eccentricity. Changes to pRF properties were observed in clinical populations, even in high-order impairments, explaining visual behavior. These pRF changes likely stem from altered interactions between brain regions. Furthermore, some studies suggested that pRF sizes change as part of cortical reorganization, and they can point towards future prognosis.

Fourier Convolution Block with global receptive field for MRI reconstruction.

Reconstructing images from under-sampled Magnetic Resonance Imaging (MRI) signals significantly reduces scan time and improves clinical practice. However, Convolutional Neural Network (CNN)-based methods, while demonstrating great performance in MRI reconstruction, may face limitations due to their restricted receptive field (RF), hindering the capture of global features. This is particularly crucial for reconstruction, as aliasing artifacts are distributed globally. Recent advancements in Vision Transformers have further emphasized the significance of a large RF. In this study, we proposed a novel global Fourier Convolution Block (FCB) with whole image RF and low computational complexity by transforming the regular spatial domain convolutions into frequency domain. Visualizations of the effective RF and trained kernels demonstrated that FCB improves the RF of reconstruction models in practice. The proposed FCB was evaluated on four popular CNN architectures using brain and knee MRI datasets. Models with FCB achieved superior PSNR and SSIM than baseline models and exhibited more details and texture recovery. The code is publicly available at https://github.com/Haozhoong/FCB.

qPRF: A system to accelerate population receptive field decoding.

Patterns of BOLD response can be decoded using the population receptive field (PRF) model to reveal how visual input is represented on the cortex (Dumoulin and Wandell, 2008). The time cost of evaluating the PRF model is high, often requiring days to decode BOLD signals for a small cohort of subjects. We introduce the qPRF, an efficient method for decoding that reduced the computation time by a factor of 1436 when compared to another widely available PRF decoder (Kay, Winawer, Mezer and Wandell, 2013) on a benchmark of data from the Human Connectome Project (HCP; Van Essen, Smith, Barch, Behrens, Yacoub and Ugurbil, 2013). With a specially designed data structure and an efficient search algorithm, the qPRF optimizes the five PRF model parameters according to a least-squares criterion. To verify the accuracy of the qPRF solutions, we compared them to those provided by Benson, Jamison, Arcaro, Vu, Glasser, Coalson, Van Essen, Yacoub, Ugurbil, Winawer and Kay (2018). Both hemispheres of the 181 subjects in the HCP data set (a total of 10,753,572 vertices, each with a unique BOLD time series of 1800 frames) were decoded by qPRF in 15.2 hours on an ordinary CPU. The absolute difference in R 2 reported by Benson et al. and achieved by the qPRF was negligible, with a median of 0.39% ( R 2 units being between 0% and 100%). In general, the qPRF yielded a slightly better fitting solution, achieving a greater R 2 on 99.7% of vertices. The qPRF may facilitate the development and computation of more elaborate models based on the PRF framework, as well as the exploration of novel clinical applications.

Sub-bundle based analysis reveals the role of human optic radiation in visual working memory.

White matter (WM) functional activity has been reliably detected through functional magnetic resonance imaging (fMRI). Previous studies have primarily examined WM bundles as unified entities, thereby obscuring the functional heterogeneity inherent within these bundles. Here, for the first time, we investigate the function of sub-bundles of a prototypical visual WM tract-the optic radiation (OR). We use the 7T retinotopy dataset from the Human Connectome Project (HCP) to reconstruct OR and further subdivide the OR into sub-bundles based on the fiber's termination in the primary visual cortex (V1). The population receptive field (pRF) model is then applied to evaluate the retinotopic properties of these sub-bundles, and the consistency of the pRF properties of sub-bundles with those of V1 subfields is evaluated. Furthermore, we utilize the HCP working memory dataset to evaluate the activations of the foveal and peripheral OR sub-bundles, along with LGN and V1 subfields, during 0-back and 2-back tasks. We then evaluate differences in 2bk-0bk contrast between foveal and peripheral sub-bundles (or subfields), and further examine potential relationships between 2bk-0bk contrast and 2-back task d-prime. The results show that the pRF properties of OR sub-bundles exhibit standard retinotopic properties and are typically similar to the properties of V1 subfields. Notably, activations during the 2-back task consistently surpass those under the 0-back task across foveal and peripheral OR sub-bundles, as well as LGN and V1 subfields. The foveal V1 displays significantly higher 2bk-0bk contrast than peripheral V1. The 2-back task d-prime shows strong correlations with 2bk-0bk contrast for foveal and peripheral OR fibers. These findings demonstrate that the blood oxygen level-dependent (BOLD) signals of OR sub-bundles encode high-fidelity visual information, underscoring the feasibility of assessing WM functional activity at the sub-bundle level. Additionally, the study highlights the role of OR in the top-down processes of visual working memory beyond the bottom-up processes for visual information transmission. Conclusively, this study innovatively proposes a novel paradigm for analyzing WM fiber tracts at the individual sub-bundle level and expands understanding of OR function.

HDConv: Heterogeneous kernel-based dilated convolutions.

Dilated convolution has been widely used in various computer vision tasks due to its ability to expand the receptive field while maintaining the resolution of feature maps. However, the critical challenge is the gridding problem caused by the isomorphic structure of the dilated convolution, where the holes filled in the dilated convolution destroy the integrity of the extracted information and cut off the relevance of neighboring pixels. In this work, a novel heterogeneous dilated convolution, called HDConv, is proposed to address this issue by setting independent dilation rates on grouped channels while keeping the general convolution operation. The heterogeneous structure can effectively avoid the gridding problem while introducing multi-scale kernels in the filters. Based on the heterogeneous structure of the proposed HDConv, we also explore the benefit of large receptive fields to feature extraction by comparing different combinations of dilated rates. Finally, a series of experiments are conducted to verify the effectiveness of some computer vision tasks, such as image segmentation and object detection. The results show the proposed HDConv can achieve a competitive performance on ADE20K, Cityscapes, COCO-Stuff10k, COCO, and a medical image dataset UESTC-COVID-19. The proposed module can readily replace conventional convolutions in existing convolutional neural networks (i.e., plug-and-play), and it is promising to further extend dilated convolution to wider scenarios in the field of image segmentation.

Heterogeneous orientation tuning in the primary visual cortex of mice diverges from Gabor-like receptive fields in primates.

A key feature of neurons in the primary visual cortex (V1) of primates is their orientation selectivity. Recent studies using deep neural network models showed that the most exciting input (MEI) for mouse V1 neurons exhibit complex spatial structures that predict non-uniform orientation selectivity across the receptive field (RF), in contrast to the classical Gabor filter model. Using local patches of drifting gratings, we identified heterogeneous orientation tuning in mouse V1 that varied up to 90° across sub-regions of the RF. This heterogeneity correlated with deviations from optimal Gabor filters and was consistent across cortical layers and recording modalities (calcium vs. spikes). In contrast, model-synthesized MEIs for macaque V1 neurons were predominantly Gabor like, consistent with previous studies. These findings suggest that complex spatial feature selectivity emerges earlier in the visual pathway in mice than in primates. This may provide a faster, though less general, method of extracting task-relevant information.

Widespread receptive field remapping in early primate visual cortex.

Predictive remapping of receptive fields (RFs) is thought to be one of the critical mechanisms for enforcing perceptual stability during eye movements. While RF remapping has been observed in several cortical areas, its role in early visual cortex and its consequences on the tuning properties of neurons have been poorly understood. Here, we track remapping RFs in hundreds of neurons from visual area V2 while subjects perform a cued saccade task. We find that remapping is widespread in area V2 across neurons from all recorded cortical layers and cell types. Furthermore, our results suggest that remapping RFs not only maintain but also transiently enhance their feature selectivity due to untuned suppression. Taken together, these findings shed light on the dynamics and prevalence of remapping in the early visual cortex, forcing us to revise current models of perceptual stability during saccadic eye movements.

Receptive Field Space for Point Cloud Analysis.

Similar to convolutional neural networks for image processing, existing analysis methods for 3D point clouds often require the designation of a local neighborhood to describe the local features of the point cloud. This local neighborhood is typically manually specified, which makes it impossible for the network to dynamically adjust the receptive field's range. If the range is too large, it tends to overlook local details, and if it is too small, it cannot establish global dependencies. To address this issue, we introduce in this paper a new concept: receptive field space (RFS). With a minor computational cost, we extract features from multiple consecutive receptive field ranges to form this new receptive field space. On this basis, we further propose a receptive field space attention mechanism, enabling the network to adaptively select the most effective receptive field range from RFS, thus equipping the network with the ability to adjust granularity adaptively. Our approach achieved state-of-the-art performance in both point cloud classification, with an overall accuracy (OA) of 94.2%, and part segmentation, achieving an mIoU of 86.0%, demonstrating the effectiveness of our method.

Improving the Reliability and Accuracy of Population Receptive Field Measures Using a Logarithmically Warped Stimulus.

The population receptive field method, which measures the region in visual space that elicits a BOLD signal in a voxel in retinotopic cortex, is a powerful tool for investigating the functional organization of human visual cortex with fMRI (Dumoulin & Wandell, 2008). However, recent work has shown that population receptive field (pRF) estimates for early retinotopic visual areas can be biased and unreliable, especially for voxels representing the fovea. Here, we show that a 'log-bar' stimulus that is logarithmically warped along the eccentricity dimension produces more reliable estimates of pRF size and location than the traditional moving bar stimulus. The log-bar stimulus was better able to identify pRFs near the foveal representation, and pRFs were smaller in size, consistent with simulation estimates of receptive field sizes in the fovea.

Stable 3D Deep Convolutional Autoencoder Method for Ultrasonic Testing of Defects in Polymer Composites.

Ultrasonic testing is widely used for defect detection in polymer composites owing to advantages such as fast processing speed, simple operation, high reliability, and real-time monitoring. However, defect information in ultrasound images is not easily detectable because of the influence of ultrasound echoes and noise. In this study, a stable three-dimensional deep convolutional autoencoder (3D-DCA) was developed to identify defects in polymer composites. Through 3D convolutional operations, it can synchronously learn the spatiotemporal properties of the data volume. Subsequently, the depth receptive field (RF) of the hidden layer in the autoencoder maps the defect information to the original depth location, thereby mitigating the effects of the defect surface and bottom echoes. In addition, a dual-layer encoder was designed to improve the hidden layer visualization results. Consequently, the size, shape, and depth of the defects can be accurately determined. The feasibility of the method was demonstrated through its application to defect detection in carbon-fiber-reinforced polymers.

Variations in touch representation in the hummingbird and zebra finch forebrain.

Somatosensation is essential for animals to perceive the external world through touch, allowing them to detect physical contact, temperature, pain, and body position. Studies on rodent vibrissae have highlighted the organization and processing in mammalian somatosensory pathways.1,2 Comparative research across vertebrates is vital for understanding evolutionary influences and ecological specialization on somatosensory systems. Birds, with their diverse morphologies, sensory abilities, and behaviors, serve as ideal models for investigating the evolution of somatosensation. Prior studies have uncovered tactile-responsive areas within the avian telencephalon, particularly in pigeons,3,4,5,6 parrots,7 and finches,8 but variations in somatosensory maps and responses across avian species are not fully understood. This study aims to explore somatotopic organization and neural coding in the telencephalon of Anna's hummingbirds (Calypte anna) and zebra finches (Taeniopygia guttata) by using in vivo extracellular electrophysiology to record activity in response to controlled tactile stimuli on various body regions. These findings reveal unique representations of body regions across distinct forebrain somatosensory nuclei, indicating significant differences in the extent of areas dedicated to certain body surfaces, which may correlate with their behavioral importance.

An efficient deep neural network for automatic classification of acute intracranial hemorrhages in brain CT scans.

BACKGROUND: Recent advancements in deep learning models have demonstrated their potential in the field of medical imaging, achieving remarkable performance surpassing human capabilities in tasks such as classification and segmentation. However, these modern state-of-the-art network architectures often demand substantial computational resources, which limits their practical application in resource-constrained settings. This study aims to propose an efficient diagnostic deep learning model specifically designed for the classification of intracranial hemorrhage in brain CT scans. METHOD: Our proposed model utilizes a combination of depthwise separable convolutions and a multi-receptive field mechanism to achieve a trade-off between performance and computational efficiency. The model was trained on RSNA datasets and validated on CQ500 dataset and PhysioNet dataset. RESULT: Through a comprehensive comparison with state-of-the-art models, our model achieves an average AUROC score of 0.952 on RSNA datasets and exhibits robust generalization capabilities, comparable to SE-ResNeXt, across other open datasets. Furthermore, the parameter count of our model is just 3 % of that of MobileNet V3. CONCLUSION: This study presents a diagnostic deep-learning model that is optimized for classifying intracranial hemorrhages in brain CT scans. The efficient characteristics make our proposed model highly promising for broader applications in medical settings.

ConvMedSegNet: A multi-receptive field depthwise convolutional neural network for medical image segmentation.

In order to achieve highly precise medical image segmentation, this paper presents ConvMedSegNet, a novel convolutional neural network designed with a U-shaped architecture that seamlessly integrates two crucial modules: the multi-receptive field depthwise convolution module (MRDC) and the guided fusion module (GF). The MRDC module's primary function is to capture texture information of varying sizes through multi-scale convolutional layers. This information is subsequently utilized to enhance the correlation of global feature data by expanding the network's width. This strategy adeptly preserves the inherent inductive biases of convolution while concurrently amplifying the network's ability to establish dependencies on global information. Conversely, the GF module assumes responsibility for implementing multi-scale feature fusion by connecting the encoder and decoder components. It facilitates the transfer of information between features that are separated over substantial distances through guided fusion, effectively minimizing the loss of critical data. In experiments conducted on public medical image datasets such as BUSI and ISIC2018, ConvMedSegNet outperforms several advanced competing methods, yielding superior results. Additionally, the code can be accessed at https://github.com/csust-yixin/ConvMedSegNet.

Convolutional neural networks develop major organizational principles of early visual cortex when enhanced with retinal sampling.

Primate visual cortex exhibits key organizational principles: cortical magnification, eccentricity-dependent receptive field size and spatial frequency tuning as well as radial bias. We provide compelling evidence that these principles arise from the interplay of the non-uniform distribution of retinal ganglion cells, and a quasi-uniform convergence rate from the retina to the cortex. We show that convolutional neural networks outfitted with a retinal sampling layer, which resamples images according to retinal ganglion cell density, develop these organizational principles. Surprisingly, our results indicate that radial bias is spatial-frequency dependent and only manifests for high spatial frequencies. For low spatial frequencies, the bias shifts towards orthogonal orientations. These findings introduce a novel hypothesis about the origin of radial bias. Quasi-uniform convergence limits the range of spatial frequencies (in retinal space) that can be resolved, while retinal sampling determines the spatial frequency content throughout the retina.

Opposite size illusions for inverted faces and letters.

Words are the primary means by which we communicate meaning and ideas, while faces provide important social cues. Studying visual illusions involving faces and words can elucidate the hierarchical processing of information as different regions of the brain are specialised for face recognition and word processing. A size illusion has previously been demonstrated for faces, whereby an inverted face is perceived as larger than the same stimulus upright. Here, two experiments replicate the face size illusion, and investigate whether the illusion is also present for individual letters (Experiment 1), and visual words and pseudowords (Experiment 2). Results confirm a robust size Illusion for faces. Letters, words and pseudowords and unfamiliar letters all show a reverse size illusion, as we previously demonstrated for human bodies. Overall, results indicate the illusion occurs in early perceptual stages upstream of semantic processing. Results are consistent with the idea of a general-purpose mechanism that encodes curvilinear shapes found in both scripts and our environment. Word and face perception rely on specialised, independent cognitive processes. The underestimation of the size of upright stimuli is specific to faces. Opposite size illusions may reflect differences in how size information is encoded and represented in stimulus-specialised neural networks, resulting in contrasting perceptual effects. Though words and faces differ visually, there is both symmetry and asymmetry in how the brain 'reads' them.

Local field potentials, spiking activity, and receptive fields in human visual cortex.

The concept of receptive field (RF) is central to sensory neuroscience. Neuronal RF properties have been substantially studied in animals, while those in humans remain nearly unexplored. Here, we measured neuronal RFs with intracranial local field potentials (LFPs) and spiking activity in human visual cortex (V1/V2/V3). We recorded LFPs via macro-contacts and discovered that RF sizes estimated from low-frequency activity (LFA, 0.5-30 Hz) were larger than those estimated from low-gamma activity (LGA, 30-60 Hz) and high-gamma activity (HGA, 60-150 Hz). We then took a rare opportunity to record LFPs and spiking activity via microwires in V1 simultaneously. We found that RF sizes and temporal profiles measured from LGA and HGA closely matched those from spiking activity. In sum, this study reveals that spiking activity of neurons in human visual cortex could be well approximated by LGA and HGA in RF estimation and temporal profile measurement, implying the pivotal functions of LGA and HGA in early visual information processing.

Visual working memories are abstractions of percepts.

Pioneering studies demonstrating that the contents of visual working memory (WM) can be decoded from the patterns of multivoxel activity in early visual cortex transformed not only how we study WM, but theories of how memories are stored. For instance, the ability to decode the orientation of memorized gratings is hypothesized to depend on the recruitment of the same neural encoding machinery used for perceiving orientations. However, decoding evidence cannot be used to test the so-called sensory recruitment hypothesis without understanding the underlying nature of what is being decoded. Although unknown during WM, during perception decoding the orientation of gratings does not simply depend on activities of orientation tuned neurons. Rather, it depends on complex interactions between the orientation of the grating, the aperture edges, and the topographic structure of the visual map. Here, our goals are to 1) test how these aperture biases described during perception may affect WM decoding, and 2) leverage carefully manipulated visual stimulus properties of gratings to test how sensory-like are WM codes. For memoranda, we used gratings multiplied by radial and angular modulators to generate orthogonal aperture biases despite having identical orientations. Therefore, if WM representations are simply maintained sensory representations, they would have similar aperture biases. If they are abstractions of sensory features, they would be unbiased and the modulator would have no effect on orientation decoding. Results indicated that fMRI patterns of delay period activity while maintaining the orientation of a grating with one modulator (eg, radial) were interchangeable with patterns while maintaining a grating with the other modulator (eg, angular). We found significant cross-classification in visual and parietal cortex, suggesting that WM representations are insensitive to aperture biases during perception. Then, we visualized memory abstractions of stimuli using a population receptive field model of the visual field maps. Regardless of aperture biases, WM representations of both modulated gratings were recoded into a single oriented line. These results provide strong evidence that visual WM representations are abstractions of percepts, immune to perceptual aperture biases, and compel revisions of WM theory.

Hierarchies in Visual Pathway: Functions and Inspired Artificial Vision.

The development of artificial intelligence has posed a challenge to machine vision based on conventional complementary metal-oxide semiconductor (CMOS) circuits owing to its high latency and inefficient power consumption originating from the data shuffling between memory and computation units. Gaining more insights into the function of every part of the visual pathway for visual perception can bring the capabilities of machine vision in terms of robustness and generality. Hardware acceleration of more energy-efficient and biorealistic artificial vision highly necessitates neuromorphic devices and circuits that are able to mimic the function of each part of the visual pathway. In this paper, we review the structure and function of the entire class of visual neurons from the retina to the primate visual cortex within reach (Chapter 2) are reviewed. Based on the extraction of biological principles, the recent hardware-implemented visual neurons located in different parts of the visual pathway are discussed in detail in Chapters 3 and 4. Furthermore, valuable applications of inspired artificial vision in different scenarios (Chapter 5) are provided. The functional description of the visual pathway and its inspired neuromorphic devices/circuits are expected to provide valuable insights for the design of next-generation artificial visual perception systems.