RESUMEN
Understanding how biological visual systems process information is challenging because of the nonlinear relationship between visual input and neuronal responses. Artificial neural networks allow computational neuroscientists to create predictive models that connect biological and machine vision. Machine learning has benefited tremendously from benchmarks that compare different model on the same task under standardized conditions. However, there was no standardized benchmark to identify state-of-the-art dynamic models of the mouse visual system. To address this gap, we established the SENSORIUM 2023 Benchmark Competition with dynamic input, featuring a new large-scale dataset from the primary visual cortex of ten mice. This dataset includes responses from 78,853 neurons to 2 hours of dynamic stimuli per neuron, together with the behavioral measurements such as running speed, pupil dilation, and eye movements. The competition ranked models in two tracks based on predictive performance for neuronal responses on a held-out test set: one focusing on predicting in-domain natural stimuli and another on out-of-distribution (OOD) stimuli to assess model generalization. As part of the NeurIPS 2023 competition track, we received more than 160 model submissions from 22 teams. Several new architectures for predictive models were proposed, and the winning teams improved the previous state-of-the-art model by 50%. Access to the dataset as well as the benchmarking infrastructure will remain online at www.sensorium-competition.net.
RESUMEN
This work presents a novel and effective method for fitting multidimensional ellipsoids (i.e., ellipsoids embedded in [Formula: see text]) to scattered data in the contamination of noise and outliers. Unlike conventional algebraic or geometric fitting paradigms that assume each measurement point is a noisy version of its nearest point on the ellipsoid, we approach the problem as a Bayesian parameter estimate process and maximize the posterior probability of a certain ellipsoidal solution given the data. We establish a more robust correlation between these points based on the predictive distribution within the Bayesian framework, i.e., considering each model point as a potential source for generating each measurement. Concretely, we incorporate a uniform prior distribution to constrain the search for primitive parameters within an ellipsoidal domain, ensuring ellipsoid-specific results regardless of inputs. We then establish the connection between measurement point and model data via Bayes' rule to enhance the method's robustness against noise. Due to independent of spatial dimensions, the proposed method not only delivers high-quality fittings to challenging elongated ellipsoids but also generalizes well to multidimensional spaces. To address outlier disturbances, often overlooked by previous approaches, we further introduce a uniform distribution on top of the predictive distribution to significantly enhance the algorithm's robustness against outliers. Thanks to the uniform prior, our maximum a posterior probability coincides with a more tractable maximum likelihood estimation problem, which is subsequently solved by a numerically stable Expectation Maximization (EM) framework. Moreover, we introduce an ε-accelerated technique to expedite the convergence of EM considerably. We also investigate the relationship between our algorithm and conventional least-squares-based ones, during which we theoretically prove our method's superior robustness. To the best of our knowledge, this is the first comprehensive method capable of performing multidimensional ellipsoid-specific fitting within the Bayesian optimization paradigm under diverse disturbances. We evaluate it across lower and higher dimensional spaces in the presence of heavy noise, outliers, and substantial variations in axis ratios. Also, we apply it to a wide range of practical applications such as microscopy cell counting, 3D reconstruction, geometric shape approximation, and magnetometer calibration tasks. In all these test contexts, our method consistently delivers flexible, robust, ellipsoid-specific performance, and achieves the state-of-the-art results.
RESUMEN
Creating an image focal stack requires multiple shots, which captures images at different depths within the same scene. Such methods are not suitable for scenes undergoing continuous changes. Achieving an all-in-focus image from a single shot poses significant challenges, due to the highly ill-posed nature of rectifying defocus and deblurring from a single image. In this paper, to restore an all-in-focus image, we introduce the neuromorphic focal stack, which is defined as neuromorphic signal streams captured by an event/ a spike camera during a continuous focal sweep, aiming to restore an all-in-focus image. Given an RGB image focused at any distance, we harness the high temporal resolution of neuromorphic signal streams. From neuromorphic signal streams, we automatically select refocusing timestamps and reconstruct corresponding refocused images to form a focal stack. Guided by the neuromorphic signal around the selected timestamps, we can merge the focal stack using proper weights and restore a sharp all-in-focus image. We test our method on two distinct neuromorphic cameras. Experimental results from both synthetic and real datasets demonstrate a marked improvement over existing state-of-the-art methods.
RESUMEN
Hippocampal place cells in freely moving rodents display both theta phase precession and procession, which is thought to play important roles in cognition, but the neural mechanism for producing theta phase shift remains largely unknown. Here, we show that firing rate adaptation within a continuous attractor neural network causes the neural activity bump to oscillate around the external input, resembling theta sweeps of decoded position during locomotion. These forward and backward sweeps naturally account for theta phase precession and procession of individual neurons, respectively. By tuning the adaptation strength, our model explains the difference between 'bimodal cells' showing interleaved phase precession and procession, and 'unimodal cells' in which phase precession predominates. Our model also explains the constant cycling of theta sweeps along different arms in a T-maze environment, the speed modulation of place cells' firing frequency, and the continued phase shift after transient silencing of the hippocampus. We hope that this study will aid an understanding of the neural mechanism supporting theta phase coding in the brain.
Asunto(s)
Potenciales de Acción , Células de Lugar , Ritmo Teta , Animales , Ritmo Teta/fisiología , Células de Lugar/fisiología , Potenciales de Acción/fisiología , Modelos Neurológicos , Hipocampo/fisiología , Hipocampo/citología , Adaptación Fisiológica , RatasRESUMEN
Purpose: A retinal mosaic, the spatial organization of a population of homotypic neurons, is thought to sample a specific visual feature into the feedforward visual pathway. The purpose of this study was to propose a universal modeling approach for precisely generating retinal mosaics and overcoming the limitations of previous models, especially in modeling abnormal mosaic patterns under disease conditions. Methods: Here, we developed the optimization-based pairwise interaction point process (O-PIPP). It incorporates optimization techniques into previous simulation approaches, enabling directional control of the simulation process according to the user-designed optimization target. For the convenience of the community, we implemented the O-PIPP approach into a Python package and a website application. Results: We showed that the O-PIPP can generate more precise neural spatial patterns of healthy and diseased mosaics compared to previous phenomenological approaches. Notably, through modeling the retinal neural circuitry with O-PIPP-simulated retinitis pigmentosa cone mosaics, we elucidated how the cone mosaic rearrangement impacted the information processing of ganglion cells. Conclusions: The O-PIPP provides a precise and universal tool to simulate realistic mosaics, which could help to investigate the function of retinal mosaics in vision.
Asunto(s)
Retinitis Pigmentosa , Humanos , Retinitis Pigmentosa/fisiopatología , Células Fotorreceptoras Retinianas Conos/fisiología , Células Ganglionares de la Retina/fisiología , Células Ganglionares de la Retina/patología , Vías Visuales/fisiología , Simulación por Computador , RetinaRESUMEN
For capturing dynamic scenes with ultra-fast motion, neuromorphic cameras with extremely high temporal resolution have demonstrated their great capability and potential. Different from the event cameras that only record relative changes in light intensity, spike camera fires a stream of spikes according to a full-time accumulation of photons so that it can recover the texture details for both static areas and dynamic areas. Recently, color spike camera has been invented to record color information of dynamic scenes using a color filter array (CFA). However, demosaicing for color spike cameras is an open and challenging problem. In this paper, we develop a demosaicing network, called CSpkNet, to reconstruct dynamic color visual signals from the spike stream captured by the color spike camera. Firstly, we develop a light inference module to convert binary spike streams to intensity estimates. In particular, a feature-based channel attention module is proposed to reduce the noises caused by quantization errors. Secondly, considering both the Bayer configuration and object motion, we propose a motion-guided filtering module to estimate the missing pixels of each color channel, without undesired motion blur. Finally, we design a refinement module to improve the intensity and details, utilizing the color correlation. Experimental results demonstrate that CSpkNet can reconstruct color images from the Bayer-pattern spike stream with promising visual quality.
RESUMEN
With the help of special neuromorphic hardware, spiking neural networks (SNNs) are expected to realize artificial intelligence (AI) with less energy consumption. It provides a promising energy-efficient way for realistic control tasks by combining SNNs with deep reinforcement learning (DRL). In this article, we focus on the task where the agent needs to learn multidimensional deterministic policies to control, which is very common in real scenarios. Recently, the surrogate gradient method has been utilized for training multilayer SNNs, which allows SNNs to achieve comparable performance with the corresponding deep networks in this task. Most existing spike-based reinforcement learning (RL) methods take the firing rate as the output of SNNs, and convert it to represent continuous action space (i.e., the deterministic policy) through a fully connected (FC) layer. However, the decimal characteristic of the firing rate brings the floating-point matrix operations to the FC layer, making the whole SNN unable to deploy on the neuromorphic hardware directly. To develop a fully spiking actor network (SAN) without any floating-point matrix operations, we draw inspiration from the nonspiking interneurons found in insects and employ the membrane voltage of the nonspiking neurons to represent the action. Before the nonspiking neurons, multiple population neurons are introduced to decode different dimensions of actions. Since each population is used to decode a dimension of action, we argue that the neurons in each population should be connected in time domain and space domain. Hence, the intralayer connections are used in output populations to enhance the representation capacity. This mechanism exists extensively in animals and has been demonstrated effectively. Finally, we propose a fully SAN with intralayer connections (ILC-SAN). Extensive experimental results demonstrate that the proposed method outperforms the state-of-the-art performance on continuous control tasks from OpenAI gym. Moreover, we estimate the theoretical energy consumption when deploying ILC-SAN on neuromorphic chips to illustrate its high energy efficiency.
RESUMEN
Sensory information transmitted to the brain activates neurons to create a series of coping behaviors. Understanding the mechanisms of neural computation and reverse engineering the brain to build intelligent machines requires establishing a robust relationship between stimuli and neural responses. Neural decoding aims to reconstruct the original stimuli that trigger neural responses. With the recent upsurge of artificial intelligence, neural decoding provides an insightful perspective for designing novel algorithms of brain-machine interface. For humans, vision is the dominant contributor to the interaction between the external environment and the brain. In this study, utilizing the retinal neural spike data collected over multi trials with visual stimuli of two movies with different levels of scene complexity, we used a neural network decoder to quantify the decoded visual stimuli with six different metrics for image quality assessment establishing comprehensive inspection of decoding. With the detailed and systematical study of the effect and single and multiple trials of data, different noise in spikes, and blurred images, our results provide an in-depth investigation of decoding dynamical visual scenes using retinal spikes. These results provide insights into the neural coding of visual scenes and services as a guideline for designing next-generation decoding algorithms of neuroprosthesis and other devices of brain-machine interface.
RESUMEN
BACKGROUND: Accumulative evidence suggests that pyroptosis plays a key role in mediating angiotensin II (Ang II)-induced cardiac remodeling However, the potential role of pyroptosis-related transcription factor (TF)-microRNA (miRNA)-gene regulatory networks in mediating Ang II-associated cardiac remodeling remains largely unknown. Therefore, we identified the pyroptosis-related hub genes and constructed a transcription factor (TF)-miRNA-target gene regulatory network using bioinformatic tools to elucidate the pathogenesis of Ang II-induced cardiac remodeling. METHODS: The pyroptosis-related differentially expressed genes (DEGs) were identified from the cardiac remodeling-related dataset GSE47420. Gene ontology (GO), Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway enrichment analysis, and protein-protein interaction (PPI) analysis were performed to identify the pyroptosis-related hub DEGs. A TF-miRNA-target gene network was constructed and further validated by quantitative real-time polymerase chain reaction (qRT-PCR) in animal experiments. The correlation between the pyroptosis-related hub DEGs and cardiac remodeling was evaluated using comparative toxicogenomics database. The drug-gene interaction analysis was performed to identify potential drugs that target the pyroptosis-related hub DEGs. RESULTS: A total of 32 pyroptosis-related DEGs were identified and enriched in the inflammation-related pathways by KEGG analysis. 13 of the 32 pyroptosis-related DEGs were identified as hub DEGs. Furthermore, a TF-miRNA-target gene regulatory network containing 16 TFs, 6 miRNAs, and 5 hub target genes was constructed. The five pyroptosis-related hub target genes (DDX3X, ELAVL1, YWHAZ, STAT3, and EED) were identified as crucial cardiac remodeling-related genes using the comparative toxicogenomics database (CTD) database. Five drugs including celecoxib were identified as potential drugs for the treatment of cardiac remodeling. Finally, the expression levels of two top-ranked TF-miRNA-target genes axis were verified by qRT-PCR in mice with Ang II-induced cardiac remodeling and found to be generally consistent with the microarray results. CONCLUSIONS: This study constructed a pyroptosis-related TF-miRNA-target gene regulatory network for Ang II-induced cardiac remodeling. Five pyroptosis-related genes (DDX3X, ELAVL1, YWHAZ, STAT3, and EED) can be considered the core genes associated with pyrotposis-related cardiac remodeling. The findings of this study provide new insights into the molecular mechanisms of Ang II-induced cardiac remodeling and may serve as potential biomarkers or therapeutic targets for Ang II-induced cardiac remodeling.
Asunto(s)
Redes Reguladoras de Genes , MicroARNs , Animales , Ratones , MicroARNs/genética , MicroARNs/metabolismo , Factores de Transcripción/metabolismo , Angiotensina II/farmacología , Angiotensina II/metabolismo , Piroptosis/genética , Remodelación Ventricular/genética , Mapas de Interacción de Proteínas/genética , Perfilación de la Expresión Génica/métodos , Regulación Neoplásica de la Expresión Génica , Biología Computacional/métodosRESUMEN
BACKGROUND: Esophageal squamous cell carcinoma (ESCC) is a highly lethal tumor type, but studies on the ESCC tumor microenvironment are limited. We found that cystatin SN (CST1) plays an important role in the ESCC tumor microenvironment. CST1 has been reported to act as an oncogene in multiple human cancers, but its clinical significance and underlying mechanism in ESCC remain elusive. METHODS: We performed ESCC gene expression profiling with data from RNA-sequencing and public databases and found CST1 upregulation in ESCC. Then, we assessed CST1 expression in ESCC by RTâqPCR and Western blot analysis. In addition, immunohistochemistry (IHC) and enzyme-linked immunosorbent assay (ELISA) were used to estimate the expression of CST1 in ESCC tissue and serum. Moreover, further functional experiments were conducted to verify that the gain and loss of CST1 in ESCC cell lines significantly influenced the proliferation and metastasis of ESCC. Mass spectrometry, coimmunoprecipitation, and gelatin zymography experiments were used to validate the interaction between CST1 and matrix metalloproteinase 2 (MMP2) and the mechanism of CST1 influence on metastasis in ESCC. RESULTS: Here, we found that CST1 expression was significantly elevated in ESCC tissues and serum. Moreover, compared with patients with low CST1 expression, patients with high CST1 expression had a worse prognosis. Overall survival (OS) and disease-free survival (DFS) were significantly unfavorable in the high CST1 expression subgroup. Likewise, the CST1 level was significantly increased in ESCC serum compared with healthy control serum, indicating that CST1 may be a potential serum biomarker for diagnosis, with an area under the curve (AUC) = 0.9702 and p < 0.0001 by receiver operating curve (ROC) analysis. Furthermore, upregulated CST1 can promote the motility and metastatic capacity of ESCC in vitro and in vivo by influencing epithelial mesenchymal transition (EMT) and interacting with MMP2 in the tumor microenvironment (TME). CONCLUSIONS: Collectively, the results of this study indicated that high CST1 expression mediated by SPI1 in ESCC may serve as a potentially prognostic and diagnostic predictor and as an oncogene to promote motility and metastatic capacity of ESCC by influencing EMT and interacting with MMP2 in the TME.
Asunto(s)
Carcinoma de Células Escamosas , Neoplasias Esofágicas , Carcinoma de Células Escamosas de Esófago , Humanos , Carcinoma de Células Escamosas de Esófago/genética , Metaloproteinasa 2 de la Matriz/genética , Metaloproteinasa 2 de la Matriz/metabolismo , Carcinoma de Células Escamosas/metabolismo , Neoplasias Esofágicas/genética , Neoplasias Esofágicas/patología , Regulación hacia Arriba , Pronóstico , Biomarcadores de Tumor/genética , Biomarcadores de Tumor/metabolismo , Línea Celular Tumoral , Regulación Neoplásica de la Expresión Génica , Proliferación Celular/genética , Transición Epitelial-Mesenquimal , Microambiente Tumoral/genéticaRESUMEN
Biophysically detailed multi-compartment models are powerful tools to explore computational principles of the brain and also serve as a theoretical framework to generate algorithms for artificial intelligence (AI) systems. However, the expensive computational cost severely limits the applications in both the neuroscience and AI fields. The major bottleneck during simulating detailed compartment models is the ability of a simulator to solve large systems of linear equations. Here, we present a novel Dendritic Hierarchical Scheduling (DHS) method to markedly accelerate such a process. We theoretically prove that the DHS implementation is computationally optimal and accurate. This GPU-based method performs with 2-3 orders of magnitude higher speed than that of the classic serial Hines method in the conventional CPU platform. We build a DeepDendrite framework, which integrates the DHS method and the GPU computing engine of the NEURON simulator and demonstrate applications of DeepDendrite in neuroscience tasks. We investigate how spatial patterns of spine inputs affect neuronal excitability in a detailed human pyramidal neuron model with 25,000 spines. Furthermore, we provide a brief discussion on the potential of DeepDendrite for AI, specifically highlighting its ability to enable the efficient training of biophysically detailed models in typical image classification tasks.
Asunto(s)
Inteligencia Artificial , Neuronas , Humanos , Algoritmos , Células Piramidales , EncéfaloRESUMEN
Neuromorphic cameras are emerging imaging technology that has advantages over conventional imaging sensors in several aspects including dynamic range, sensing latency, and power consumption. However, the signal-to-noise level and the spatial resolution still fall behind the state of conventional imaging sensors. In this article, we address the denoising and super-resolution problem for modern neuromorphic cameras. We employ 3D U-Net as the backbone neural architecture for such a task. The networks are trained and tested on two types of neuromorphic cameras: a dynamic vision sensor and a spike camera. Their pixels generate signals asynchronously, the former is based on perceived light changes and the latter is based on accumulated light intensity. To collect the datasets for training such networks, we design a display-camera system to record high frame-rate videos at multiple resolutions, providing supervision for denoising and super-resolution. The networks are trained in a noise-to-noise fashion, where the two ends of the network are unfiltered noisy data. The output of the networks has been tested for downstream applications including event-based visual object tracking and image reconstruction. Experimental results demonstrate the effectiveness of improving the quality of neuromorphic events and spikes, and the corresponding improvement to downstream applications with state-of-the-art performance.
RESUMEN
Object recognition is often viewed as a feedforward, bottom-up process in machine learning, but in real neural systems, object recognition is a complicated process which involves the interplay between two signal pathways. One is the parvocellular pathway (P-pathway), which is slow and extracts fine features of objects; the other is the magnocellular pathway (M-pathway), which is fast and extracts coarse features of objects. It has been suggested that the interplay between the two pathways endows the neural system with the capacity of processing visual information rapidly, adaptively, and robustly. However, the underlying computational mechanism remains largely unknown. In this study, we build a two-pathway model to elucidate the computational properties associated with the interactions between two visual pathways. Specifically, we model two visual pathways using two convolution neural networks: one mimics the P-pathway, referred to as FineNet, which is deep, has small-size kernels, and receives detailed visual inputs; the other mimics the M-pathway, referred to as CoarseNet, which is shallow, has large-size kernels, and receives blurred visual inputs. We show that CoarseNet can learn from FineNet through imitation to improve its performance, FineNet can benefit from the feedback of CoarseNet to improve its robustness to noise; and the two pathways interact with each other to achieve rough-to-fine information processing. Using visual backward masking as an example, we further demonstrate that our model can explain visual cognitive behaviors that involve the interplay between two pathways. We hope that this study gives us insight into understanding the interaction principles between two visual pathways.
Asunto(s)
Cognición , Percepción Visual , Aprendizaje Automático , Redes Neurales de la Computación , Vías VisualesRESUMEN
MOTIVATION: Cell membrane segmentation in electron microscopy (EM) images is a crucial step in EM image processing. However, while popular approaches have achieved performance comparable to that of humans on low-resolution EM datasets, they have shown limited success when applied to high-resolution EM datasets. The human visual system, on the other hand, displays consistently excellent performance on both low and high resolutions. To better understand this limitation, we conducted eye movement and perceptual consistency experiments. Our data showed that human observers are more sensitive to the structure of the membrane while tolerating misalignment, contrary to commonly used evaluation criteria. Additionally, our results indicated that the human visual system processes images in both global-local and coarse-to-fine manners. RESULTS: Based on these observations, we propose a computational framework for membrane segmentation that incorporates these characteristics of human perception. This framework includes a novel evaluation metric, the perceptual Hausdorff distance (PHD), and an end-to-end network called the PHD-guided segmentation network (PS-Net) that is trained using adaptively tuned PHD loss functions and a multiscale architecture. Our subjective experiments showed that the PHD metric is more consistent with human perception than other criteria, and our proposed PS-Net outperformed state-of-the-art methods on both low- and high-resolution EM image datasets as well as other natural image datasets. AVAILABILITY AND IMPLEMENTATION: The code and dataset can be found at https://github.com/EmmaSRH/PS-Net.
Asunto(s)
Procesamiento de Imagen Asistido por Computador , Percepción , Humanos , Membrana Celular , Microscopía ElectrónicaRESUMEN
High-speed imaging can help us understand some phenomena that are too fast to be captured by our eyes. Although ultra-fast frame-based cameras (e.g., Phantom) can record millions of fps at reduced resolution, they are too expensive to be widely used. Recently, a retina-inspired vision sensor, spiking camera, has been developed to record external information at 40, 000 Hz. The spiking camera uses the asynchronous binary spike streams to represent visual information. Despite this, how to reconstruct dynamic scenes from asynchronous spikes remains challenging. In this paper, we introduce novel high-speed image reconstruction models based on the short-term plasticity (STP) mechanism of the brain, termed TFSTP and TFMDSTP. We first derive the relationship between states of STP and spike patterns. Then, in TFSTP, by setting up the STP model at each pixel, the scene radiance can be inferred by the states of the models. In TFMDSTP, we use the STP to distinguish the moving and stationary regions, and then use two sets of STP models to reconstruct them respectively. In addition, we present a strategy for correcting error spikes. Experimental results show that the STP-based reconstruction methods can effectively reduce noise with less computing time, and achieve the best performances on both real-world and simulated datasets.
RESUMEN
Reconstruction of high dynamic range image from a single low dynamic range image captured by a conventional RGB camera, which suffers from over- or under-exposure, is an ill-posed problem. In contrast, recent neuromorphic cameras like event camera and spike camera can record high dynamic range scenes in the form of intensity maps, but with much lower spatial resolution and no color information. In this article, we propose a hybrid imaging system (denoted as NeurImg) that captures and fuses the visual information from a neuromorphic camera and ordinary images from an RGB camera to reconstruct high-quality high dynamic range images and videos. The proposed NeurImg-HDR+ network consists of specially designed modules, which bridges the domain gaps on resolution, dynamic range, and color representation between two types of sensors and images to reconstruct high-resolution, high dynamic range images and videos. We capture a test dataset of hybrid signals on various HDR scenes using the hybrid camera, and analyze the advantages of the proposed fusing strategy by comparing it to state-of-the-art inverse tone mapping methods and merging two low dynamic range images approaches. Quantitative and qualitative experiments on both synthetic data and real-world scenarios demonstrate the effectiveness of the proposed hybrid high dynamic range imaging system. Code and dataset can be found at: https://github.com/hjynwa/NeurImg-HDR.
RESUMEN
Biophysically detailed neuron simulation is a powerful tool to explore the mechanisms behind biological experiments and bridge the gap between various scales in neuroscience research. However, the extremely high computational complexity of detailed neuron simulation restricts the modeling and exploration of detailed network models. The bottleneck is solving the system of linear equations. To accelerate detailed simulation, we propose a heuristic tree-partition-based parallel method (HTP) to parallelize the computation of the Hines algorithm, the kernel for solving linear equations, and leverage the strong parallel capability of the graphic processing unit (GPU) to achieve further speedup. We formulate the problem of how to get a fine parallel process as a tree-partition problem. Next, we present a heuristic partition algorithm to obtain an effective partition to efficiently parallelize the equation-solving process in detailed simulation. With further optimization on GPU, our HTP method achieves 2.2 to 8.5 folds speedup compared to the state-of-the-art GPU method and 36 to 660 folds speedup compared to the typical Hines algorithm.
Asunto(s)
Heurística , Árboles , Simulación por Computador , AlgoritmosRESUMEN
Neuromorphic vision sensor is a new bio-inspired imaging paradigm emerged in recent years. It uses the asynchronous spike signals instead of the traditional frame-based manner to achieve ultra-high speed sampling. Unlike the dynamic vision sensor (DVS) that perceives movement by imitating the retinal periphery, the spike camera was developed recently to perceive fine textures by simulating a small retinal region called the fovea. For this new type of neuromorphic camera, how to reconstruct ultra-high speed visual images from spike data becomes an important yet challenging issue in visual scene perception, analysis, and recognition applications. In this paper, a bio-inspired visual reconstruction framework for the spike camera is proposed for the first time. Its core idea is to use the biologically inspired adaptive adjustment mechanisms, combined with the spatiotemporal spike information extracted by the proposed model, to reconstruct the full texture of natural scenes in an ultra-high temporal resolution. Specifically, the proposed model consists of a motion local excitation layer, a spike refining layer and a visual reconstruction layer motivated by the bio-realistic leaky integrate-and-fire (LIF) neurons and synapse connection with spike-timing dependent plasticity (STDP) rule. To evaluate the performance, a spike dataset was constructed for normal and high-speed scenes in real-world recorded by the spike camera. The experimental results show that the proposed approach can reconstruct the visual images with 40,000 frames per second in both normal and high-speed scenes, while achieving high dynamic range and high image quality.
Asunto(s)
Algoritmos , Neuronas , Percepción Visual/fisiología , Retina , Modelos NeurológicosRESUMEN
Esophageal squamous cell carcinoma (ESCC) is one of the most common malignant tumors in China. However, there are no targets to treat ESCC because the molecular mechanism behind the cancer is still unclear. Here, we found a novel long noncoding RNA LINC02820 was upregulated in ESCC and associated with the ESCC clinicopathological stage. Through a series of functional experiments, we observed that LINC02820 only promoted the migration and invasion capabilities of ESCC cell lines. Mechanically, we found that LINC02820 may affect the cytoskeletal remodeling, interact with splice factor 3B subunit 3 (SF3B3), and cooperate with TNFα to amplify the NF-κB signaling pathway, which can lead to ESCC metastasis. Overall, our findings revealed that LINC02820 is a potential biomarker and therapeutic target for the diagnosis and treatment of ESCC.