PS-Net: human perception-guided segmentation network for EM cell membrane.

MOTIVATION: Cell membrane segmentation in electron microscopy (EM) images is a crucial step in EM image processing. However, while popular approaches have achieved performance comparable to that of humans on low-resolution EM datasets, they have shown limited success when applied to high-resolution EM datasets. The human visual system, on the other hand, displays consistently excellent performance on both low and high resolutions. To better understand this limitation, we conducted eye movement and perceptual consistency experiments. Our data showed that human observers are more sensitive to the structure of the membrane while tolerating misalignment, contrary to commonly used evaluation criteria. Additionally, our results indicated that the human visual system processes images in both global-local and coarse-to-fine manners. RESULTS: Based on these observations, we propose a computational framework for membrane segmentation that incorporates these characteristics of human perception. This framework includes a novel evaluation metric, the perceptual Hausdorff distance (PHD), and an end-to-end network called the PHD-guided segmentation network (PS-Net) that is trained using adaptively tuned PHD loss functions and a multiscale architecture. Our subjective experiments showed that the PHD metric is more consistent with human perception than other criteria, and our proposed PS-Net outperformed state-of-the-art methods on both low- and high-resolution EM image datasets as well as other natural image datasets. AVAILABILITY AND IMPLEMENTATION: The code and dataset can be found at https://github.com/EmmaSRH/PS-Net.

Heuristic Tree-Partition-Based Parallel Method for Biophysically Detailed Neuron Simulation.

Biophysically detailed neuron simulation is a powerful tool to explore the mechanisms behind biological experiments and bridge the gap between various scales in neuroscience research. However, the extremely high computational complexity of detailed neuron simulation restricts the modeling and exploration of detailed network models. The bottleneck is solving the system of linear equations. To accelerate detailed simulation, we propose a heuristic tree-partition-based parallel method (HTP) to parallelize the computation of the Hines algorithm, the kernel for solving linear equations, and leverage the strong parallel capability of the graphic processing unit (GPU) to achieve further speedup. We formulate the problem of how to get a fine parallel process as a tree-partition problem. Next, we present a heuristic partition algorithm to obtain an effective partition to efficiently parallelize the equation-solving process in detailed simulation. With further optimization on GPU, our HTP method achieves 2.2 to 8.5 folds speedup compared to the state-of-the-art GPU method and 36 to 660 folds speedup compared to the typical Hines algorithm.

An FPGA Accelerator for High-Speed Moving Objects Detection and Tracking With a Spike Camera.

Ultra-high-speed object detection and tracking are crucial in fields such as fault detection and scientific observation. Existing solutions to this task have deficiencies in processing speeds. To deal with this difficulty, we propose a neural-inspired ultra-high-speed moving object filtering, detection, and tracking scheme, as well as a corresponding accelerator based on a high-speed spike camera. We parallelize the filtering module and divide the detection module to accelerate the algorithm and balance latency among modules for the benefit of the task-level pipeline. To be specific, a block-based parallel computation model is proposed to accelerate the filtering module, and the detection module is accelerated by a parallel connected component labeling algorithm modeling spike sparsity and spatial connectivity of moving objects with a searching tree. The hardware optimizations include processing the LIF layer with a group of multiplexers to reduce ADD operations and replacing expensive exponential operations with multiplications of preprocessed fixed-point values to increase processing speed and minimize resource consumption. We design an accelerator with the above techniques, achieving 19 times acceleration over the serial version after 25-way parallelization. A processing system for the accelerator is also implemented on the Xilinx ZCU-102 board to validate its functionality and performance. Our accelerator can process more than 20,000 spike images with 250 × 400 resolution per second with 1.618 W dynamic power consumption.

Decoding Pixel-Level Image Features From Two-Photon Calcium Signals of Macaque Visual Cortex.

Images of visual scenes comprise essential features important for visual cognition of the brain. The complexity of visual features lies at different levels, from simple artificial patterns to natural images with different scenes. It has been a focus of using stimulus images to predict neural responses. However, it remains unclear how to extract features from neuronal responses. Here we address this question by leveraging two-photon calcium neural data recorded from the visual cortex of awake macaque monkeys. With stimuli including various categories of artificial patterns and diverse scenes of natural images, we employed a deep neural network decoder inspired by image segmentation technique. Consistent with the notation of sparse coding for natural images, a few neurons with stronger responses dominated the decoding performance, whereas decoding of ar tificial patterns needs a large number of neurons. When natural images using the model pretrained on artificial patterns are decoded, salient features of natural scenes can be extracted, as well as the conventional category information. Altogether, our results give a new perspective on studying neural encoding principles using reverse-engineering decoding strategies.

Terahertz non-label subwavelength imaging with composite photonics-plasmonics structured illumination.

Inspired by the capability of structured illumination microscopy (SIM) in subwavelength imaging, many researchers devoted themselves to investigating this methodology. However, due to the free-propagating feature of the traditional structured illumination fields, the resolution can be only improved up to two-fold of the diffraction-limited microscopy. Besides, most of the previous studies, relying on incoherent illumination sources, are restricted to fluorescent samples. In this work, a subwavelength non-fluorescent imaging method is proposed based on the illumination of terahertz traveling waves and plasmonics. Excited along with a metal grating, the spoof surface plasmons (SSPs) are employed as one of the illuminating sources. When the scattering waves with the SSPs illumination are captured, the sample's high-order spatial frequencies (SF) components are already encoded into the obtainable low-order ones. Then, a modified post-processing algorithm is exploited to shift the modulated SF components to their actual positions in the SF domain. In this manner, the fine information of samples is introduced to reconstruct the desired imaging, leading to an enhancement of the resolution up to 0.12λ0. Encouragingly, the resolution can be further enhanced by attaching extra illumination of SSPs with an elaborately selected frequency. This method holds promise for some important applications in terahertz non-fluorescent microscopy and sample detection with weak scattering.

Terahertz subwavelength edge detection based on dispersion-induced plasmons.

Terahertz imaging has recently attracted great attention owing to the abilities of high penetration and low ionizing damages. However, the low resolution and low contrast resulting from the diffraction limit and unwanted background illumination significantly hinder the extensive usage. In this Letter, we propose and numerically demonstrate a terahertz subwavelength imaging method capable of extracting only the edges and fine features of the targets. The underlying physics is the efficient transmission of the scattering evanescent waves related to key geometric information while blocking the propagating components. By exploiting the structurally induced plasmons in a bounded metallic waveguide, the transmission channel for evanescent waves is realized by hyperbolic metamaterials through periodically stacking dielectric layers. On this basis, high-contrast edge detection with a resolution up to ${0.1}\lambda$ is demonstrated at terahertz wavelengths. The proposed terahertz imaging method may find important applications in non-destructive testing, weak scattering object detection, and high-contrast microscopy.

Bifunctional Luneburg-fish-eye lens based on the manipulation of spoof surface plasmons.

Manipulation of spoof surface plasmons (SSPs) has recently intrigued enormous interest due to the capability of guiding waves with subwavelength footsteps. However, most of the previous studies, manifested for a single functionality, are not suitable for multifunctional integrated devices. Herein, a bifunctional Luneburg-fish-eye lens is proposed based on a 2D metal pillar array. First, by tuning the dimension of the metal pillars in the array, its ability to precisely manipulate the SSPs along one direction is confirmed, achieving subwavelength focusing and imaging with a resolution up to 0.14λ. Then, separately controlling the propagation of the SSPs along the orthotropic directions is further implemented, and the bifunctional Luneburg-fish-eye lens is realized. It is experimentally characterized as a Luneburg lens along the x axis, whereas in the y axis, it presents the properties of a Maxwell fish-eye lens. This bifunctional lens can reduce the system complexity and exert flexibility in multifunctional applications, while the proposed metal pillar-based design method broadens the application range of the gradient refractive-index lens in microwave, terahertz, and even optical ranges.

FUT11 is a target gene of HIF1α that promotes the progression of hepatocellular carcinoma.

Hypoxia promotes the progression of hepatocellular carcinoma. However, the hypoxia regulatory network in hepatocellular carcinoma is known to be limited. Thus, this study aimed to identify the crucial hypoxia-associated genes and to explore their effects and molecular mechanisms in hepatocellular carcinoma cells. FUT11 was first identified as a crucial hypoxia-associated gene through bioinformatics analysis. High FUT11 mRNA levels were positively correlated with poor clinical parameters. FUT11 knockdown under normoxia and hypoxia both decreased hepatocellular carcinoma cell proliferation, colony formation, migration, and invasion. HIF1α binds to the promoter of FUT11 and increases its transcription and co-expression with FUT11 in hepatocellular carcinoma tissues. Overexpression of FUT11 in HIF1α knockdown cells reversed the inhibitory effects of HIF1α suppression on hepatocellular carcinoma cell proliferation and mobility under hypoxia. Therefore, our findings indicate that FUT11 is a key target gene of HIF1α, which can promote the proliferation and mobility of hepatocellular carcinoma cells. FUT11 may be a novel and effective target for blocking the hypoxia response of hepatocellular carcinoma cells.

AKR1C2 acts as a targetable oncogene in esophageal squamous cell carcinoma via activating PI3K/AKT signaling pathway.

The aldo-keto reductases family 1 member C2 (AKR1C2) has critical roles in the tumorigenesis and progression of malignant tumours. However, it was also discovered to have ambiguous functions in multiple cancers and till present, its clinical significance and molecular mechanism in oesophageal squamous cell carcinoma (ESCC) has been unclear. The aim of this study was to explore the role of AKR1C2 in the tumorigenesis of ESCC. Here, we showed that AKR1C2 expression was found to be up-regulated in ESCC tissues and was significantly associated with pathological stage, lymph node metastasis and worse outcomes. Functional assays demonstrated that an ectopic expression of AKR1C2 in ESCC cells resulted in increased proliferation, migration and cisplatin resistance, while knockdown led to inversing effects. Bioinformation analyses and mechanistic studies demonstrated that AKR1C2 activated the PI3K/AKT signalling pathway, furthermore, the inhibitor of PI3K or the selective inhibitor of AKR1C2 enzyme activity could reverse the aggressiveness and showed synergistic antitumour effect when combined with cisplatin, both in vitro and in vivo. In conclusion, Our findings revealed that AKR1C2 could function as an oncogene by activating the PI3K/AKT pathway, as a novel prognostic biomarker and/or as a potential therapeutic target to ESCC.

High-efficiency terahertz spin-decoupled meta-coupler for spoof surface plasmon excitation and beam steering.

Spoof surface plasmon (SSP) meta-couplers that efficiently integrate other diversified functionalities into a single ultrathin device are highly desirable in the modern microwave and terahertz fields. However, the diversified functionalities, to the best of our knowledge, have not been applied to circular polarization meta-couplers because of the spin coupling between the orthogonal incident waves. In this paper, we propose and demonstrate a terahertz spin-decoupled bifunctional meta-coupler for SSP excitation and beam steering. The designed meta-coupler is composed of a coupling metasurface and a propagating metasurface. The former aims at realizing anomalous reflection or converting the incident waves into SSP under the illumination of the right or left circular polarization waves, respectively, and the latter are used to guide out the excited SSP. The respective converting efficiency can reach 82% and 70% at 0.3THz for the right and left circular polarization incident waves. Besides, by appropriately adjusting the reflection phase distribution, many other functionalities can also be integrated into the meta-coupler. Our study may open up new routes for polarization-related SSP couplers, detectors, and other practical terahertz devices.

Free-electron-driven beam-scanning terahertz radiation.

A beam-scanning terahertz (THz) radiation mechanism in a free-electron-driven grating system is proposed for THz applications. By loading a period-asynchronous rod array above the grating, the spoof surface plasmon (SSP) originally excited by the electron changes its radiation characteristics owing to the rod-induced Brillouin zone folding effect. The rod array functions as an antenna and converts the SSP into a spatial coherent THz radiation. The radiation frequency and direction can be precisely controlled by the electron energy. The field intensity of the radiation is increased approximately 20 times compared with that of the conventional Smith-Purcell radiation in the same frequency range. In addition, a microwave-band scaling prototype is fabricated and the frequency-controlled radiation is measured. Excellent agreement between the experimental and simulated results is obtained. This study paves the way for the development of on-chip THz sources for advanced communication and detection applications.

Terahertz multichannel metasurfaces with sparse unit cells.

Reflective multichannel metasurfaces are flat reflectors that can control incident and reflected waves in a number of propagating directions simultaneously. However, they are always densely discretized with a high spatial resolution, which increases the manufacturing complexity. In this Letter, to the best of our knowledge, a new method that combines the array antenna theory with the metagratings theory is proposed. We demonstrate that the unit cells with a linear gradient phase in each period of the metasurfaces can eliminate specific space harmonics. With this method, multichannel metasurfaces can be designed with sparse unit cells, and high efficiency is maintained simultaneously. As proofs of the method, we design three different terahertz multichannel metasurfaces with no more than three unit cells per period. The simplification of structures can efficiently reduce the manufacturing complexity. This work may open up new routes in designing multichannel metasurfaces.

High-efficiency directional excitation of spoof surface plasmons by periodic scattering cylinders.

In this Letter, we propose and experimentally demonstrate a simple but efficient method to excite spoof surface plasmons (SSP) through periodic metallic cylinders at microwave frequencies. The rigorous multiple scattering theory indicates that most of the incident propagating waves can pass the cylinders and be converted into the desired harmonics. Furthermore, by tuning the incident angle, controlling the directions of the excited SSP at different frequencies is also realized. The numerical simulations achieve a bidirectional efficiency of 90% at 9.68 GHz and unidirectional efficiency of 79%-85% at 7.46-9.7 GHz, when the incident angle changes from 60° to 120°. Meanwhile, the maximum contrast ratio between the powers of SSP launched in two opposite directions can reach up to 34 dB. The experimental results under 90° and 77.5° illuminations at 9.68 and 8.56 GHz provide strong support for the coupling mechanism. This method may provide technique support in the SSP-based communication and imaging systems.

Superfocusing of terahertz wave through spoof surface plasmons.

In this paper, we propose and numerically demonstrate a new way to realize superfocusing of terahertz waves via the spoof surface plasmons (SSP). With the assist of a modified subwavelength metallic grating, a near-field rapid oscillation can be formed, originating from the Fabry-Perot resonances due to the reflection of SSP waves at terminations. We show that the field pattern of oscillation on textured metallic surface can be engineered by adjusting groove width and grating number. This produces a desired modulation of phase and amplitude for the radiationless electromagnetic interference (REI) focusing. The effective focusing depth through the corrugated metal is evaluated by the full-width-half-maximum (FWHM) beamwidth. At the situation of third-order Fabry-Perot resonance, the FWMH reaches up to 0.069λ at a distance of 0.1λ, improving the beamwidth by more than 540% compared with a single slit. The FWHM is optimized to 0.06λ as the order of Fabry-Perot resonance becomes seven, leading to the superfocusing metric of 1.67. On the basis of this, we further show the focusing ability can be held on the ultra-thin metallic grating. Two-dimensional subwavelength focusing behavior is also numerically verified. Our study may extend the working distance of sensing and super-resolution imaging devices at terahertz frequency.

Terahertz super-resolution imaging using four-wave mixing in graphene.

A perfect lens made from negative refraction (NR) materials is utilized to overcome the diffraction limit. However, these NR lenses are realized by metamaterials, which suffer from high losses, and the volume is bulky. In this Letter, we propose a terahertz NR lens by using a four-wave mixing (FWM) process in graphene. NR is demonstrated because of the phase matching along the surface of graphene. Evanescent waves that store high spatial frequency information can be converted into propagating waves in the nonlinear NR process. An image with subwavelength resolution is reconstructed at the FWM wavelength. Theoretical analysis and numerical simulations are performed to demonstrate the capability of such imaging. The lens has a subwavelength resolution of around λ/5. The lens needs low field intensity due to the strong nonlinear response of graphene in the terahertz frequency. This Letter may have applications in terahertz microscopy.

Experimental demonstration of an ultra-broadband subwavelength resolution probe from microwave to terahertz regime.

An ultra-broadband subwavelength resolution probe that consists of a Teflon rod and six metallic strips is developed for the near-field imaging system. The slit between two metallic strips maintains quasi-TEM modes, avoiding the problem of low coupling efficiency caused by the cutoff effect. The numerical calculations visualize the process of energy compression into a 0.047λ diameter spot with great field enhancement at the taper apex, and the probe holds subwavelength focusing behavior from 10 GHz to 0.25 THz. Although limited by the fabrication, the resolution of 0.16 and 0.25λ are still experimentally demonstrated at 14 GHz and 0.1 THz. The properties of easy fabrication and no cutoff frequency would lower the threshold of a high-resolution near-field imaging system.

Optimization-Based Pairwise Interaction Point Process (O-PIPP): A Precise and Universal Retinal Mosaic Modeling Approach.

Purpose: A retinal mosaic, the spatial organization of a population of homotypic neurons, is thought to sample a specific visual feature into the feedforward visual pathway. The purpose of this study was to propose a universal modeling approach for precisely generating retinal mosaics and overcoming the limitations of previous models, especially in modeling abnormal mosaic patterns under disease conditions. Methods: Here, we developed the optimization-based pairwise interaction point process (O-PIPP). It incorporates optimization techniques into previous simulation approaches, enabling directional control of the simulation process according to the user-designed optimization target. For the convenience of the community, we implemented the O-PIPP approach into a Python package and a website application. Results: We showed that the O-PIPP can generate more precise neural spatial patterns of healthy and diseased mosaics compared to previous phenomenological approaches. Notably, through modeling the retinal neural circuitry with O-PIPP-simulated retinitis pigmentosa cone mosaics, we elucidated how the cone mosaic rearrangement impacted the information processing of ganglion cells. Conclusions: The O-PIPP provides a precise and universal tool to simulate realistic mosaics, which could help to investigate the function of retinal mosaics in vision.

Fully Spiking Actor Network With Intralayer Connections for Reinforcement Learning.

With the help of special neuromorphic hardware, spiking neural networks (SNNs) are expected to realize artificial intelligence (AI) with less energy consumption. It provides a promising energy-efficient way for realistic control tasks by combining SNNs with deep reinforcement learning (DRL). In this article, we focus on the task where the agent needs to learn multidimensional deterministic policies to control, which is very common in real scenarios. Recently, the surrogate gradient method has been utilized for training multilayer SNNs, which allows SNNs to achieve comparable performance with the corresponding deep networks in this task. Most existing spike-based reinforcement learning (RL) methods take the firing rate as the output of SNNs, and convert it to represent continuous action space (i.e., the deterministic policy) through a fully connected (FC) layer. However, the decimal characteristic of the firing rate brings the floating-point matrix operations to the FC layer, making the whole SNN unable to deploy on the neuromorphic hardware directly. To develop a fully spiking actor network (SAN) without any floating-point matrix operations, we draw inspiration from the nonspiking interneurons found in insects and employ the membrane voltage of the nonspiking neurons to represent the action. Before the nonspiking neurons, multiple population neurons are introduced to decode different dimensions of actions. Since each population is used to decode a dimension of action, we argue that the neurons in each population should be connected in time domain and space domain. Hence, the intralayer connections are used in output populations to enhance the representation capacity. This mechanism exists extensively in animals and has been demonstrated effectively. Finally, we propose a fully SAN with intralayer connections (ILC-SAN). Extensive experimental results demonstrate that the proposed method outperforms the state-of-the-art performance on continuous control tasks from OpenAI gym. Moreover, we estimate the theoretical energy consumption when deploying ILC-SAN on neuromorphic chips to illustrate its high energy efficiency.

Hybrid All-in-focus Imaging from Neuromorphic Focal Stack.

Creating an image focal stack requires multiple shots, which captures images at different depths within the same scene. Such methods are not suitable for scenes undergoing continuous changes. Achieving an all-in-focus image from a single shot poses significant challenges, due to the highly ill-posed nature of rectifying defocus and deblurring from a single image. In this paper, to restore an all-in-focus image, we introduce the neuromorphic focal stack, which is defined as neuromorphic signal streams captured by an event/ a spike camera during a continuous focal sweep, aiming to restore an all-in-focus image. Given an RGB image focused at any distance, we harness the high temporal resolution of neuromorphic signal streams. From neuromorphic signal streams, we automatically select refocusing timestamps and reconstruct corresponding refocused images to form a focal stack. Guided by the neuromorphic signal around the selected timestamps, we can merge the focal stack using proper weights and restore a sharp all-in-focus image. We test our method on two distinct neuromorphic cameras. Experimental results from both synthetic and real datasets demonstrate a marked improvement over existing state-of-the-art methods.