Meta-Attention Network Based Spectral Reconstruction with Snapshot Near-Infrared Metasurface.

Near-infrared (NIR) spectral information is important for detecting and analyzing material compositions. However, snapshot NIR spectral imaging systems still pose significant challenges owing to the lack of high-performance NIR filters and bulky setups, preventing effective encoding and integration with mobile devices. This study introduces a snapshot spectral imaging system that employs a compact NIR metasurface featuring 25 distinct C4 symmetry structures. Benefitting from the sufficient spectral variety and low correlation coefficient among these structures, center-wavelength accuracy of 0.05 nm and full width at half maximum accuracy of 0.13 nm are realized. The system maintains good performance within an incident angle of 1°. A novel meta-attention network prior iterative denoising reconstruction (MAN-IDR) algorithm is developed to achieve high-quality NIR spectral imaging. By leveraging the designed metasurface and MAN-IDR, the NIR spectral images, exhibiting precise textures, minimal artifacts in the spatial dimension, and little crosstalk between spectral channels, are reconstructed from a single grayscale recording image. The proposed NIR metasurface and MAN-IDR hold great promise for further integration with smartphones and drones, guaranteeing the adoption of NIR spectral imaging in real-world scenarios such as aerospace, health diagnostics, and machine vision.

Intracerebral hemorrhage-induced brain injury in mice: The role of peroxiredoxin 2-Toll-like receptor 4 inflammatory axis.

BACKGROUND: Peroxiredoxin 2 (Prx2), an intracellular protein that regulates redox reactions, released from red blood cells is involved in inflammatory brain injury after intracerebral hemorrhage (ICH). Toll-like receptor 4 (TLR4) may be crucial in this process. This study investigated the role of the Prx2-TLR4 inflammatory axis in brain injury following experimental ICH in mice. METHODS: First, C57BL/6 mice received an intracaudate injection of autologous arterial blood or saline and their brains were harvested on day 1 to measure Prx2 levels. Second, mice received an intracaudate injection of either recombinant mouse Prx2 or saline. Third, the mice were co-injected with autologous arterial blood and conoidin A, a Prx2 inhibitor, or vehicle. Fourth, the mice received a Prx2 injection and were treated with TAK-242, a TLR4 antagonist, or saline (intraperitoneally). Behavioral tests, magnetic resonance imaging, western blot, immunohistochemistry/immunofluorescence staining, and RNA sequencing (RNA-seq) were performed. RESULTS: Brain Prx2 levels were elevated after autologous arterial blood injection. Intracaudate injection of Prx2 caused brain swelling, microglial activation, neutrophil infiltration, neuronal death, and neurological deficits. Co-injection of conoidin A attenuated autologous arterial blood-induced brain injury. TLR4 was expressed on the surface of microglia/macrophages and neutrophils and participated in Prx2-induced inflammation. TAK-242 treatment attenuated Prx2-induced inflammation and neurological deficits. CONCLUSIONS: Prx2 can cause brain injury following ICH through the TLR4 pathway, revealing the Prx2-TLR4 inflammatory axis as a potential therapeutic target.

Global-optimal semi-supervised learning for single-pixel image-free sensing.

Single-pixel sensing offers low-cost detection and reliable perception, and the image-free sensing technique enhances its efficiency by extracting high-level features directly from compressed measurements. However, the conventional methods have great limitations in practical applications, due to their high dependence on large labelled data sources and incapability to do complex tasks. In this Letter, we report an image-free semi-supervised sensing framework based on GAN and achieve an end-to-end global optimization on the part-labelled datasets. Simulation on the MNIST realizes 94.91% sensing accuracy at 0.1 sampling ratio, with merely 0.3% of the dataset holding its classification label. When comparing to the conventional single-pixel sensing methods, the reported technique not only contributes to a high-robust result in both conventional (98.49% vs. 97.36%) and resource-constrained situations (94.91% vs. 83.83%) but also offers a more practical and powerful detection fashion for single-pixel sensing, with much less human effort and computation resources.

Sparse single-pixel imaging via optimization in nonuniform sampling sparsity.

Reducing the imaging time while maintaining reconstruction accuracy remains challenging for single-pixel imaging. One cost-effective approach is nonuniform sparse sampling. The existing methods lack intuitive and intrinsic analysis in sparsity. The lack impedes our comprehension of the form's adjustable range and may potentially limit our ability to identify an optimal distribution form within a confined adjustable range, consequently impacting the method's overall performance. In this Letter, we report a sparse sampling method with a wide adjustable range and define a sparsity metric to guide the selection of sampling forms. Through a comprehensive analysis and discussion, we select a sampling form that yields satisfying accuracy. These works will make up for the existing methods' lack of sparsity analysis and help adjust methods to accommodate different situations and needs.

Deep nonlocal low-rank regularization for complex-domain pixel super-resolution.

Pixel super-resolution (PSR) has emerged as a promising technique to break the sampling limit for phase imaging systems. However, due to the inherent nonconvexity of phase retrieval problem and super-resolution process, PSR algorithms are sensitive to noise, leading to reconstruction quality inevitably deteriorating. Following the plug-and-play framework, we introduce the nonlocal low-rank (NLR) regularization for accurate and robust PSR, achieving a state-of-the-art performance. Inspired by the NLR prior, we further develop the complex-domain nonlocal low-rank network (CNLNet) regularization to perform nonlocal similarity matching and low-rank approximation in the deep feature domain rather than the spatial domain of conventional NLR. Through visual and quantitative comparisons, CNLNet-based reconstruction shows an average 1.4 dB PSNR improvement over conventional NLR, outperforming existing algorithms under various scenarios.

High-resolution single-photon imaging with physics-informed deep learning.

High-resolution single-photon imaging remains a big challenge due to the complex hardware manufacturing craft and noise disturbances. Here, we introduce deep learning into SPAD, enabling super-resolution single-photon imaging with enhancement of bit depth and imaging quality. We first studied the complex photon flow model of SPAD electronics to accurately characterize multiple physical noise sources, and collected a real SPAD image dataset (64 × 32 pixels, 90 scenes, 10 different bit depths, 3 different illumination flux, 2790 images in total) to calibrate noise model parameters. With this physical noise model, we synthesized a large-scale realistic single-photon image dataset (image pairs of 5 different resolutions with maximum megapixels, 17250 scenes, 10 different bit depths, 3 different illumination flux, 2.6 million images in total) for subsequent network training. To tackle the severe super-resolution challenge of SPAD inputs with low bit depth, low resolution, and heavy noise, we further built a deep transformer network with a content-adaptive self-attention mechanism and gated fusion modules, which can dig global contextual features to remove multi-source noise and extract full-frequency details. We applied the technique in a series of experiments including microfluidic inspection, Fourier ptychography, and high-speed imaging. The experiments validate the technique's state-of-the-art super-resolution SPAD imaging performance.

Robust Kramers-Kronig holographic imaging with Hilbert-Huang transform.

Holography based on Kramers-Kronig relations (KKR) is a promising technique due to its high-space-bandwidth product. However, the absence of an iterative process limits its noise robustness, primarily stemming from the lack of a regularization constraint. This Letter reports a generalized framework aimed at enhancing the noise robustness of KKR holography. Our proposal involves employing the Hilbert-Huang transform to connect the real and imaginary parts of an analytic function. The real part is initially processed by bidimensional empirical mode decomposition into a series of intrinsic mode functions (IMFs) and a residual term. They are then selected to remove the noise and bias terms. Finally, the imaginary part can be obtained using the Hilbert transform. In this way, we efficiently suppress the noise in the synthetic complex function, facilitating high-fidelity wavefront reconstruction using ∼20% of the exposure time required by existing methods. Our work is expected to expand the applications of KKR holography, particularly in low phototoxicity biological imaging and other related scenarios.

Masked autoencoder for highly compressed single-pixel imaging.

The single-pixel imaging technique uses multiple patterns to modulate the entire scene and then reconstructs a two-dimensional (2-D) image from the single-pixel measurements. Inspired by the statistical redundancy of natural images that distinct regions of an image contain similar information, we report a highly compressed single-pixel imaging technique with a decreased sampling ratio. This technique superimposes an occluded mask onto modulation patterns, realizing that only the unmasked region of the scene is modulated and acquired. In this way, we can effectively decrease 75% modulation patterns experimentally. To reconstruct the entire image, we designed a highly sparse input and extrapolation network consisting of two modules: the first module reconstructs the unmasked region from one-dimensional (1-D) measurements, and the second module recovers the entire scene image by extrapolation from the neighboring unmasked region. Simulation and experimental results validate that sampling 25% of the region is enough to reconstruct the whole scene. Our technique exhibits significant improvements in peak signal-to-noise ratio (PSNR) of 1.5 dB and structural similarity index measure (SSIM) of 0.2 when compared with conventional methods at the same sampling ratios. The proposed technique can be widely applied in various resource-limited platforms and occluded scene imaging.

Self-supervised learning for single-pixel imaging via dual-domain constraints.

Deep-learning-augmented single-pixel imaging (SPI) provides an efficient solution for target compressive sensing. However, the conventional supervised strategy suffers from laborious training and poor generalization. In this Letter, we report a self-supervised learning method for SPI reconstruction. It introduces dual-domain constraints to integrate the SPI physics model into a neural network. Specifically, in addition to the traditional measurement constraint, an extra transformation constraint is employed to ensure target plane consistency. The transformation constraint uses the invariance of reversible transformation to impose an implicit prior, which avoids the non-uniqueness of measurement constraint. A series of experiments validate that the reported technique realizes self-supervised reconstruction in various complex scenes without any paired data, ground truth, or pre-trained prior. It can tackle the underdetermined degradation and noise, with ∼3.7-dB improvement on the PSNR index compared with the existing method.

Phase retrieval via nonlocal complex-domain sparsity.

Phase retrieval is indispensable for a number of coherent imaging systems. Owing to limited exposure, it is a challenge for traditional phase retrieval algorithms to reconstruct fine details in the presence of noise. In this Letter, we report an iterative framework for noise-robust phase retrieval with high fidelity. In the framework, we investigate nonlocal structural sparsity in the complex domain by low-rank regularization, which effectively suppresses artifacts caused by measurement noise. The joint optimization of sparsity regularization and data fidelity with forward models enables satisfying detail recovery. To further improve computational efficiency, we develop an adaptive iteration strategy that automatically adjusts matching frequency. The effectiveness of the reported technique has been validated for coherent diffraction imaging and Fourier ptychography, with ≈7 dB higher peak SNR (PSNR) on average, compared with conventional alternating projection reconstruction.

U-Shape Transformer for Underwater Image Enhancement.

The light absorption and scattering of underwater impurities lead to poor underwater imaging quality. The existing data-driven based underwater image enhancement (UIE) techniques suffer from the lack of a large-scale dataset containing various underwater scenes and high-fidelity reference images. Besides, the inconsistent attenuation in different color channels and space areas is not fully considered for boosted enhancement. In this work, we built a large scale underwater image (LSUI) dataset, which covers more abundant underwater scenes and better visual quality reference images than existing underwater datasets. The dataset contains 4279 real-world underwater image groups, in which each raw image's clear reference images, semantic segmentation map and medium transmission map are paired correspondingly. We also reported an U-shape Transformer network where the transformer model is for the first time introduced to the UIE task. The U-shape Transformer is integrated with a channel-wise multi-scale feature fusion transformer (CMSFFT) module and a spatial-wise global feature modeling transformer (SGFMT) module specially designed for UIE task, which reinforce the network's attention to the color channels and space areas with more serious attenuation. Meanwhile, in order to further improve the contrast and saturation, a novel loss function combining RGB, LAB and LCH color spaces is designed following the human vision principle. The extensive experiments on available datasets validate the state-of-the-art performance of the reported technique with more than 2dB superiority. The dataset and demo code are available at https://bianlab.github.io/.

Image-free single-pixel object detection.

Recently developed image-free sensing techniques have achieved remarkable performance in various vision tasks. However, existing image-free methods still cannot simultaneously obtain the category, location, and size information of all objects. In this Letter, we report a novel image-free single-pixel object detection (SPOD) technique. SPOD enables efficient and robust multi-object detection directly from a small number of measurements, eliminating the requirement for complicated image reconstruction. Different from the conventional full-size pattern sampling method, the reported small-size optimized pattern sampling method achieves higher image-free sensing accuracy with fewer pattern parameters (∼1 order of magnitude). Moreover, instead of simply stacking CNN layers, we design the SPOD network based on the transformer architecture. It can better model global features and reinforce the network's attention to the targets in the scene, thus improving the object detection performance. We demonstrate the effectiveness of SPOD on the Voc dataset, which achieves a detection accuracy of 82.41% mAP at a sampling rate of 5% with a refresh rate of 63 f.p.s.

Computational spectrometer based on local feature-weighted spectral reconstruction.

The computational spectrometer enables the reconstruction of spectra from precalibrated information encoded. In the last decade, it has emerged as an integrated and low-cost paradigm with vast potential for applications, especially in portable or handheld spectral analysis devices. The conventional methods utilize a local-weighted strategy in feature spaces. These methods overlook the fact that the coefficients of important features could be too large to reflect differences in more detailed feature spaces during calculations. In this work, we report a local feature-weighted spectral reconstruction (LFWSR) method, and construct a high-accuracy computational spectrometer. Different from existing methods, the reported method learns a spectral dictionary via L 4-norm maximization for representing spectral curve features, and considers the statistical ranking of features. According to the ranking, weight features and update coefficients then calculate the similarity. What's more, the inverse distance weighted is utilized to pick samples and weight a local training set. Finally, the final spectrum is reconstructed utilizing the local training set and measurements. Experiments indicate that the reported method's two weighting processes produce state-of-the-art high accuracy.

Tuning-free and self-supervised image enhancement against ill exposure.

Complex lighting conditions and the limited dynamic range of imaging devices result in captured images with ill exposure and information loss. Existing image enhancement methods based on histogram equalization, Retinex-inspired decomposition model, and deep learning suffer from manual tuning or poor generalization. In this work, we report an image enhancement method against ill exposure with self-supervised learning, enabling tuning-free correction. First, a dual illumination estimation network is constructed to estimate the illumination for under- and over-exposed areas. Thus, we get the corresponding intermediate corrected images. Second, given the intermediate corrected images with different best-exposed areas, Mertens' multi-exposure fusion strategy is utilized to fuse the intermediate corrected images to acquire a well-exposed image. The correction-fusion manner allows adaptive dealing with various types of ill-exposed images. Finally, the self-supervised learning strategy is studied which learns global histogram adjustment for better generalization. Compared to training on paired datasets, we only need ill-exposed images. This is crucial in cases where paired data is inaccessible or less than perfect. Experiments show that our method can reveal more details with better visual perception than other state-of-the-art methods. Furthermore, the weighted average scores of image naturalness matrics NIQE and BRISQUE, and contrast matrics CEIQ and NSS on five real-world image datasets are boosted by 7%, 15%, 4%, and 2%, respectively, over the recent exposure correction method.

Polarization-Dependent Metasurface Enables Near-Infrared Dual-Modal Single-Pixel Sensing.

Infrared single-pixel sensing with the two most representative modes, bright-field imaging and edge-enhanced imaging, has great application potential in biomedical diagnosis and defect inspection. Building a multifunctional and miniature optical computing device for infrared single-pixel sensing is extremely intriguing. Here, we propose and validate a dual-modal device based on a well-designed metasurface, which enables near-infrared bright-field and edge-enhanced single-pixel imaging. By changing the polarization of the incident beam, these two different modes can be switched. Simulations validate that our device can achieve high-fidelity dual-modal single-pixel sensing at 0.9 µm with certain noise robustness. We also investigate the generalization of our metasurface-based device and validate that different illumination patterns are applied to our device. Moreover, these output images by our device can be efficiently utilized for biomedical image segmentation. We envision this novel device may open a vista in dual-modal infrared single-pixel sensing.

INFWIDE: Image and Feature Space Wiener Deconvolution Network for Non-Blind Image Deblurring in Low-Light Conditions.

Under low-light environment, handheld photography suffers from severe camera shake under long exposure settings. Although existing deblurring algorithms have shown promising performance on well-exposed blurry images, they still cannot cope with low-light snapshots. Sophisticated noise and saturation regions are two dominating challenges in practical low-light deblurring: the former violates the Gaussian or Poisson assumption widely used in most existing algorithms and thus degrades their performance badly, while the latter introduces non-linearity to the classical convolution-based blurring model and makes the deblurring task even challenging. In this work, we propose a novel non-blind deblurring method dubbed image and feature space Wiener deconvolution network (INFWIDE) to tackle these problems systematically. In terms of algorithm design, INFWIDE proposes a two-branch architecture, which explicitly removes noise and hallucinates saturated regions in the image space and suppresses ringing artifacts in the feature space, and integrates the two complementary outputs with a subtle multi-scale fusion network for high quality night photograph deblurring. For effective network training, we design a set of loss functions integrating a forward imaging model and backward reconstruction to form a close-loop regularization to secure good convergence of the deep neural network. Further, to optimize INFWIDE's applicability in real low-light conditions, a physical-process-based low-light noise model is employed to synthesize realistic noisy night photographs for model training. Taking advantage of the traditional Wiener deconvolution algorithm's physically driven characteristics and deep neural network's representation ability, INFWIDE can recover fine details while suppressing the unpleasant artifacts during deblurring. Extensive experiments on synthetic data and real data demonstrate the superior performance of the proposed approach.

Robust phase unwrapping via non-local regularization.

Phase unwrapping is an indispensable step in recovering the true phase from a modulo-2π phase. Conventional phase unwrapping methods suffer from error propagation under severe noise. In this Letter, we propose an iterative framework for robust phase unwrapping with high fidelity. The proposed method utilizes the transport-of-intensity equation to solve the phase unwrapping problem with high computational efficiency. To further improve reconstruction accuracy, we take advantage of non-local structural similarity using low-rank regularization. Meanwhile, we use an adaptive iteration strategy that dynamically and automatically updates the denoising parameter to avoid over-smoothing and preserve image details. A set of simulation and experimental results validates the proposed method, which can provide satisfying results under severe noise conditions, and outperform existing state-of-the-art phase unwrapping methods with at least 6 dB higher peak SNR (PSNR).

Systemic inflammation and immune index predicting outcomes in patients with intracerebral hemorrhage.

OBJECT: Recent evidence has suggested that systemic inflammatory and immune index (SIRI) and systematic inflammation index (SII) could predict prognosis in stroke patients. This study aimed to determine the effects of SIRI and SII on predicting in-hospital infections and unfavorable outcomes in patients with acute intracerebral hemorrhage (ICH). METHODS: We used the data from a prospective and registry-based study recruiting ICH patients between January 2014 and September 2016 in a single comprehensive stroke center. All patients were stratified by quartiles of SIRI or SII. Logistic regression analysis was used to estimate the associations with follow-up prognosis. The receiver operating characteristics (ROC) curves were performed to examine the predictive utility of these indexes for infections and prognosis. RESULTS: Six hundred and forty spontaneous ICH patients were enrolled in this study. Compared with the lowest quartile (Q1), SIRI or SII values both showed positive correlations with increased risks for poor 1-month outcomes (adjusted ORs in Q4 was 2.162 [95% CI: 1.240-3.772] for SIRI, 1.797 [95% CI: 1.052-3.070] for SII). Additionally, a higher level of SIRI, but not SII, was independently associated with a higher risk of infections and an unfavorable 3-month prognosis. The C-statistic for the combined SIRI and ICH score was higher than SIRI or ICH score alone for predicting in-hospital infections and poor outcomes. CONCLUSION: Elevated SIRI values were associated with in-hospital infections and poor functional outcomes. It may provide a new biomarker for ICH prognosis prediction, especially in the acute stage.

Subarachnoid extension and unfavorable outcomes in patients with supratentorial intracerebral hemorrhage.

OBJECTIVE: Our study aimed to investigate the association between the subarachnoid extension of intracranial hemorrhage (SAHE) and clinical outcomes in patients with supratentorial intracerebral hemorrhage (ICH). METHODS: We analyzed the data from a prospective, multi-center, and registry-based database. Two experienced investigators independently assessed ICH imaging data. We compared baseline characteristics and follow-up outcomes. Multivariable logistic regression analysis was used to evaluate the association between SAHE and poor clinical outcomes. We also performed Kaplan-Meier curves and Cox proportional hazards regression analyses to analyze whether SAHE was relevant to a higher mortality rate. RESULTS: A total of 931 patients were included in this study (SAHE vs. no SAHE, 121 [13.0%] vs. 810 [87.0%]). Patients with SAHE had more severe neurological deficits, higher scores of the mRS, and more remarkable mortality rates at follow-up (all p values < 0.05). In multivariable-adjusted models, SAHE was independently associated with a higher risk of poor outcomes (adjusted OR [95%CI]: 2.030 [1.142-3.608] at 3 months; 2.348 [1.337-4.123] at 1 year). In addition, SAHE remained an independent association with an increased death rate at 1 year (adjusted HR [95%CI], 1.314[1.057-1.635]). In the subgroup analysis, the correlation between SAHE and prognosis exists in patients with lobar or deep ICH. CONCLUSIONS: SAHE is independently associated with poor outcomes in patients with supratentorial ICH. It may provide a promising target for developing new predictive tools targeting ICH.

Artificial intelligence-based recognition for variant pathogenicity of BRCA1 using AlphaFold2-predicted structures.

With the surge of the high-throughput sequencing technologies, many genetic variants have been identified in the past decade. The vast majority of these variants are defined as variants of uncertain significance (VUS), as their significance to the function or health of an organism is not known. It is urgently needed to develop intelligent models for the clinical interpretation of VUS. State-of-the-art artificial intelligence (AI)-based variant effect predictors only learn features from primary amino acid sequences, leaving out information about the most important three-dimensional structure that is more related to its function. Methods: We proposed a deep convolutional neural network model named variant effect recognition network for BRCA1 (vERnet-B) to recognize the clinical pathogenicity of missense single-nucleotide variants in the BRCT domain of BRCA1. vERnet-B learned features associated with the pathogenicity from the tertiary protein structures of variants predicted by AlphaFold2. Results: After performing a series of validation and analyses on vERnet-B, we discovered that it exhibited significant advances over previous works. Recognizing the phenotypic consequences of VUS is one of the most daunting challenges in genetic informatics; however, we achieved 85% accuracy in recognizing disease BRCA1 variants with an ideal balance of false-positive and true-positive detection rates. vERnet-B correctly recognized the pathogenicity of variant A1708E, which was poorly predicted by AlphaFold2 as previously described. The vERnet-B web server is freely available from URL: http://ai-lab.bjrz.org.cn/vERnet. Conclusions: We applied protein tertiary structures to successfully recognize the pathogenic missense SNVs, which were difficult to be addressed by classical approaches based on sequences. Our work demonstrated that AlphaFold2-predicted structures were expected to be used for rich feature learning and revealed unique insights into the clinical interpretation of VUS in disease-related genes, using vERnet-B as a discovery tool.