Highly robust reconstruction framework for three-dimensional optical imaging based on physical model constrained neural networks.

Objective. The reconstruction of three-dimensional optical imaging that can quantitatively acquire the target distribution from surface measurements is a serious ill-posed problem. Traditional regularization-based reconstruction can solve such ill-posed problem to a certain extent, but its accuracy is highly dependent ona priorinformation, resulting in a less stable and adaptable method. Data-driven deep learning-based reconstruction avoids the errors of light propagation models and the reliance on experience and a prior by learning the mapping relationship between the surface light distribution and the target directly from the dataset. However, the acquisition of the training dataset and the training of the network itself are time consuming, and the high dependence of the network performance on the training dataset results in a low generalization ability. The objective of this work is to develop a highly robust reconstruction framework to solve the existing problems.Approach. This paper proposes a physical model constrained neural networks-based reconstruction framework. In the framework, the neural networks are to generate a target distribution from surface measurements, while the physical model is used to calculate the surface light distribution based on this target distribution. The mean square error between the calculated surface light distribution and the surface measurements is then used as a loss function to optimize the neural network. To further reduce the dependence ona prioriinformation, a movable region is randomly selected and then traverses the entire solution interval. We reconstruct the target distribution in this movable region and the results are used as the basis for its next movement.Main Results. The performance of the proposed framework is evaluated with a series of simulations andin vivoexperiment, including accuracy robustness of different target distributions, noise immunity, depth robustness, and spatial resolution. The results collectively demonstrate that the framework can reconstruct targets with a high accuracy, stability and versatility.Significance. The proposed framework has high accuracy and robustness, as well as good generalizability. Compared with traditional regularization-based reconstruction methods, it eliminates the need to manually delineate feasible regions and adjust regularization parameters. Compared with emerging deep learning assisted methods, it does not require any training dataset, thus saving a lot of time and resources and solving the problem of poor generalization and robustness of deep learning methods. Thus, the framework opens up a new perspective for the reconstruction of three-dimension optical imaging.

High-Resolution Swin Transformer for Automatic Medical Image Segmentation.

The resolution of feature maps is a critical factor for accurate medical image segmentation. Most of the existing Transformer-based networks for medical image segmentation adopt a U-Net-like architecture, which contains an encoder that converts the high-resolution input image into low-resolution feature maps using a sequence of Transformer blocks and a decoder that gradually generates high-resolution representations from low-resolution feature maps. However, the procedure of recovering high-resolution representations from low-resolution representations may harm the spatial precision of the generated segmentation masks. Unlike previous studies, in this study, we utilized the high-resolution network (HRNet) design style by replacing the convolutional layers with Transformer blocks, continuously exchanging feature map information with different resolutions generated by the Transformer blocks. The proposed Transformer-based network is named the high-resolution Swin Transformer network (HRSTNet). Extensive experiments demonstrated that the HRSTNet can achieve performance comparable with that of the state-of-the-art Transformer-based U-Net-like architecture on the 2021 Brain Tumor Segmentation dataset, the Medical Segmentation Decathlon's liver dataset, and the BTCV multi-organ segmentation dataset.

Phase function estimation from a diffuse optical image via deep learning.

Objective.The phase function is a key element of a light propagation model for Monte Carlo (MC) simulation, which is usually fitted with an analytic function with associated parameters. In recent years, machine learning methods were reported to estimate the parameters of the phase function of a particular form such as the Henyey-Greenstein phase function but, to our knowledge, no studies have been performed to determine the form of the phase function.Approach.Here we design a convolutional neural network (CNN) to estimate the phase function from a diffuse optical image without any explicit assumption on the form of the phase function. Specifically, we use a Gaussian mixture model (GMM) as an example to represent the phase function generally and learn the model parameters accurately. The GMM is selected because it provides the analytic expression of phase function to facilitate deflection angle sampling in MC simulation, and does not significantly increase the number of free parameters.Main Results.Our proposed method is validated on MC-simulated reflectance images of typical biological tissues using the Henyey-Greenstein phase function with different anisotropy factors. The mean squared error of the phase function is 0.01 and the relative error of the anisotropy factor is 3.28%.Significance.We propose the first data-driven CNN-based inverse MC model to estimate the form of scattering phase function. The effects of field of view and spatial resolution are analyzed and the findings provide guidelines for optimizing the experimental protocol in practical applications.

Special Patterns of Dynamic Brain Networks Discriminate Between Face and Non-face Processing: A Single-Trial EEG Study.

Face processing is a spatiotemporal dynamic process involving widely distributed and closely connected brain regions. Although previous studies have examined the topological differences in brain networks between face and non-face processing, the time-varying patterns at different processing stages have not been fully characterized. In this study, dynamic brain networks were used to explore the mechanism of face processing in human brain. We constructed a set of brain networks based on consecutive short EEG segments recorded during face and non-face (ketch) processing respectively, and analyzed the topological characteristic of these brain networks by graph theory. We found that the topological differences of the backbone of original brain networks (the minimum spanning tree, MST) between face and ketch processing changed dynamically. Specifically, during face processing, the MST was more line-like over alpha band in 0-100 ms time window after stimuli onset, and more star-like over theta and alpha bands in 100-200 and 200-300 ms time windows. The results indicated that the brain network was more efficient for information transfer and exchange during face processing compared with non-face processing. In the MST, the nodes with significant differences of betweenness centrality and degree were mainly located in the left frontal area and ventral visual pathway, which were involved in the face-related regions. In addition, the special MST patterns can discriminate between face and ketch processing by an accuracy of 93.39%. Our results suggested that special MST structures of dynamic brain networks reflected the potential mechanism of face processing in human brain.

A Thrombin-Responsive Nanoprobe for In Vivo Visualization of Thrombus Formation through Three-Dimensional Optical/Computed Tomography Hybrid Imaging.

Early spontaneous detection of thrombin activation benefits precise theranostics for thrombotic vascular disease. Herein, a thrombin-responsive nanoprobe conjugated by a FITC dye, PEGylated Fe3O4 nanoparticles, and a thrombin-sensitive peptide (LASG) was constructed to visualize thrombin activation and subsequent thrombosis in vivo. The FITC dye was linked to the LASG coated on the Fe3O4 nanoparticles for sensing the thrombin activity via the Förster resonance energy transfer effect. In vitro fluorescence imaging showed that the fluorescence signal intensity increased significantly after incubation with thrombin in contrast to that of the control group (p < 0.05), and the signal intensity was enhanced with the increase in thrombin concentration. Further in vivo fluorescence imaging also revealed that the signal elevated markedly in the left common carotid artery (LCCA) lesion of the mice thrombosis model after nanoprobe injection, in contrast to that of the control + nanoprobe group (p < 0.05). Moreover, the thrombin inhibitor bivalirudin could decrease the filling defect of the LCCA. Three-dimensional fusion images of micro-CT and fluorescence confirmed that filling defects in the LCCA were nicely colocalized with fluorescence signal caused by nanoprobes. The nanoplatform based on a thrombin-activatable visualization system could provide smart responsive and dynamic imaging of thrombosis in vivo.

How Native Background Affects Human Performance in Real-World Visual Object Detection: An Event-Related Potential Study.

Visual processing refers to the process of perceiving, analyzing, synthesizing, manipulating, transforming, and thinking of visual objects. It is modulated by both stimulus-driven and goal-directed factors and manifested in neural activities that extend from visual cortex to high-level cognitive areas. Extensive body of studies have investigated the neural mechanisms of visual object processing using synthetic or curated visual stimuli. However, synthetic or curated images generally do not accurately reflect the semantic links between objects and their backgrounds, and previous studies have not provided answers to the question of how the native background affects visual target detection. The current study bridged this gap by constructing a stimulus set of natural scenes with two levels of complexity and modulating participants' attention to actively or passively attend to the background contents. Behaviorally, the decision time was elongated when the background was complex or when the participants' attention was distracted from the detection task, and the object detection accuracy was decreased when the background was complex. The results of event-related potentials (ERP) analysis explicated the effects of scene complexity and attentional state on the brain responses in occipital and centro-parietal areas, which were suggested to be associated with varied attentional cueing and sensory evidence accumulation effects in different experimental conditions. Our results implied that efficient visual processing of real-world objects may involve a competition process between context and distractors that co-exist in the native background, and extensive attentional cues and fine-grained but semantically irrelevant scene information were perhaps detrimental to real-world object detection.

Short-range and long-range neuronal oscillatory coupling in multiple frequency bands during face perception.

Neuronal oscillatory activity has been considered to play a key role in face processing through its functional effect on information flow and exchange in human brain. Specifically, most neuronal oscillatory activity is measured in different rhythm based on the electrophysiological signal at single channel level. Although, the neuronal oscillatory coupling between neuronal assembles is associated with the information flow and exchange between brain regions, few studies focus on this type of neuronal oscillatory activity in face processing. In this study, the neuronal oscillatory coupling was investigated based on electroencephalographic (EEG) data of 20 participants, which were recorded when the participants were in a face/non-face perceptual task. The phase lag index (PLI) was used to assess the neuronal oscillatory coupling between brain regions in typical frequency bands. Enhanced short-range coupling was observed in theta (4-8 Hz) and alpha (8-12 Hz) band over the frontal region, in gamma1 (30-49 Hz) band over the left posterior and occipito-temporal regions, and in gamma2 (51-75 Hz) over the right temporal region during face perception compared with non-face perception. Long-range coupling was increased in theta and gamma band over the right hemisphere during face perception. Moreover, increased long-range coupling was observed in alpha band over the left and right hemisphere respectively during face perception. The results suggested that frequency-specific neuronal oscillatory coupling within and between regions of frontal cortex and the ventral visual pathway played an important role in face perception, which might reflect underlying neural mechanism of face perception.

Multi-atlas registration and adaptive hexahedral voxel discretization for fast bioluminescence tomography.

Bioluminescence tomography (BLT) has been a valuable optical molecular imaging technique to non-invasively depict the cellular and molecular processes in living animals with high sensitivity and specificity. Due to the inherent ill-posedness of BLT, a priori information of anatomical structure is usually incorporated into the reconstruction. The structural information is usually provided by computed tomography (CT) or magnetic resonance imaging (MRI). In order to obtain better quantitative results, BLT reconstruction with heterogeneous tissues needs to segment the internal organs and discretize them into meshes with the finite element method (FEM). It is time-consuming and difficult to handle the segmentation and discretization problems. In this paper, we present a fast reconstruction method for BLT based on multi-atlas registration and adaptive voxel discretization to relieve the complicated data processing procedure involved in the hybrid BLT/CT system. A multi-atlas registration method is first adopted to estimate the internal organ distribution of the imaged animal. Then, the animal volume is adaptively discretized into hexahedral voxels, which are fed into FEM for the following BLT reconstruction. The proposed method is validated in both numerical simulation and an in vivo study. The results demonstrate that the proposed method can reconstruct the bioluminescence source efficiently with satisfactory accuracy.

Hybrid simplified spherical harmonics with diffusion equation for light propagation in tissues.

Aiming at the limitations of the simplified spherical harmonics approximation (SPN) and diffusion equation (DE) in describing the light propagation in tissues, a hybrid simplified spherical harmonics with diffusion equation (HSDE) based diffuse light transport model is proposed. In the HSDE model, the living body is first segmented into several major organs, and then the organs are divided into high scattering tissues and other tissues. DE and SPN are employed to describe the light propagation in these two kinds of tissues respectively, which are finally coupled using the established boundary coupling condition. The HSDE model makes full use of the advantages of SPN and DE, and abandons their disadvantages, so that it can provide a perfect balance between accuracy and computation time. Using the finite element method, the HSDE is solved for light flux density map on body surface. The accuracy and efficiency of the HSDE are validated with both regular geometries and digital mouse model based simulations. Corresponding results reveal that a comparable accuracy and much less computation time are achieved compared with the SPN model as well as a much better accuracy compared with the DE one.

Molecular Optical Simulation Environment (MOSE): a platform for the simulation of light propagation in turbid media.

The study of light propagation in turbid media has attracted extensive attention in the field of biomedical optical molecular imaging. In this paper, we present a software platform for the simulation of light propagation in turbid media named the "Molecular Optical Simulation Environment (MOSE)". Based on the gold standard of the Monte Carlo method, MOSE simulates light propagation both in tissues with complicated structures and through free-space. In particular, MOSE synthesizes realistic data for bioluminescence tomography (BLT), fluorescence molecular tomography (FMT), and diffuse optical tomography (DOT). The user-friendly interface and powerful visualization tools facilitate data analysis and system evaluation. As a major measure for resource sharing and reproducible research, MOSE aims to provide freeware for research and educational institutions, which can be downloaded at http://www.mosetm.net.

Influence investigation of a void region on modeling light propagation in a heterogeneous medium.

A void region exists in some biological tissues, and previous studies have shown that inaccurate images would be obtained if it were not processed. A hybrid radiosity-diffusion method (HRDM) that couples the radiosity theory and the diffusion equation has been proposed to deal with the void problem and has been well demonstrated in two-dimensional and three-dimensional (3D) simple models. However, the extent of the impact of the void region on the accuracy of modeling light propagation has not been investigated. In this paper, we first implemented and verified the HRDM in 3D models, including both the regular geometries and a digital mouse model, and then investigated the influences of the void region on modeling light propagation in a heterogeneous medium. Our investigation results show that the influence of the region can be neglected when the size of the void is less than a certain range, and other cases must be taken into account.