Learning a stable approximation of an existing but unknown inverse mapping: application to the half-time circular Radon transform.

Supervised deep learning-based methods have inspired a new wave of image reconstruction methods that implicitly learn effective regularization strategies from a set of training data. While they hold potential for improving image quality, they have also raised concerns regarding their robustness. Instabilities can manifest when learned methods are applied to find approximate solutions to ill-posed image reconstruction problems for which a unique and stable inverse mapping does not exist, which is a typical use case. In this study, we investigate the performance of supervised deep learning-based image reconstruction in an alternate use case in which a stable inverse mapping is known to exist but is not yet analytically available in closed form. For such problems, a deep learning-based method can learn a stable approximation of the unknown inverse mapping that generalizes well to data that differ significantly from the training set. The learned approximation of the inverse mapping eliminates the need to employ an implicit (optimization-based) reconstruction method and can potentially yield insights into the unknown analytic inverse formula. The specific problem addressed is image reconstruction from a particular case of radially truncated circular Radon transform (CRT) data, referred to as 'half-time' measurement data. For the half-time image reconstruction problem, we develop and investigate a learned filtered backprojection method that employs a convolutional neural network to approximate the unknown filtering operation. We demonstrate that this method behaves stably and readily generalizes to data that differ significantly from training data. The developed method may find application to wave-based imaging modalities that include photoacoustic computed tomography.

A Multivariate Functional Connectivity Approach to Mapping Brain Networks and Imputing Neural Activity in Mice.

Temporal correlation analysis of spontaneous brain activity (e.g., Pearson "functional connectivity," FC) has provided insights into the functional organization of the human brain. However, bivariate analysis techniques such as this are often susceptible to confounding physiological processes (e.g., sleep, Mayer-waves, breathing, motion), which makes it difficult to accurately map connectivity in health and disease as these physiological processes affect FC. In contrast, a multivariate approach to imputing individual neural networks from spontaneous neuroimaging data could be influential to our conceptual understanding of FC and provide performance advantages. Therefore, we analyzed neural calcium imaging data from Thy1-GCaMP6f mice while either awake, asleep, anesthetized, during low and high bouts of motion, or before and after photothrombotic stroke. A linear support vector regression approach was used to determine the optimal weights for integrating the signals from the remaining pixels to accurately predict neural activity in a region of interest (ROI). The resultant weight maps for each ROI were interpreted as multivariate functional connectivity (MFC), resembled anatomical connectivity, and demonstrated a sparser set of strong focused positive connections than traditional FC. While global variations in data have large effects on standard correlation FC analysis, the MFC mapping methods were mostly impervious. Lastly, MFC analysis provided a more powerful connectivity deficit detection following stroke compared to traditional FC.

Computing a projection operator onto the null space of a linear imaging operator: tutorial.

Many imaging systems can be approximately described by a linear operator that maps an object property to a collection of discrete measurements. However, even in the absence of measurement noise, such operators are generally "blind" to certain components of the object, and hence information is lost in the imaging process. Mathematically, this is explained by the fact that the imaging operator can possess a null space. All objects in the null space, by definition, are mapped to a collection of identically zero measurements and are hence invisible to the imaging system. As such, characterizing the null space of an imaging operator is of fundamental importance when comparing and/or designing imaging systems. A characterization of the null space can also facilitate the design of regularization strategies for image reconstruction methods. Characterizing the null space via an associated projection operator is, in general, a computationally demanding task. In this tutorial, computational procedures for establishing projection operators that map an object to the null space of a discrete-to-discrete imaging operator are surveyed. A new machine-learning-based approach that employs a linear autoencoder is also presented. The procedures are demonstrated by use of biomedical imaging examples, and their computational complexities and memory requirements are compared.

Decoding visual information from high-density diffuse optical tomography neuroimaging data.

BACKGROUND: Neural decoding could be useful in many ways, from serving as a neuroscience research tool to providing a means of augmented communication for patients with neurological conditions. However, applications of decoding are currently constrained by the limitations of traditional neuroimaging modalities. Electrocorticography requires invasive neurosurgery, magnetic resonance imaging (MRI) is too cumbersome for uses like daily communication, and alternatives like functional near-infrared spectroscopy (fNIRS) offer poor image quality. High-density diffuse optical tomography (HD-DOT) is an emerging modality that uses denser optode arrays than fNIRS to combine logistical advantages of optical neuroimaging with enhanced image quality. Despite the resulting promise of HD-DOT for facilitating field applications of neuroimaging, decoding of brain activity as measured by HD-DOT has yet to be evaluated. OBJECTIVE: To assess the feasibility and performance of decoding with HD-DOT in visual cortex. METHODS AND RESULTS: To establish the feasibility of decoding at the single-trial level with HD-DOT, a template matching strategy was used to decode visual stimulus position. A receiver operating characteristic (ROC) analysis was used to quantify the sensitivity, specificity, and reproducibility of binary visual decoding. Mean areas under the curve (AUCs) greater than 0.97 across 10 imaging sessions in a highly sampled participant were observed. ROC analyses of decoding across 5 participants established both reproducibility in multiple individuals and the feasibility of inter-individual decoding (mean AUCs > 0.7), although decoding performance varied between individuals. Phase-encoded checkerboard stimuli were used to assess more complex, non-binary decoding with HD-DOT. Across 3 highly sampled participants, the phase of a 60° wide checkerboard wedge rotating 10° per second through 360° was decoded with a within-participant error of 25.8±24.7°. Decoding between participants was also feasible based on permutation-based significance testing. CONCLUSIONS: Visual stimulus information can be decoded accurately, reproducibly, and across a range of detail (for both binary and non-binary outcomes) at the single-trial level (without needing to block-average test data) using HD-DOT data. These results lay the foundation for future studies of more complex decoding with HD-DOT and applications in clinical populations.

Comparison of data-acquisition designs for single-shot edge-illumination X-ray phase-contrast tomography.

Edge-illumination X-ray phase-contrast tomography (EIXPCT) is an emerging technique that enables practical phase-contrast imaging with laboratory-based X-ray sources. A joint reconstruction method was proposed for reconstructing EIXPCT images, enabling novel flexible data-acquisition designs. However, only limited efforts have been devoted to optimizing data-acquisition designs for use with the joint reconstruction method. In this study, several promising designs are introduced, such as the constant aperture position (CAP) strategy and the alternating aperture position (AAP) strategy covering different angular ranges. In computer-simulation studies, these designs are analyzed and compared. Experimental data are employed to test the designs in real-world applications. All candidate designs are also compared for their implementation complexity. The tradeoff between data-acquisition time and image quality is discussed.

Single-shot edge illumination x-ray phase-contrast tomography enabled by joint image reconstruction.

Edge illumination x-ray phase-contrast tomography (EIXPCT) is an emerging x-ray phase-contrast tomography technique for reconstructing the complex-valued x-ray refractive index distribution of an object. Conventional image reconstruction approaches for EIXPCT require multiple images to be acquired at each tomographic view angle. This contributes to prolonged data-acquisition times and elevated radiation doses, which can hinder in vivo applications. In this work, a new "single-shot" method is proposed for joint reconstruction (JR) of the real and imaginary-valued components of the refractive index distribution from a tomographic data set that contains only a single image acquired at each view angle. The proposed method is predicated on a nonlinear formulation of the inverse problem that is solved by using a gradient-based optimization method. The method is validated and investigated using computer-simulated and experimental EIXPCT data sets.

Joint reconstruction of the initial pressure and speed of sound distributions from combined photoacoustic and ultrasound tomography measurements.

The initial pressure and speed of sound (SOS) distributions cannot both be stably recovered from photoacoustic computed tomography (PACT) measurements alone. Adjunct ultrasound computed tomography (USCT) measurements can be employed to estimate the SOS distribution. Under the conventional image reconstruction approach for combined PACT/USCT systems, the SOS is estimated from the USCT measurements alone and the initial pressure is estimated from the PACT measurements by use of the previously estimated SOS. This approach ignores the acoustic information in the PACT measurements and may require many USCT measurements to accurately reconstruct the SOS. In this work, a joint reconstruction method where the SOS and initial pressure distributions are simultaneously estimated from combined PACT/USCT measurements is proposed. This approach allows accurate estimation of both the initial pressure distribution and the SOS distribution while requiring few USCT measurements.

Impact of nonstationary optical illumination on image reconstruction in optoacoustic tomography.

Optoacoustic tomography (OAT), also known as photoacoustic tomography, is a rapidly emerging hybrid imaging technique that possesses great potential for a wide range of biomedical imaging applications. In OAT, a laser is employed to illuminate the tissue of interest and acoustic signals are produced via the photoacoustic effect. From these data, an estimate of the distribution of the absorbed optical energy density within the tissue is reconstructed, referred to as the object function. This quantity is defined, in part, by the distribution of light fluence within the tissue that is established by the laser source. When performing three-dimensional imaging of large objects, such as a female human breast, it can be difficult to achieve a relatively uniform coverage of light fluence within the volume of interest when the position of the laser source is fixed. To circumvent this, researchers have proposed illumination schemes in which the relative position of the laser source and ultrasound probe is fixed, and both are rotated together to acquire a tomographic dataset. A problem with this rotating-illumination scheme is that the tomographic data are inconsistent; namely, the acoustic data recorded at each tomographic view angle (i.e., probe position) are produced by a distinct object function. In this work, the impact of this data inconsistency on image reconstruction accuracy is investigated systematically. This is accomplished by use of computer-simulation studies and application of mathematical results from the theory of microlocal analysis. These studies specify the set of image discontinuities that can be stably reconstructed with a nonstationary optical illumination setup. The study also includes a comparison of the ability of iterative and analytic image reconstruction methods to mitigate artifacts attributable to the data inconsistency.

A comparative study based on image quality and clinical task performance for CT reconstruction algorithms in radiotherapy.

CT image reconstruction is typically evaluated based on the ability to reduce the radiation dose to as-low-as-reasonably-achievable (ALARA) while maintaining acceptable image quality. However, the determination of common image quality metrics, such as noise, contrast, and contrast-to-noise ratio, is often insufficient for describing clinical radiotherapy task performance. In this study we designed and implemented a new comparative analysis method associating image quality, radiation dose, and patient size with radiotherapy task performance, with the purpose of guiding the clinical radiotherapy usage of CT reconstruction algorithms. The iDose4 iterative reconstruction algorithm was selected as the target for comparison, wherein filtered back-projection (FBP) reconstruction was regarded as the baseline. Both phantom and patient images were analyzed. A layer-adjustable anthropomorphic pelvis phantom capable of mimicking 38-58 cm lateral diameter-sized patients was imaged and reconstructed by the FBP and iDose4 algorithms with varying noise-reduction-levels, respectively. The resulting image sets were quantitatively assessed by two image quality indices, noise and contrast-to-noise ratio, and two clinical task-based indices, target CT Hounsfield number (for electron density determination) and structure contouring accuracy (for dose-volume calculations). Additionally, CT images of 34 patients reconstructed with iDose4 with six noise reduction levels were qualitatively evaluated by two radiation oncologists using a five-point scoring mechanism. For the phantom experiments, iDose4 achieved noise reduction up to 66.1% and CNR improvement up to 53.2%, compared to FBP without considering the changes of spatial resolution among images and the clinical acceptance of reconstructed images. Such improvements consistently appeared across different iDose4 noise reduction levels, exhibiting limited interlevel noise (< 5 HU) and target CT number variations (< 1 HU). The radiation dose required to achieve similar contouring accuracy decreased when using iDose4 in place of FBP, up to 32%. Contouring accuracy improvement for iDose4 images, when compared to FBP, was greater in larger patients than smaller-sized patients. Overall, the iDose4 algorithm provided superior radiation dose control while maintaining or improving task performance, when compared to FBP. The reader study on image quality improvement of patient cases shows that physicians preferred iDose4-reconstructed images on all cases compared to those from FBP algorithm with overall quality score: 1.21 vs. 3.15, p = 0.0022. However, qualitative evaluation strongly indicated that the radiation oncologists chose iDose4 noise reduction levels of 3-4 with additional consideration of task performance, instead of image quality metrics alone. Although higher iDose4 noise reduction levels improved the CNR through the further reduction of noise, there was pixelization of anatomical/tumor structures. Very-low-dose scans yielded severe photon starvation artifacts, which decreased target visualization on both FBP and iDose4 reconstructions, especially for the 58 cm phantom size. The iDose4 algorithm with a moderate noise reduction level is hence suggested for CT simulation and treatment planning. Quantitative task-based image quality metrics should be further investigated to accommodate additional clinical applications.

X-ray phase contrast imaging of calcified tissue and biomaterial structure in bioreactor engineered tissues.

Tissues engineered in bioreactor systems have been used clinically to replace damaged tissues and organs. In addition, these systems are under continued development for many tissue engineering applications. The ability to quantitatively assess material structure and tissue formation is critical for evaluating bioreactor efficacy and for preimplantation assessment of tissue quality. Techniques that allow for the nondestructive and longitudinal monitoring of large engineered tissues within the bioreactor systems will be essential for the translation of these strategies to viable clinical therapies. X-ray Phase Contrast (XPC) imaging techniques have shown tremendous promise for a number of biomedical applications owing to their ability to provide image contrast based on multiple X-ray properties, including absorption, refraction, and scatter. In this research, mesenchymal stem cell-seeded alginate hydrogels were prepared and cultured under osteogenic conditions in a perfusion bioreactor. The constructs were imaged at various time points using XPC microcomputed tomography (µCT). Imaging was performed with systems using both synchrotron- and tube-based X-ray sources. XPC µCT allowed for simultaneous three-dimensional (3D) quantification of hydrogel size and mineralization, as well as spatial information on hydrogel structure and mineralization. Samples were processed for histological evaluation and XPC showed similar features to histology and quantitative analysis consistent with the histomorphometry. These results provide evidence of the significant potential of techniques based on XPC for noninvasive 3D imaging engineered tissues grown in bioreactors.

A Test Statistic Estimation-Based Approach for Establishing Self-Interpretable CNN-Based Binary Classifiers.

Interpretability is highly desired for deep neural network-based classifiers, especially when addressing high-stake decisions in medical imaging. Commonly used post-hoc interpretability methods have the limitation that they can produce plausible but different interpretations of a given model, leading to ambiguity about which one to choose. To address this problem, a novel decision-theory-inspired approach is investigated to establish a self-interpretable model, given a pre-trained deep binary black-box medical image classifier. This approach involves utilizing a self-interpretable encoder-decoder model in conjunction with a single-layer fully connected network with unity weights. The model is trained to estimate the test statistic of the given trained black-box deep binary classifier to maintain a similar accuracy. The decoder output image, referred to as an equivalency map, is an image that represents a transformed version of the to-be-classified image that, when processed by the fixed fully connected layer, produces the same test statistic value as the original classifier. The equivalency map provides a visualization of the transformed image features that directly contribute to the test statistic value and, moreover, permits quantification of their relative contributions. Unlike the traditional post-hoc interpretability methods, the proposed method is self-interpretable, quantitative. Detailed quantitative and qualitative analyses have been performed with three different medical image binary classification tasks.

Investigating the use of signal detection information in supervised learning-based image denoising with consideration of task-shift.

Purpose: Recently, learning-based denoising methods that incorporate task-relevant information into the training procedure have been developed to enhance the utility of the denoised images. However, this line of research is relatively new and underdeveloped, and some fundamental issues remain unexplored. Our purpose is to yield insights into general issues related to these task-informed methods. This includes understanding the impact of denoising on objective measures of image quality (IQ) when the specified task at inference time is different from that employed for model training, a phenomenon we refer to as "task-shift." Approach: A virtual imaging test bed comprising a stylized computational model of a chest X-ray computed tomography imaging system was employed to enable a controlled and tractable study design. A canonical, fully supervised, convolutional neural network-based denoising method was purposely adopted to understand the underlying issues that may be relevant to a variety of applications and more advanced denoising or image reconstruction methods. Signal detection and signal detection-localization tasks under signal-known-statistically with background-known-statistically conditions were considered, and several distinct types of numerical observers were employed to compute estimates of the task performance. Studies were designed to reveal how a task-informed transfer-learning approach can influence the tradeoff between conventional and task-based measures of image quality within the context of the considered tasks. In addition, the impact of task-shift on these image quality measures was assessed. Results: The results indicated that certain tradeoffs can be achieved such that the resulting AUC value was significantly improved and the degradation of physical IQ measures was statistically insignificant. It was also observed that introducing task-shift degrades the task performance as expected. The degradation was significant when a relatively simple task was considered for network training and observer performance on a more complex one was assessed at inference time. Conclusions: The presented results indicate that the task-informed training method can improve the observer performance while providing control over the tradeoff between traditional and task-based measures of image quality. The behavior of a task-informed model fine-tuning procedure was demonstrated, and the impact of task-shift on task-based image quality measures was investigated.

Quantitative phase imaging techniques for measuring scattering properties of cells and tissues: a review-part II.

Significance: Quantitative phase imaging (QPI) is a non-invasive, label-free technique that provides intrinsic information about the sample under study. Such information includes the structure, function, and dynamics of the sample. QPI overcomes the limitations of conventional fluorescence microscopy in terms of phototoxicity to the sample and photobleaching of the fluorophore. As such, the application of QPI in estimating the three-dimensional (3D) structure and dynamics is well-suited for a range of samples from intracellular organelles to highly scattering multicellular samples while allowing for longer observation windows. Aim: We aim to provide a comprehensive review of 3D QPI and related phase-based measurement techniques along with a discussion of methods for the estimation of sample dynamics. Approach: We present information collected from 106 publications that cover the theoretical description of 3D light scattering and the implementation of related measurement techniques for the study of the structure and dynamics of the sample. We conclude with a discussion of the applications of the reviewed techniques in the biomedical field. Results: QPI has been successfully applied to 3D sample imaging. The scattering-based contrast provides measurements of intrinsic quantities of the sample that are indicative of disease state, stage of growth, or overall dynamics. Conclusions: We reviewed state-of-the-art QPI techniques for 3D imaging and dynamics estimation of biological samples. Both theoretical and experimental aspects of various techniques were discussed. We also presented the applications of the discussed techniques as applied to biomedicine and biology research.

Quantitative phase imaging techniques for measuring scattering properties of cells and tissues: a review-part I.

Significance: Quantitative phase imaging (QPI) techniques offer intrinsic information about the sample of interest in a label-free, noninvasive manner and have an enormous potential for wide biomedical applications with negligible perturbations to the natural state of the sample in vitro. Aim: We aim to present an in-depth review of the scattering formulation of light-matter interactions as applied to biological samples such as cells and tissues, discuss the relevant quantitative phase measurement techniques, and present a summary of various reported applications. Approach: We start with scattering theory and scattering properties of biological samples followed by an exploration of various microscopy configurations for 2D QPI for measurement of structure and dynamics. Results: We reviewed 157 publications and presented a range of QPI techniques and discussed suitable applications for each. We also presented the theoretical frameworks for phase reconstruction associated with the discussed techniques and highlighted their domains of validity. Conclusions: We provide detailed theoretical as well as system-level information for a wide range of QPI techniques. Our study can serve as a guideline for new researchers looking for an exhaustive literature review of QPI methods and relevant applications.

Assessing the Capacity of a Denoising Diffusion Probabilistic Model to Reproduce Spatial Context.

Diffusion models have emerged as a popular family of deep generative models (DGMs). In the literature, it has been claimed that one class of diffusion models-denoising diffusion probabilistic models (DDPMs)-demonstrate superior image synthesis performance as compared to generative adversarial networks (GANs). To date, these claims have been evaluated using either ensemble-based methods designed for natural images, or conventional measures of image quality such as structural similarity. However, there remains an important need to understand the extent to which DDPMs can reliably learn medical imaging domain-relevant information, which is referred to as 'spatial context' in this work. To address this, a systematic assessment of the ability of DDPMs to learn spatial context relevant to medical imaging applications is reported for the first time. A key aspect of the studies is the use of stochastic context models (SCMs) to produce training data. In this way, the ability of the DDPMs to reliably reproduce spatial context can be quantitatively assessed by use of post-hoc image analyses. Error-rates in DDPM-generated ensembles are reported, and compared to those corresponding to other modern DGMs. The studies reveal new and important insights regarding the capacity of DDPMs to learn spatial context. Notably, the results demonstrate that DDPMs hold significant capacity for generating contextually correct images that are 'interpolated' between training samples, which may benefit data-augmentation tasks in ways that GANs cannot.

High-Dimensional MR Reconstruction Integrating Subspace and Adaptive Generative Models.

OBJECTIVE: To develop a new method that integrates subspace and generative image models for high-dimensional MR image reconstruction. METHODS: We proposed a formulation that synergizes a low-dimensional subspace model of high-dimensional images, an adaptive generative image prior serving as spatial constraints on the sequence of "contrast-weighted" images or spatial coefficients of the subspace model, and a conventional sparsity regularization. A special pretraining plus subject-specific network adaptation strategy was proposed to construct an accurate generative-network-based representation for images with varying contrasts. An iterative algorithm was introduced to jointly update the subspace coefficients and the multi-resolution latent space of the generative image model that leveraged an recently proposed intermediate layer optimization technique for network inversion. RESULTS: We evaluated the utility of the proposed method for two high-dimensional imaging applications: accelerated MR parameter mapping and high-resolution MR spectroscopic imaging. Improved performance over state-of-the-art subspace-based methods was demonstrated in both cases. CONCLUSION: The proposed method provided a new way to address high-dimensional MR image reconstruction problems by incorporating an adaptive generative model as a data-driven spatial prior for constraining subspace reconstruction. SIGNIFICANCE: Our work demonstrated the potential of integrating data-driven and adaptive generative priors with canonical low-dimensional modeling for high-dimensional imaging problems.

Investigating the Use of Traveltime and Reflection Tomography for Deep Learning-Based Sound-Speed Estimation in Ultrasound Computed Tomography.

Ultrasound computed tomography (USCT) quantifies acoustic tissue properties such as the speed-of-sound (SOS). Although full-waveform inversion (FWI) is an effective method for accurate SOS reconstruction, it can be computationally challenging for large-scale problems. Deep learning-based image-to-image learned reconstruction (IILR) methods can offer computationally efficient alternatives. This study investigates the impact of the chosen input modalities on IILR methods for high-resolution SOS reconstruction in USCT. The selected modalities are traveltime tomography (TT) and reflection tomography (RT), which produce a low-resolution SOS map and a reflectivity map, respectively. These modalities have been chosen for their lower computational cost relative to FWI and their capacity to provide complementary information: TT offers a direct SOS measure, while RT reveals tissue boundary information. Systematic analyses were facilitated by employing a virtual USCT imaging system with anatomically realistic numerical breast phantoms. Within this testbed, a supervised convolutional neural network (CNN) was trained to map dual-channel (TT and RT images) to a high-resolution SOS map. Single-input CNNs were trained separately using inputs from each modality alone (TT or RT) for comparison. The accuracy of the methods was systematically assessed using normalized root mean squared error (NRMSE), structural similarity index measure (SSIM), and peak signal-to-noise ratio (PSNR). For tumor detection performance, receiver operating characteristic analysis was employed. The dual-channel IILR method was also tested on clinical human breast data. Ensemble average of the NRMSE, SSIM, and PSNR evaluated on this clinical dataset were 0.2355, 0.8845, and 28.33 dB, respectively.

Deep-learning-based image super-resolution of an end-expandable optical fiber probe for application in esophageal cancer diagnostics.

Significance: Endoscopic screening for esophageal cancer (EC) may enable early cancer diagnosis and treatment. While optical microendoscopic technology has shown promise in improving specificity, the limited field of view (<1 mm) significantly reduces the ability to survey large areas efficiently in EC screening. Aim: To improve the efficiency of endoscopic screening, we propose a novel concept of end-expandable endoscopic optical fiber probe for larger field of visualization and for the first time evaluate a deep-learning-based image super-resolution (DL-SR) method to overcome the issue of limited sampling capability. Approach: To demonstrate feasibility of the end-expandable optical fiber probe, DL-SR was applied on simulated low-resolution microendoscopic images to generate super-resolved (SR) ones. Varying the degradation model of image data acquisition, we identified the optimal parameters for optical fiber probe prototyping. The proposed screening method was validated with a human pathology reading study. Results: For various degradation parameters considered, the DL-SR method demonstrated different levels of improvement of traditional measures of image quality. The endoscopists' interpretations of the SR images were comparable to those performed on the high-resolution ones. Conclusions: This work suggests avenues for development of DL-SR-enabled sparse image reconstruction to improve high-yield EC screening and similar clinical applications.

Spatiotemporal image reconstruction to enable high-frame-rate dynamic photoacoustic tomography with rotating-gantry volumetric imagers.

Significance: Dynamic photoacoustic computed tomography (PACT) is a valuable imaging technique for monitoring physiological processes. However, current dynamic PACT imaging techniques are often limited to two-dimensional spatial imaging. Although volumetric PACT imagers are commercially available, these systems typically employ a rotating measurement gantry in which the tomographic data are sequentially acquired as opposed to being acquired simultaneously at all views. Because the dynamic object varies during the data-acquisition process, the sequential data-acquisition process poses substantial challenges to image reconstruction associated with data incompleteness. The proposed image reconstruction method is highly significant in that it will address these challenges and enable volumetric dynamic PACT imaging with existing preclinical imagers. Aim: The aim of this study is to develop a spatiotemporal image reconstruction (STIR) method for dynamic PACT that can be applied to commercially available volumetric PACT imagers that employ a sequential scanning strategy. The proposed reconstruction method aims to overcome the challenges caused by the limited number of tomographic measurements acquired per frame. Approach: A low-rank matrix estimation-based STIR (LRME-STIR) method is proposed to enable dynamic volumetric PACT. The LRME-STIR method leverages the spatiotemporal redundancies in the dynamic object to accurately reconstruct a four-dimensional (4D) spatiotemporal image. Results: The conducted numerical studies substantiate the LRME-STIR method's efficacy in reconstructing 4D dynamic images from tomographic measurements acquired with a rotating measurement gantry. The experimental study demonstrates the method's ability to faithfully recover the flow of a contrast agent with a frame rate of 10 frames per second, even when only a single tomographic measurement per frame is available. Conclusions: The proposed LRME-STIR method offers a promising solution to the challenges faced by enabling 4D dynamic imaging using commercially available volumetric PACT imagers. By enabling accurate STIRs, this method has the potential to significantly advance preclinical research and facilitate the monitoring of critical physiological biomarkers.

Attention-Based CNN-BiLSTM for Sleep State Classification of Spatiotemporal Wide-Field Calcium Imaging Data.

BACKGROUND: Wide-field calcium imaging (WFCI) with genetically encoded calcium indicators allows for spatiotemporal recordings of neuronal activity in mice. When applied to the study of sleep, WFCI data are manually scored into the sleep states of wakefulness, non-REM (NREM) and REM by use of adjunct EEG and EMG recordings. However, this process is time-consuming, invasive and often suffers from low inter- and intra-rater reliability. Therefore, an automated sleep state classification method that operates on spatiotemporal WFCI data is desired. NEW METHOD: A hybrid network architecture consisting of a convolutional neural network (CNN) to extract spatial features of image frames and a bidirectional long short-term memory network (BiLSTM) with attention mechanism to identify temporal dependencies among different time points was proposed to classify WFCI data into states of wakefulness, NREM and REM sleep. RESULTS: Sleep states were classified with an accuracy of 84% and Cohen's kappa of 0.64. Gradient-weighted class activation maps revealed that the frontal region of the cortex carries more importance when classifying WFCI data into NREM sleep while posterior area contributes most to the identification of wakefulness. The attention scores indicated that the proposed network focuses on short- and long-range temporal dependency in a state-specific manner. COMPARISON WITH EXISTING METHOD: On a 3-hour WFCI recording, the CNN-BiLSTM achieved a kappa of 0.67, comparable to a kappa of 0.65 corresponding to the human EEG/EMG-based scoring. CONCLUSIONS: The CNN-BiLSTM effectively classifies sleep states from spatiotemporal WFCI data and will enable broader application of WFCI in sleep.