HyperSLICE: HyperBand optimized spiral for low-latency interactive cardiac examination.

PURPOSE: Interactive cardiac MRI is used for fast scan planning and MR-guided interventions. However, the requirement for real-time acquisition and near-real-time visualization constrains the achievable spatio-temporal resolution. This study aims to improve interactive imaging resolution through optimization of undersampled spiral sampling and leveraging of deep learning for low-latency reconstruction (deep artifact suppression). METHODS: A variable density spiral trajectory was parametrized and optimized via HyperBand to provide the best candidate trajectory for rapid deep artifact suppression. Training data consisted of 692 breath-held CINEs. The developed interactive sequence was tested in simulations and prospectively in 13 subjects (10 for image evaluation, 2 during catheterization, 1 during exercise). In the prospective study, the optimized framework-HyperSLICE- was compared with conventional Cartesian real-time and breath-hold CINE imaging in terms quantitative and qualitative image metrics. Statistical differences were tested using Friedman chi-squared tests with post hoc Nemenyi test (p < 0.05). RESULTS: In simulations the normalized RMS error, peak SNR, structural similarity, and Laplacian energy were all statistically significantly higher using optimized spiral compared to radial and uniform spiral sampling, particularly after scan plan changes (structural similarity: 0.71 vs. 0.45 and 0.43). Prospectively, HyperSLICE enabled a higher spatial and temporal resolution than conventional Cartesian real-time imaging. The pipeline was demonstrated in patients during catheter pull back, showing sufficiently fast reconstruction for interactive imaging. CONCLUSION: HyperSLICE enables high spatial and temporal resolution interactive imaging. Optimizing the spiral sampling enabled better overall image quality and superior handling of image transitions compared with radial and uniform spiral trajectories.

FReSCO: Flow Reconstruction and Segmentation for low-latency Cardiac Output monitoring using deep artifact suppression and segmentation.

PURPOSE: Real-time monitoring of cardiac output (CO) requires low-latency reconstruction and segmentation of real-time phase-contrast MR, which has previously been difficult to perform. Here we propose a deep learning framework for "FReSCO" (Flow Reconstruction and Segmentation for low latency Cardiac Output monitoring). METHODS: Deep artifact suppression and segmentation U-Nets were independently trained. Breath-hold spiral phase-contrast MR data (N = 516) were synthetically undersampled using a variable-density spiral sampling pattern and gridded to create aliased data for training of the artifact suppression U-net. A subset of the data (N = 96) was segmented and used to train the segmentation U-net. Real-time spiral phase-contrast MR was prospectively acquired and then reconstructed and segmented using the trained models (FReSCO) at low latency at the scanner in 10 healthy subjects during rest, exercise, and recovery periods. Cardiac output obtained via FReSCO was compared with a reference rest CO and rest and exercise compressed-sensing CO. RESULTS: The FReSCO framework was demonstrated prospectively at the scanner. Beat-to-beat heartrate, stroke volume, and CO could be visualized with a mean latency of 622 ms. No significant differences were noted when compared with reference at rest (bias = -0.21 ± 0.50 L/min, p = 0.246) or compressed sensing at peak exercise (bias = 0.12 ± 0.48 L/min, p = 0.458). CONCLUSIONS: The FReSCO framework was successfully demonstrated for real-time monitoring of CO during exercise and could provide a convenient tool for assessment of the hemodynamic response to a range of stressors.

Unsupervised knowledge-transfer for learned image reconstruction.

Deep learning-based image reconstruction approaches have demonstrated impressive empirical performance in many imaging modalities. These approaches usually require a large amount of high-quality paired training data, which is often not available in medical imaging. To circumvent this issue we develop a novel unsupervised knowledge-transfer paradigm for learned reconstruction within a Bayesian framework. The proposed approach learns a reconstruction network in two phases. The first phase trains a reconstruction network with a set of ordered pairs comprising of ground truth images of ellipses and the corresponding simulated measurement data. The second phase fine-tunes the pretrained network to more realistic measurement data without supervision. By construction, the framework is capable of delivering predictive uncertainty information over the reconstructed image. We present extensive experimental results on low-dose and sparse-view computed tomography showing that the approach is competitive with several state-of-the-art supervised and unsupervised reconstruction techniques. Moreover, for test data distributed differently from the training data, the proposed framework can significantly improve reconstruction quality not only visually, but also quantitatively in terms of PSNR and SSIM, when compared with learned methods trained on the synthetic dataset only.

Real-time deep artifact suppression using recurrent U-Nets for low-latency cardiac MRI.

PURPOSE: Real-time low latency MRI is performed to guide various cardiac interventions. Real-time acquisitions often require iterative image reconstruction strategies, which lead to long reconstruction times. In this study, we aim to reconstruct highly undersampled radial real-time data with low latency using deep learning. METHODS: A 2D U-Net with convolutional long short-term memory layers is proposed to exploit spatial and preceding temporal information to reconstruct highly accelerated tiny golden radial data with low latency. The network was trained using a dataset of breath-hold CINE data (including 770 time series from 7 different orientations). Synthetic paired data were created by retrospectively undersampling the magnitude images, and the network was trained to recover the target images. In the spirit of interventional imaging, the network was trained and tested for varying acceleration rates and orientations. Data were prospectively acquired and reconstructed in real time in 1 healthy subject interactively and in 3 patients who underwent catheterization. Images were visually compared to sliding window and compressed sensing reconstructions and a conventional Cartesian real-time sequence. RESULTS: The proposed network generalized well to different acceleration rates and unseen orientations for all considered metrics in simulated data (less than 4% reduction in structural similarity index compared to similar acceleration and orientation-specific networks). The proposed reconstruction was demonstrated interactively, successfully depicting catheters in vivo with low latency (39 ms, including 19 ms for deep artifact suppression) and an image quality comparing favorably to other reconstructions. CONCLUSION: Deep artifact suppression was successfully demonstrated in the time-critical application of non-Cartesian real-time interventional cardiac MR.

(An overview of) Synergistic reconstruction for multimodality/multichannel imaging methods.

Imaging is omnipresent in modern society with imaging devices based on a zoo of physical principles, probing a specimen across different wavelengths, energies and time. Recent years have seen a change in the imaging landscape with more and more imaging devices combining that which previously was used separately. Motivated by these hardware developments, an ever increasing set of mathematical ideas is appearing regarding how data from different imaging modalities or channels can be synergistically combined in the image reconstruction process, exploiting structural and/or functional correlations between the multiple images. Here we review these developments, give pointers to important challenges and provide an outlook as to how the field may develop in the forthcoming years. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 1'.

Rapid whole-heart CMR with single volume super-resolution.

BACKGROUND: Three-dimensional, whole heart, balanced steady state free precession (WH-bSSFP) sequences provide delineation of intra-cardiac and vascular anatomy. However, they have long acquisition times. Here, we propose significant speed-ups using a deep-learning single volume super-resolution reconstruction, to recover high-resolution features from rapidly acquired low-resolution WH-bSSFP images. METHODS: A 3D residual U-Net was trained using synthetic data, created from a library of 500 high-resolution WH-bSSFP images by simulating 50% slice resolution and 50% phase resolution. The trained network was validated with 25 synthetic test data sets. Additionally, prospective low-resolution data and high-resolution data were acquired in 40 patients. In the prospective data, vessel diameters, quantitative and qualitative image quality, and diagnostic scoring was compared between the low-resolution, super-resolution and reference high-resolution WH-bSSFP data. RESULTS: The synthetic test data showed a significant increase in image quality of the low-resolution images after super-resolution reconstruction. Prospectively acquired low-resolution data was acquired ~× 3 faster than the prospective high-resolution data (173 s vs 488 s). Super-resolution reconstruction of the low-resolution data took < 1 s per volume. Qualitative image scores showed super-resolved images had better edge sharpness, fewer residual artefacts and less image distortion than low-resolution images, with similar scores to high-resolution data. Quantitative image scores showed super-resolved images had significantly better edge sharpness than low-resolution or high-resolution images, with significantly better signal-to-noise ratio than high-resolution data. Vessel diameters measurements showed over-estimation in the low-resolution measurements, compared to the high-resolution data. No significant differences and no bias was found in the super-resolution measurements in any of the great vessels. However, a small but significant for the underestimation was found in the proximal left coronary artery diameter measurement from super-resolution data. Diagnostic scoring showed that although super-resolution did not improve accuracy of diagnosis, it did improve diagnostic confidence compared to low-resolution imaging. CONCLUSION: This paper demonstrates the potential of using a residual U-Net for super-resolution reconstruction of rapidly acquired low-resolution whole heart bSSFP data within a clinical setting. We were able to train the network using synthetic training data from retrospective high-resolution whole heart data. The resulting network can be applied very quickly, making these techniques particularly appealing within busy clinical workflow. Thus, we believe that this technique may help speed up whole heart CMR in clinical practice.

Real-time cardiovascular MR with spatio-temporal artifact suppression using deep learning-proof of concept in congenital heart disease.

PURPOSE: Real-time assessment of ventricular volumes requires high acceleration factors. Residual convolutional neural networks (CNN) have shown potential for removing artifacts caused by data undersampling. In this study, we investigated the ability of CNNs to reconstruct highly accelerated radial real-time data in patients with congenital heart disease (CHD). METHODS: A 3D (2D plus time) CNN architecture was developed and trained using synthetic training data created from previously acquired breath hold cine images from 250 CHD patients. The trained CNN was then used to reconstruct actual real-time, tiny golden angle (tGA) radial SSFP data (13 × undersampled) acquired in 10 new patients with CHD. The same real-time data was also reconstructed with compressed sensing (CS) to compare image quality and reconstruction time. Ventricular volume measurements made using both the CNN and CS reconstructed images were compared to reference standard breath hold data. RESULTS: It was feasible to train a CNN to remove artifact from highly undersampled radial real-time data. The overall reconstruction time with the CNN (including creation of aliased images) was shown to be >5 × faster than the CS reconstruction. In addition, the image quality and accuracy of biventricular volumes measured from the CNN reconstructed images were superior to the CS reconstructions. CONCLUSION: This article has demonstrated the potential for the use of a CNN for reconstruction of real-time radial data within the clinical setting. Clinical measures of ventricular volumes using real-time data with CNN reconstruction are not statistically significantly different from gold-standard, cardiac-gated, breath-hold techniques.

Model-based reconstruction framework for correction of signal pile-up and geometric distortions in prostate diffusion MRI.

PURPOSE: Prostate diffusion-weighted MRI scans can suffer from geometric distortions, signal pileup, and signal dropout attributed to differences in tissue susceptibility values at the interface between the prostate and rectal air. The aim of this work is to present and validate a novel model based reconstruction method that can correct for these distortions. METHODS: In regions of severe signal pileup, standard techniques for distortion correction have difficulty recovering the underlying true signal. Furthermore, because of drifts and inaccuracies in the determination of center frequency, echo planar imaging (EPI) scans can be shifted in the phase-encoding direction. In this work, using a B0 field map and a set of EPI data acquired with blip-up and blip-down phase encoding gradients, we model the distortion correction problem linking the distortion-free image to the acquired raw corrupted k-space data and solve it in a manner analogous to the sensitivity encoding method. Both a quantitative and qualitative assessment of the proposed method is performed in vivo in 10 patients. RESULTS: Without distortion correction, mean Dice similarity scores between a reference T2W and the uncorrected EPI images were 0.64 and 0.60 for b-values of 0 and 500 s/mm2 , respectively. Compared to the Topup (distortion correction method commonly used for neuro imaging), the proposed method achieved Dice scores (0.87 and 0.85 versus 0.82 and 0.80) and better qualitative results in patients where signal pileup was present because of high rectal gas residue. CONCLUSION: Model-based reconstruction can be used for distortion correction in prostate diffusion MRI.

Dynamic causal modelling on infant fNIRS data: A validation study on a simultaneously recorded fNIRS-fMRI dataset.

Tracking the connectivity of the developing brain from infancy through childhood is an area of increasing research interest, and fNIRS provides an ideal method for studying the infant brain as it is compact, safe and robust to motion. However, data analysis methods for fNIRS are still underdeveloped compared to those available for fMRI. Dynamic causal modelling (DCM) is an advanced connectivity technique developed for fMRI data, that aims to estimate the coupling between brain regions and how this might be modulated by changes in experimental conditions. DCM has recently been applied to adult fNIRS, but not to infants. The present paper provides a proof-of-principle for the application of this method to infant fNIRS data and a demonstration of the robustness of this method using a simultaneously recorded fMRI-fNIRS single case study, thereby allowing the use of this technique in future infant studies. fMRI and fNIRS were simultaneously recorded from a 6-month-old sleeping infant, who was presented with auditory stimuli in a block design. Both fMRI and fNIRS data were preprocessed using SPM, and analysed using a general linear model approach. The main challenges that adapting DCM for fNIRS infant data posed included: (i) the import of the structural image of the participant for spatial pre-processing, (ii) the spatial registration of the optodes on the structural image of the infant, (iii) calculation of an accurate 3-layer segmentation of the structural image, (iv) creation of a high-density mesh as well as (v) the estimation of the NIRS optical sensitivity functions. To assess our results, we compared the values obtained for variational Free Energy (F), Bayesian Model Selection (BMS) and Bayesian Model Average (BMA) with the same set of possible models applied to both the fMRI and fNIRS datasets. We found high correspondence in F, BMS, and BMA between fMRI and fNIRS data, therefore showing for the first time high reliability of DCM applied to infant fNIRS data. This work opens new avenues for future research on effective connectivity in infancy by contributing a data analysis pipeline and guidance for applying DCM to infant fNIRS data.

Slice-illuminated optical projection tomography.

To improve the imaging performance of optical projection tomography (OPT) in live samples, we have explored a parallelized implementation of semi-confocal line illumination and detection to discriminate against scattered photons. Slice-illuminated OPT (sl-OPT) improves reconstruction quality in scattering samples by reducing interpixel crosstalk at the cost of increased acquisition time. For in vivo imaging, this can be ameliorated through the use of compressed sensing on angularly undersampled OPT data sets. Here, we demonstrate sl-OPT applied to 3D imaging of bead phantoms and live adult zebrafish.

Three dimensional photoacoustic tomography in Bayesian framework.

The image reconstruction problem (or inverse problem) in photoacoustic tomography is to resolve the initial pressure distribution from detected ultrasound waves generated within an object due to an illumination by a short light pulse. Recently, a Bayesian approach to photoacoustic image reconstruction with uncertainty quantification was proposed and studied with two dimensional numerical simulations. In this paper, the approach is extended to three spatial dimensions and, in addition to numerical simulations, experimental data are considered. The solution of the inverse problem is obtained by computing point estimates, i.e., maximum a posteriori estimate and posterior covariance. These are computed iteratively in a matrix-free form using a biconjugate gradient stabilized method utilizing the adjoint of the acoustic forward operator. The results show that the Bayesian approach can produce accurate estimates of the initial pressure distribution in realistic measurement geometries and that the reliability of these estimates can be assessed.

Multiple-view diffuse optical tomography system based on time-domain compressive measurements.

Compressive sensing is a powerful tool to efficiently acquire and reconstruct an image even in diffuse optical tomography (DOT) applications. In this work, a time-resolved DOT system based on structured light illumination, compressive detection, and multiple view acquisition has been proposed and experimentally validated on a biological tissue-mimicking phantom. The experimental scheme is based on two digital micromirror devices for illumination and detection modulation, in combination with a time-resolved single element detector. We fully validated the method and demonstrated both the imaging and tomographic capabilities of the system, providing state-of-the-art reconstruction quality.

Fast silicon photomultiplier improves signal harvesting and reduces complexity in time-domain diffuse optics.

We present a proof of concept prototype of a time-domain diffuse optics probe exploiting a fast Silicon PhotoMultiplier (SiPM), featuring a timing resolution better than 80 ps, a fast tail with just 90 ps decay time-constant and a wide active area of 1 mm2. The detector is hosted into the probe and used in direct contact with the sample under investigation, thus providing high harvesting efficiency by exploiting the whole SiPM numerical aperture and also reducing complexity by avoiding the use of cumbersome fiber bundles. Our tests also demonstrate high accuracy and linearity in retrieving the optical properties and suitable contrast and depth sensitivity for detecting localized inhomogeneities. In addition to a strong improvement in both instrumentation cost and size with respect to legacy solutions, the setup performances are comparable to those of state-of-the-art time-domain instrumentation, thus opening a new way to compact, low-cost and high-performance time-resolved devices for diffuse optical imaging and spectroscopy.

Multi-contrast attenuation map synthesis for PET/MR scanners: assessment on FDG and Florbetapir PET tracers.

Positron Emission Tomography/Magnetic Resonance Imaging (PET/MR) scanners are expected to offer a new range of clinical applications. Attenuation correction is an essential requirement for quantification of PET data but MRI images do not directly provide a patient-specific attenuation map. Methods We further validate and extend a Computed Tomography (CT) and attenuation map (µ-map) synthesis method based on pre-acquired MRI-CT image pairs. The validation consists of comparing the CT images synthesised with the proposed method to the original CT images. PET images were acquired using two different tracers ((18)F-FDG and (18)F-florbetapir). They were then reconstructed and corrected for attenuation using the synthetic µ-maps and compared to the reference PET images corrected with the CT-based µ-maps. During the validation, we observed that the CT synthesis was inaccurate in areas such as the neck and the cerebellum, and propose a refinement to mitigate these problems, as well as an extension of the method to multi-contrast MRI data. Results With the improvements proposed, a significant enhancement in CT synthesis, which results in a reduced absolute error and a decrease in the bias when reconstructing PET images, was observed. For both tracers, on average, the absolute difference between the reference PET images and the PET images corrected with the proposed method was less than 2%, with a bias inferior to 1%. Conclusion With the proposed method, attenuation information can be accurately derived from MRI images by synthesising CT using routine anatomical sequences. MRI sequences, or combination of sequences, can be used to synthesise CT images, as long as they provide sufficient anatomical information.

A fast boundary element method for the scattering analysis of high-intensity focused ultrasound.

High-intensity focused ultrasound (HIFU) techniques are promising modalities for the non-invasive treatment of cancer. For HIFU therapies of, e.g., liver cancer, one of the main challenges is the accurate focusing of the acoustic field inside a ribcage. Computational methods can play an important role in the patient-specific planning of these transcostal HIFU treatments. This requires the accurate modeling of acoustic scattering at ribcages. The use of a boundary element method (BEM) is an effective approach for this purpose because only the boundaries of the ribs have to be discretized instead of the standard approach to model the entire volume around the ribcage. This paper combines fast algorithms that improve the efficiency of BEM specifically for the high-frequency range necessary for transcostal HIFU applications. That is, a Galerkin discretized Burton-Miller formulation is used in combination with preconditioning and matrix compression techniques. In particular, quick convergence is achieved with the operator preconditioner that has been designed with on-surface radiation conditions for the high-frequency approximation of the Neumann-to-Dirichlet map. Realistic computations of acoustic scattering at 1 MHz on a human ribcage model demonstrate the effectiveness of this dedicated BEM algorithm for HIFU scattering analysis.

A 4D neonatal head model for diffuse optical imaging of pre-term to term infants.

Diffuse optical tomography is most accurate when an individual's MRI data can be used as a spatial prior for image reconstruction and for visualization of the resulting images of changes in oxy- and deoxy-hemoglobin concentration. As this necessitates an MRI scan to be performed for each study, which undermines many of the advantages of diffuse optical methods, the use of registered atlases to model the individual's anatomy is becoming commonplace. Infant studies require carefully age-matched atlases because of the rapid growth and maturation of the infant brain. In this paper, we present a 4D neonatal head model which, for each week from 29 to 44 weeks post-menstrual age, includes: 1) a multi-layered tissue mask which identifies extra-cerebral layers, cerebrospinal fluid, gray matter, white matter, cerebellum and brainstem, 2) a high-density tetrahedral head mesh, 3) surface meshes for the scalp, gray-matter and white matter layers and 4) cranial landmarks and 10-5 locations on the scalp surface. This package, freely available online at www.ucl.ac.uk/medphys/research/4dneonatalmodel can be applied by users of near-infrared spectroscopy and diffuse optical tomography to optimize probe locations, optimize image reconstruction, register data to cortical locations and ultimately improve the accuracy and interpretation of diffuse optical techniques in newborn populations.

Compensation of modeling errors due to unknown domain boundary in diffuse optical tomography.

Diffuse optical tomography is a highly unstable problem with respect to modeling and measurement errors. During clinical measurements, the body shape is not always known, and an approximate model domain has to be employed. The use of an incorrect model domain can, however, lead to significant artifacts in the reconstructed images. Recently, the Bayesian approximation error theory has been proposed to handle model-based errors. In this work, the feasibility of the Bayesian approximation error approach to compensate for modeling errors due to unknown body shape is investigated. The approach is tested with simulations. The results show that the Bayesian approximation error method can be used to reduce artifacts in reconstructed images due to unknown domain shape.

A hybrid neural ordinary differential equation model of the cardiovascular system.

In the human cardiovascular system (CVS), the interaction between the left and right ventricles of the heart is influenced by the septum and the pericardium. Computational models of the CVS can capture this interaction, but this often involves approximating solutions to complex nonlinear equations numerically. As a result, numerous models have been proposed, where these nonlinear equations are either simplified, or ventricular interaction is ignored. In this work, we propose an alternative approach to modelling ventricular interaction, using a hybrid neural ordinary differential equation (ODE) structure. First, a lumped parameter ODE model of the CVS (including a Newton-Raphson procedure as the numerical solver) is simulated to generate synthetic time-series data. Next, a hybrid neural ODE based on the same model is constructed, where ventricular interaction is instead set to be governed by a neural network. We use a short range of the synthetic data (with various amounts of added measurement noise) to train the hybrid neural ODE model. Symbolic regression is used to convert the neural network into analytic expressions, resulting in a partially learned mechanistic model. This approach was able to recover parsimonious functions with good predictive capabilities and was robust to measurement noise.

Training deep learning based dynamic MR image reconstruction using open-source natural videos.

To develop and assess a deep learning (DL) pipeline to learn dynamic MR image reconstruction from publicly available natural videos (Inter4K). Learning was performed for a range of DL architectures (VarNet, 3D UNet, FastDVDNet) and corresponding sampling patterns (Cartesian, radial, spiral) either from true multi-coil cardiac MR data (N = 692) or from synthetic MR data simulated from Inter4K natural videos (N = 588). Real-time undersampled dynamic MR images were reconstructed using DL networks trained with cardiac data and natural videos, and compressed sensing (CS). Differences were assessed in simulations (N = 104 datasets) in terms of MSE, PSNR, and SSIM and prospectively for cardiac cine (short axis, four chambers, N = 20) and speech cine (N = 10) data in terms of subjective image quality ranking, SNR and Edge sharpness. Friedman Chi Square tests with post-hoc Nemenyi analysis were performed to assess statistical significance. In simulated data, DL networks trained with cardiac data outperformed DL networks trained with natural videos, both of which outperformed CS (p < 0.05). However, in prospective experiments DL reconstructions using both training datasets were ranked similarly (and higher than CS) and presented no statistical differences in SNR and Edge Sharpness for most conditions.The developed pipeline enabled learning dynamic MR reconstruction from natural videos preserving DL reconstruction advantages such as high quality fast and ultra-fast reconstructions while overcoming some limitations (data scarcity or sharing). The natural video dataset, code and pre-trained networks are made readily available on github.

Quantitative fluorescence diffuse optical tomography in the presence of heterogeneities.

In fluorescence diffuse optical tomography (fDOT), the accuracy of reconstructed fluorescence distributions highly depends on the knowledge of the tissue optical heterogeneities for correct modeling of light propagation. Common approaches are to assume homogeneous optical properties or, when structural information is available, assign optical properties to various segmented organs, which is likely to result in inaccurate reconstructions. Furthermore, DOT based only on intensity (continuous wave-DOT) is a nonunique inverse problem, and hence, cannot be used to retrieve simultaneously maps of absorption and diffusion coefficients. We propose a method that reconstructs a single parameter from the excitation measurements, which is used in the fDOT problem to accurately recover fluorescence distribution.