A hybrid neural ordinary differential equation model of the cardiovascular system.

In the human cardiovascular system (CVS), the interaction between the left and right ventricles of the heart is influenced by the septum and the pericardium. Computational models of the CVS can capture this interaction, but this often involves approximating solutions to complex nonlinear equations numerically. As a result, numerous models have been proposed, where these nonlinear equations are either simplified, or ventricular interaction is ignored. In this work, we propose an alternative approach to modelling ventricular interaction, using a hybrid neural ordinary differential equation (ODE) structure. First, a lumped parameter ODE model of the CVS (including a Newton-Raphson procedure as the numerical solver) is simulated to generate synthetic time-series data. Next, a hybrid neural ODE based on the same model is constructed, where ventricular interaction is instead set to be governed by a neural network. We use a short range of the synthetic data (with various amounts of added measurement noise) to train the hybrid neural ODE model. Symbolic regression is used to convert the neural network into analytic expressions, resulting in a partially learned mechanistic model. This approach was able to recover parsimonious functions with good predictive capabilities and was robust to measurement noise.

HyperSLICE: HyperBand optimized spiral for low-latency interactive cardiac examination.

PURPOSE: Interactive cardiac MRI is used for fast scan planning and MR-guided interventions. However, the requirement for real-time acquisition and near-real-time visualization constrains the achievable spatio-temporal resolution. This study aims to improve interactive imaging resolution through optimization of undersampled spiral sampling and leveraging of deep learning for low-latency reconstruction (deep artifact suppression). METHODS: A variable density spiral trajectory was parametrized and optimized via HyperBand to provide the best candidate trajectory for rapid deep artifact suppression. Training data consisted of 692 breath-held CINEs. The developed interactive sequence was tested in simulations and prospectively in 13 subjects (10 for image evaluation, 2 during catheterization, 1 during exercise). In the prospective study, the optimized framework-HyperSLICE- was compared with conventional Cartesian real-time and breath-hold CINE imaging in terms quantitative and qualitative image metrics. Statistical differences were tested using Friedman chi-squared tests with post hoc Nemenyi test (p < 0.05). RESULTS: In simulations the normalized RMS error, peak SNR, structural similarity, and Laplacian energy were all statistically significantly higher using optimized spiral compared to radial and uniform spiral sampling, particularly after scan plan changes (structural similarity: 0.71 vs. 0.45 and 0.43). Prospectively, HyperSLICE enabled a higher spatial and temporal resolution than conventional Cartesian real-time imaging. The pipeline was demonstrated in patients during catheter pull back, showing sufficiently fast reconstruction for interactive imaging. CONCLUSION: HyperSLICE enables high spatial and temporal resolution interactive imaging. Optimizing the spiral sampling enabled better overall image quality and superior handling of image transitions compared with radial and uniform spiral trajectories.

A variable heart rate multi-compartmental coupled model of the cardiovascular and respiratory systems.

Current mathematical models of the cardiovascular system that are based on systems of ordinary differential equations are limited in their ability to mimic important features of measured patient data, such as variable heart rates (HR). Such limitations present a significant obstacle in the use of such models for clinical decision-making, as it is the variations in vital signs such as HR and systolic and diastolic blood pressure that are monitored and recorded in typical critical care bedside monitoring systems. In this paper, novel extensions to well-established multi-compartmental models of the cardiovascular and respiratory systems are proposed that permit the simulation of variable HR. Furthermore, a correction to current models is also proposed to stabilize the respiratory behaviour and enable realistic simulation of vital signs over the longer time scales required for clinical management. The results of the extended model developed here show better agreement with measured bio-signals, and these extensions provide an important first step towards estimating model parameters from patient data, using methods such as neural ordinary differential equations. The approach presented is generalizable to many other similar multi-compartmental models of the cardiovascular and respiratory systems.

A learned Born series for highly-scattering media.

A new method for solving the wave equation is presented, called the learned Born series (LBS), which is derived from a convergent Born series but its components are found through training. The LBS is shown to be significantly more accurate than the convergent Born series for the same number of iterations, in the presence of high contrast scatterers, while maintaining a comparable computational complexity. The LBS is able to generate a reasonable prediction of the global pressure field with a small number of iterations, and the errors decrease with the number of learned iterations.

On the inverse problem in optical coherence tomography.

We examine the inverse problem of retrieving sample refractive index information in the context of optical coherence tomography. Using two separate approaches, we discuss the limitations of the inverse problem which lead to it being ill-posed, primarily as a consequence of the limited viewing angles available in the reflection geometry. This is first considered from the theoretical point of view of diffraction tomography under a weak scattering approximation. We then investigate the full non-linear inverse problem using a variational approach. This presents another illustration of the non-uniqueness of the solution, and shows that even the non-linear (strongly scattering) scenario suffers a similar fate as the linear problem, with the observable spatial Fourier components of the sample occupying a limited support. Through examples we demonstrate how the solutions to the inverse problem compare when using the variational and diffraction-tomography approaches.

Enhancement of instrumented ultrasonic tracking images using deep learning.

PURPOSE: Instrumented ultrasonic tracking provides needle localisation during ultrasound-guided minimally invasive percutaneous procedures. Here, a post-processing framework based on a convolutional neural network (CNN) is proposed to improve the spatial resolution of ultrasonic tracking images. METHODS: The custom ultrasonic tracking system comprised a needle with an integrated fibre-optic ultrasound (US) transmitter and a clinical US probe for receiving those transmissions and for acquiring B-mode US images. For post-processing of tracking images reconstructed from the received fibre-optic US transmissions, a recently-developed framework based on ResNet architecture, trained with a purely synthetic dataset, was employed. A preliminary evaluation of this framework was performed with data acquired from needle insertions in the heart of a fetal sheep in vivo. The axial and lateral spatial resolution of the tracking images were used as performance metrics of the trained network. RESULTS: Application of the CNN yielded improvements in the spatial resolution of the tracking images. In three needle insertions, in which the tip depth ranged from 23.9 to 38.4 mm, the lateral resolution improved from 2.11 to 1.58 mm, and the axial resolution improved from 1.29 to 0.46 mm. CONCLUSION: The results provide strong indications of the potential of CNNs to improve the spatial resolution of ultrasonic tracking images and thereby to increase the accuracy of needle tip localisation. These improvements could have broad applicability and impact across multiple clinical fields, which could lead to improvements in procedural efficiency and reductions in risk of complications.

An Investigation of Stochastic Variance Reduction Algorithms for Relative Difference Penalized 3D PET Image Reconstruction.

Penalised PET image reconstruction algorithms are often accelerated during early iterations with the use of subsets. However, these methods may exhibit limit cycle behaviour at later iterations due to variations between subsets. Desirable converged images can be achieved for a subclass of these algorithms via the implementation of a relaxed step size sequence, but the heuristic selection of parameters will impact the quality of the image sequence and algorithm convergence rates. In this work, we demonstrate the adaption and application of a class of stochastic variance reduction gradient algorithms for PET image reconstruction using the relative difference penalty and numerically compare convergence performance to BSREM. The two investigated algorithms are: SAGA and SVRG. These algorithms require the retention in memory of recently computed subset gradients, which are utilised in subsequent updates. We present several numerical studies based on Monte Carlo simulated data and a patient data set for fully 3D PET acquisitions. The impact of the number of subsets, different preconditioners and step size methods on the convergence of regions of interest values within the reconstructed images is explored. We observe that when using constant preconditioning, SAGA and SVRG demonstrate reduced variations in voxel values between subsequent updates and are less reliant on step size hyper-parameter selection than BSREM reconstructions. Furthermore, SAGA and SVRG can converge significantly faster to the penalised maximum likelihood solution than BSREM, particularly in low count data.

FReSCO: Flow Reconstruction and Segmentation for low-latency Cardiac Output monitoring using deep artifact suppression and segmentation.

PURPOSE: Real-time monitoring of cardiac output (CO) requires low-latency reconstruction and segmentation of real-time phase-contrast MR, which has previously been difficult to perform. Here we propose a deep learning framework for "FReSCO" (Flow Reconstruction and Segmentation for low latency Cardiac Output monitoring). METHODS: Deep artifact suppression and segmentation U-Nets were independently trained. Breath-hold spiral phase-contrast MR data (N = 516) were synthetically undersampled using a variable-density spiral sampling pattern and gridded to create aliased data for training of the artifact suppression U-net. A subset of the data (N = 96) was segmented and used to train the segmentation U-net. Real-time spiral phase-contrast MR was prospectively acquired and then reconstructed and segmented using the trained models (FReSCO) at low latency at the scanner in 10 healthy subjects during rest, exercise, and recovery periods. Cardiac output obtained via FReSCO was compared with a reference rest CO and rest and exercise compressed-sensing CO. RESULTS: The FReSCO framework was demonstrated prospectively at the scanner. Beat-to-beat heartrate, stroke volume, and CO could be visualized with a mean latency of 622 ms. No significant differences were noted when compared with reference at rest (bias = -0.21 ± 0.50 L/min, p = 0.246) or compressed sensing at peak exercise (bias = 0.12 ± 0.48 L/min, p = 0.458). CONCLUSIONS: The FReSCO framework was successfully demonstrated for real-time monitoring of CO during exercise and could provide a convenient tool for assessment of the hemodynamic response to a range of stressors.

Adaptive stochastic Gauss-Newton method with optical Monte Carlo for quantitative photoacoustic tomography.

SIGNIFICANCE: The image reconstruction problem in quantitative photoacoustic tomography (QPAT) is an ill-posed inverse problem. Monte Carlo method for light transport can be utilized in solving this image reconstruction problem. AIM: The aim was to develop an adaptive image reconstruction method where the number of photon packets in Monte Carlo simulation is varied to achieve a sufficient accuracy with reduced computational burden. APPROACH: The image reconstruction problem was formulated as a minimization problem. An adaptive stochastic Gauss-Newton (A-SGN) method combined with Monte Carlo method for light transport was developed. In the algorithm, the number of photon packets used on Gauss-Newton (GN) iteration was varied utilizing a so-called norm test. RESULTS: The approach was evaluated with numerical simulations. With the proposed approach, the number of photon packets needed for solving the inverse problem was significantly smaller than in a conventional approach where the number of photon packets was fixed for each GN iteration. CONCLUSIONS: The A-SGN method with a norm test can be utilized in QPAT to provide accurate and computationally efficient solutions.

Neural Network Kalman Filtering for 3-D Object Tracking From Linear Array Ultrasound Data.

Many interventional surgical procedures rely on medical imaging to visualize and track instruments. Such imaging methods not only need to be real time capable but also provide accurate and robust positional information. In ultrasound (US) applications, typically, only 2-D data from a linear array are available, and as such, obtaining accurate positional estimation in three dimensions is nontrivial. In this work, we first train a neural network, using realistic synthetic training data, to estimate the out-of-plane offset of an object with the associated axial aberration in the reconstructed US image. The obtained estimate is then combined with a Kalman filtering approach that utilizes positioning estimates obtained in previous time frames to improve localization robustness and reduce the impact of measurement noise. The accuracy of the proposed method is evaluated using simulations, and its practical applicability is demonstrated on experimental data obtained using a novel optical US imaging setup. Accurate and robust positional information is provided in real time. Axial and lateral coordinates for out-of-plane objects are estimated with a mean error of 0.1 mm for simulated data and a mean error of 0.2 mm for experimental data. The 3-D localization is most accurate for elevational distances larger than 1 mm, with a maximum distance of 6 mm considered for a 25-mm aperture.

Evaluation of a pipeline for simulation, reconstruction, and classification in ultrasound-aided diffuse optical tomography of breast tumors.

SIGNIFICANCE: Diffuse optical tomography is an ill-posed problem. Combination with ultrasound can improve the results of diffuse optical tomography applied to the diagnosis of breast cancer and allow for classification of lesions. AIM: To provide a simulation pipeline for the assessment of reconstruction and classification methods for diffuse optical tomography with concurrent ultrasound information. APPROACH: A set of breast digital phantoms with benign and malignant lesions was simulated building on the software VICTRE. Acoustic and optical properties were assigned to the phantoms for the generation of B-mode images and optical data. A reconstruction algorithm based on a two-region nonlinear fitting and incorporating the ultrasound information was tested. Machine learning classification methods were applied to the reconstructed values to discriminate lesions into benign and malignant after reconstruction. RESULTS: The approach allowed us to generate realistic US and optical data and to test a two-region reconstruction method for a large number of realistic simulations. When information is extracted from ultrasound images, at least 75% of lesions are correctly classified. With ideal two-region separation, the accuracy is higher than 80%. CONCLUSIONS: A pipeline for the generation of realistic ultrasound and diffuse optics data was implemented. Machine learning methods applied to a optical reconstruction with a nonlinear optical model and morphological information permit to discriminate malignant lesions from benign ones.

Unsupervised knowledge-transfer for learned image reconstruction.

Deep learning-based image reconstruction approaches have demonstrated impressive empirical performance in many imaging modalities. These approaches usually require a large amount of high-quality paired training data, which is often not available in medical imaging. To circumvent this issue we develop a novel unsupervised knowledge-transfer paradigm for learned reconstruction within a Bayesian framework. The proposed approach learns a reconstruction network in two phases. The first phase trains a reconstruction network with a set of ordered pairs comprising of ground truth images of ellipses and the corresponding simulated measurement data. The second phase fine-tunes the pretrained network to more realistic measurement data without supervision. By construction, the framework is capable of delivering predictive uncertainty information over the reconstructed image. We present extensive experimental results on low-dose and sparse-view computed tomography showing that the approach is competitive with several state-of-the-art supervised and unsupervised reconstruction techniques. Moreover, for test data distributed differently from the training data, the proposed framework can significantly improve reconstruction quality not only visually, but also quantitatively in terms of PSNR and SSIM, when compared with learned methods trained on the synthetic dataset only.

A Model-Based Iterative Learning Approach for Diffuse Optical Tomography.

Diffuse optical tomography (DOT) utilises near-infrared light for imaging spatially distributed optical parameters, typically the absorption and scattering coefficients. The image reconstruction problem of DOT is an ill-posed inverse problem, due to the non-linear light propagation in tissues and limited boundary measurements. The ill-posedness means that the image reconstruction is sensitive to measurement and modelling errors. The Bayesian approach for the inverse problem of DOT offers the possibility of incorporating prior information about the unknowns, rendering the problem less ill-posed. It also allows marginalisation of modelling errors utilising the so-called Bayesian approximation error method. A more recent trend in image reconstruction techniques is the use of deep learning, which has shown promising results in various applications from image processing to tomographic reconstructions. In this work, we study the non-linear DOT inverse problem of estimating the (absolute) absorption and scattering coefficients utilising a 'model-based' learning approach, essentially intertwining learned components with the model equations of DOT. The proposed approach was validated with 2D simulations and 3D experimental data. We demonstrated improved absorption and scattering estimates for targets with a mix of smooth and sharp image features, implying that the proposed approach could learn image features that are difficult to model using standard Gaussian priors. Furthermore, it was shown that the approach can be utilised in compensating for modelling errors due to coarse discretisation enabling computationally efficient solutions. Overall, the approach provided improved computation times compared to a standard Gauss-Newton iteration.

Deep Learning for Instrumented Ultrasonic Tracking: From Synthetic Training Data to In Vivo Application.

Instrumented ultrasonic tracking is used to improve needle localization during ultrasound guidance of minimally invasive percutaneous procedures. Here, it is implemented with transmitted ultrasound pulses from a clinical ultrasound imaging probe, which is detected by a fiber-optic hydrophone integrated into a needle. The detected transmissions are then reconstructed to form the tracking image. Two challenges are considered with the current implementation of ultrasonic tracking. First, tracking transmissions are interleaved with the acquisition of B-mode images, and thus, the effective B-mode frame rate is reduced. Second, it is challenging to achieve an accurate localization of the needle tip when the signal-to-noise ratio is low. To address these challenges, we present a framework based on a convolutional neural network (CNN) to maintain spatial resolution with fewer tracking transmissions and enhance signal quality. A major component of the framework included the generation of realistic synthetic training data. The trained network was applied to unseen synthetic data and experimental in vivo tracking data. The performance of needle localization was investigated when reconstruction was performed with fewer (up to eightfold) tracking transmissions. CNN-based processing of conventional reconstructions showed that the axial and lateral spatial resolutions could be improved even with an eightfold reduction in tracking transmissions. The framework presented in this study will significantly improve the performance of ultrasonic tracking, leading to faster image acquisition rates and increased localization accuracy.

Deep artifact suppression for spiral real-time phase contrast cardiac magnetic resonance imaging in congenital heart disease.

PURPOSE: Real-time spiral phase contrast MR (PCMR) enables rapid free-breathing assessment of flow. Target spatial and temporal resolutions require high acceleration rates often leading to long reconstruction times. Here we propose a deep artifact suppression framework for fast and accurate flow quantification. METHODS: U-Nets were trained for deep artifact suppression using 520 breath-hold gated spiral PCMR aortic datasets collected in congenital heart disease patients. Two spiral trajectories (uniform and perturbed) and two losses (Mean Absolute Error -MAE- and average structural similarity index measurement -SSIM-) were compared in synthetic data in terms of MAE, peak SNR (PSNR) and SSIM. Perturbed spiral PCMR was prospectively acquired in 20 patients. Stroke Volume (SV), peak mean velocity and edge sharpness measurements were compared to Compressed Sensing (CS) and Cartesian reference. RESULTS: In synthetic data, perturbed spiral consistently outperformed uniform spiral for the different image metrics. U-Net MAE showed better MAE and PSNR while U-Net SSIM showed higher SSIM based metrics. In-vivo, there were no significant differences in SV between any of the real-time reconstructions and the reference standard Cartesian data. However, U-Net SSIM had better image sharpness and lower biases for peak velocity when compared to U-Net MAE. Reconstruction of 96 frames took ~59 s for CS and 3.9 s for U-Nets. CONCLUSION: Deep artifact suppression of complex valued images using an SSIM based loss was successfully demonstrated in a cohort of congenital heart disease patients for fast and accurate flow quantification.

(An overview of) Synergistic reconstruction for multimodality/multichannel imaging methods.

Imaging is omnipresent in modern society with imaging devices based on a zoo of physical principles, probing a specimen across different wavelengths, energies and time. Recent years have seen a change in the imaging landscape with more and more imaging devices combining that which previously was used separately. Motivated by these hardware developments, an ever increasing set of mathematical ideas is appearing regarding how data from different imaging modalities or channels can be synergistically combined in the image reconstruction process, exploiting structural and/or functional correlations between the multiple images. Here we review these developments, give pointers to important challenges and provide an outlook as to how the field may develop in the forthcoming years. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 1'.

Real-time deep artifact suppression using recurrent U-Nets for low-latency cardiac MRI.

PURPOSE: Real-time low latency MRI is performed to guide various cardiac interventions. Real-time acquisitions often require iterative image reconstruction strategies, which lead to long reconstruction times. In this study, we aim to reconstruct highly undersampled radial real-time data with low latency using deep learning. METHODS: A 2D U-Net with convolutional long short-term memory layers is proposed to exploit spatial and preceding temporal information to reconstruct highly accelerated tiny golden radial data with low latency. The network was trained using a dataset of breath-hold CINE data (including 770 time series from 7 different orientations). Synthetic paired data were created by retrospectively undersampling the magnitude images, and the network was trained to recover the target images. In the spirit of interventional imaging, the network was trained and tested for varying acceleration rates and orientations. Data were prospectively acquired and reconstructed in real time in 1 healthy subject interactively and in 3 patients who underwent catheterization. Images were visually compared to sliding window and compressed sensing reconstructions and a conventional Cartesian real-time sequence. RESULTS: The proposed network generalized well to different acceleration rates and unseen orientations for all considered metrics in simulated data (less than 4% reduction in structural similarity index compared to similar acceleration and orientation-specific networks). The proposed reconstruction was demonstrated interactively, successfully depicting catheters in vivo with low latency (39 ms, including 19 ms for deep artifact suppression) and an image quality comparing favorably to other reconstructions. CONCLUSION: Deep artifact suppression was successfully demonstrated in the time-critical application of non-Cartesian real-time interventional cardiac MR.

Dual wavelength spread-spectrum time-resolved diffuse optical instrument for the measurement of human brain functional responses.

Near-infrared spectroscopy has proven to be a valuable method to monitor tissue oxygenation and haemodynamics non-invasively and in real-time. Quantification of such parameters requires measurements of the time-of-flight of light through tissue, typically achieved using picosecond pulsed lasers, with their associated cost, complexity, and size. In this work, we present an alternative approach that employs spread-spectrum excitation to enable the development of a small, low-cost, dual-wavelength system using vertical-cavity surface-emitting lasers. Since the optimal wavelengths and drive parameters for optical spectroscopy are not served by commercially available modules as used in our previous single-wavelength demonstration platform, we detail the design of a custom instrument and demonstrate its performance in resolving haemodynamic changes in human subjects during apnoea and cognitive task experiments.

Characterization of B0-field fluctuations in prostate MRI.

Multi-parametric MRI is increasingly used for prostate cancer detection. Improving information from current sequences, such as T2-weighted and diffusion-weighted (DW) imaging, and additional sequences, such as magnetic resonance spectroscopy (MRS) and chemical exchange saturation transfer (CEST), may enhance the performance of multi-parametric MRI. The majority of these techniques are sensitive to B0-field variations and may result in image distortions including signal pile-up and stretching (echo planar imaging (EPI) based DW-MRI) or unwanted shifts in the frequency spectrum (CEST and MRS). Our aim is to temporally and spatially characterize B0-field changes in the prostate. Ten male patients are imaged using dual-echo gradient echo sequences with varying repetitions on a 3 T scanner to evaluate the temporal B0-field changes within the prostate. A phantom is also imaged to consider no physiological motion. The spatial B0-field variations in the prostate are reported as B0-field values (Hz), their spatial gradients (Hz/mm) and the resultant distortions in EPI based DW-MRI images (b-value = 0 s/mm2 and two oppositely phase encoded directions). Over a period of minutes, temporal changes in B0-field values were ≤19 Hz for minimal bowel motion and ≥30 Hz for large motion. Spatially across the prostate, the B0-field values had an interquartile range of ≤18 Hz (minimal motion) and ≤44 Hz (large motion). The B0-field gradients were between -2 and 5 Hz/mm (minimal motion) and 2 and 12 Hz/mm (large motion). Overall, B0-field variations can affect DW, MRS and CEST imaging of the prostate.

Efficient inversion strategies for estimating optical properties with Monte Carlo radiative transport models.

SIGNIFICANCE: Indirect imaging problems in biomedical optics generally require repeated evaluation of forward models of radiative transport, for which Monte Carlo is accurate yet computationally costly. We develop an approach to reduce this bottleneck, which has significant implications for quantitative tomographic imaging in a variety of medical and industrial applications. AIM: Our aim is to enable computationally efficient image reconstruction in (hybrid) diffuse optical modalities using stochastic forward models. APPROACH: Using Monte Carlo, we compute a fully stochastic gradient of an objective function for a given imaging problem. Leveraging techniques from the machine learning community, we then adaptively control the accuracy of this gradient throughout the iterative inversion scheme to substantially reduce computational resources at each step. RESULTS: For example problems of quantitative photoacoustic tomography and ultrasound-modulated optical tomography, we demonstrate that solutions are attainable using a total computational expense that is comparable to (or less than) that which is required for a single high-accuracy forward run of the same Monte Carlo model. CONCLUSIONS: This approach demonstrates significant computational savings when approaching the full nonlinear inverse problem of optical property estimation using stochastic methods.