Unsupervised knowledge-transfer for learned image reconstruction.

Deep learning-based image reconstruction approaches have demonstrated impressive empirical performance in many imaging modalities. These approaches usually require a large amount of high-quality paired training data, which is often not available in medical imaging. To circumvent this issue we develop a novel unsupervised knowledge-transfer paradigm for learned reconstruction within a Bayesian framework. The proposed approach learns a reconstruction network in two phases. The first phase trains a reconstruction network with a set of ordered pairs comprising of ground truth images of ellipses and the corresponding simulated measurement data. The second phase fine-tunes the pretrained network to more realistic measurement data without supervision. By construction, the framework is capable of delivering predictive uncertainty information over the reconstructed image. We present extensive experimental results on low-dose and sparse-view computed tomography showing that the approach is competitive with several state-of-the-art supervised and unsupervised reconstruction techniques. Moreover, for test data distributed differently from the training data, the proposed framework can significantly improve reconstruction quality not only visually, but also quantitatively in terms of PSNR and SSIM, when compared with learned methods trained on the synthetic dataset only.

(An overview of) Synergistic reconstruction for multimodality/multichannel imaging methods.

Imaging is omnipresent in modern society with imaging devices based on a zoo of physical principles, probing a specimen across different wavelengths, energies and time. Recent years have seen a change in the imaging landscape with more and more imaging devices combining that which previously was used separately. Motivated by these hardware developments, an ever increasing set of mathematical ideas is appearing regarding how data from different imaging modalities or channels can be synergistically combined in the image reconstruction process, exploiting structural and/or functional correlations between the multiple images. Here we review these developments, give pointers to important challenges and provide an outlook as to how the field may develop in the forthcoming years. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 1'.

Slice-illuminated optical projection tomography.

To improve the imaging performance of optical projection tomography (OPT) in live samples, we have explored a parallelized implementation of semi-confocal line illumination and detection to discriminate against scattered photons. Slice-illuminated OPT (sl-OPT) improves reconstruction quality in scattering samples by reducing interpixel crosstalk at the cost of increased acquisition time. For in vivo imaging, this can be ameliorated through the use of compressed sensing on angularly undersampled OPT data sets. Here, we demonstrate sl-OPT applied to 3D imaging of bead phantoms and live adult zebrafish.

Three dimensional photoacoustic tomography in Bayesian framework.

The image reconstruction problem (or inverse problem) in photoacoustic tomography is to resolve the initial pressure distribution from detected ultrasound waves generated within an object due to an illumination by a short light pulse. Recently, a Bayesian approach to photoacoustic image reconstruction with uncertainty quantification was proposed and studied with two dimensional numerical simulations. In this paper, the approach is extended to three spatial dimensions and, in addition to numerical simulations, experimental data are considered. The solution of the inverse problem is obtained by computing point estimates, i.e., maximum a posteriori estimate and posterior covariance. These are computed iteratively in a matrix-free form using a biconjugate gradient stabilized method utilizing the adjoint of the acoustic forward operator. The results show that the Bayesian approach can produce accurate estimates of the initial pressure distribution in realistic measurement geometries and that the reliability of these estimates can be assessed.

Multi-contrast attenuation map synthesis for PET/MR scanners: assessment on FDG and Florbetapir PET tracers.

Positron Emission Tomography/Magnetic Resonance Imaging (PET/MR) scanners are expected to offer a new range of clinical applications. Attenuation correction is an essential requirement for quantification of PET data but MRI images do not directly provide a patient-specific attenuation map. Methods We further validate and extend a Computed Tomography (CT) and attenuation map (µ-map) synthesis method based on pre-acquired MRI-CT image pairs. The validation consists of comparing the CT images synthesised with the proposed method to the original CT images. PET images were acquired using two different tracers ((18)F-FDG and (18)F-florbetapir). They were then reconstructed and corrected for attenuation using the synthetic µ-maps and compared to the reference PET images corrected with the CT-based µ-maps. During the validation, we observed that the CT synthesis was inaccurate in areas such as the neck and the cerebellum, and propose a refinement to mitigate these problems, as well as an extension of the method to multi-contrast MRI data. Results With the improvements proposed, a significant enhancement in CT synthesis, which results in a reduced absolute error and a decrease in the bias when reconstructing PET images, was observed. For both tracers, on average, the absolute difference between the reference PET images and the PET images corrected with the proposed method was less than 2%, with a bias inferior to 1%. Conclusion With the proposed method, attenuation information can be accurately derived from MRI images by synthesising CT using routine anatomical sequences. MRI sequences, or combination of sequences, can be used to synthesise CT images, as long as they provide sufficient anatomical information.

A 4D neonatal head model for diffuse optical imaging of pre-term to term infants.

Diffuse optical tomography is most accurate when an individual's MRI data can be used as a spatial prior for image reconstruction and for visualization of the resulting images of changes in oxy- and deoxy-hemoglobin concentration. As this necessitates an MRI scan to be performed for each study, which undermines many of the advantages of diffuse optical methods, the use of registered atlases to model the individual's anatomy is becoming commonplace. Infant studies require carefully age-matched atlases because of the rapid growth and maturation of the infant brain. In this paper, we present a 4D neonatal head model which, for each week from 29 to 44 weeks post-menstrual age, includes: 1) a multi-layered tissue mask which identifies extra-cerebral layers, cerebrospinal fluid, gray matter, white matter, cerebellum and brainstem, 2) a high-density tetrahedral head mesh, 3) surface meshes for the scalp, gray-matter and white matter layers and 4) cranial landmarks and 10-5 locations on the scalp surface. This package, freely available online at www.ucl.ac.uk/medphys/research/4dneonatalmodel can be applied by users of near-infrared spectroscopy and diffuse optical tomography to optimize probe locations, optimize image reconstruction, register data to cortical locations and ultimately improve the accuracy and interpretation of diffuse optical techniques in newborn populations.

Compensation of modeling errors due to unknown domain boundary in diffuse optical tomography.

Diffuse optical tomography is a highly unstable problem with respect to modeling and measurement errors. During clinical measurements, the body shape is not always known, and an approximate model domain has to be employed. The use of an incorrect model domain can, however, lead to significant artifacts in the reconstructed images. Recently, the Bayesian approximation error theory has been proposed to handle model-based errors. In this work, the feasibility of the Bayesian approximation error approach to compensate for modeling errors due to unknown body shape is investigated. The approach is tested with simulations. The results show that the Bayesian approximation error method can be used to reduce artifacts in reconstructed images due to unknown domain shape.

On the inverse problem in optical coherence tomography.

We examine the inverse problem of retrieving sample refractive index information in the context of optical coherence tomography. Using two separate approaches, we discuss the limitations of the inverse problem which lead to it being ill-posed, primarily as a consequence of the limited viewing angles available in the reflection geometry. This is first considered from the theoretical point of view of diffraction tomography under a weak scattering approximation. We then investigate the full non-linear inverse problem using a variational approach. This presents another illustration of the non-uniqueness of the solution, and shows that even the non-linear (strongly scattering) scenario suffers a similar fate as the linear problem, with the observable spatial Fourier components of the sample occupying a limited support. Through examples we demonstrate how the solutions to the inverse problem compare when using the variational and diffraction-tomography approaches.

Tomographic imaging with polarized light.

We report three-dimensional tomographic reconstruction of optical parameters for the mesoscopic light scattering regime from experimentally obtained datasets by using polarized light. We present a numerically inexpensive approximation to the radiative transfer equation governing the polarized light transport. This approximation is employed in the reconstruction algorithm, which computes two optical parameters by using parallel and perpendicular polarizations of transmitted light. Datasets were obtained by imaging a scattering phantom embedding highly absorbing inclusions. Reconstruction results are presented and discussed.

A Model-Based Iterative Learning Approach for Diffuse Optical Tomography.

Diffuse optical tomography (DOT) utilises near-infrared light for imaging spatially distributed optical parameters, typically the absorption and scattering coefficients. The image reconstruction problem of DOT is an ill-posed inverse problem, due to the non-linear light propagation in tissues and limited boundary measurements. The ill-posedness means that the image reconstruction is sensitive to measurement and modelling errors. The Bayesian approach for the inverse problem of DOT offers the possibility of incorporating prior information about the unknowns, rendering the problem less ill-posed. It also allows marginalisation of modelling errors utilising the so-called Bayesian approximation error method. A more recent trend in image reconstruction techniques is the use of deep learning, which has shown promising results in various applications from image processing to tomographic reconstructions. In this work, we study the non-linear DOT inverse problem of estimating the (absolute) absorption and scattering coefficients utilising a 'model-based' learning approach, essentially intertwining learned components with the model equations of DOT. The proposed approach was validated with 2D simulations and 3D experimental data. We demonstrated improved absorption and scattering estimates for targets with a mix of smooth and sharp image features, implying that the proposed approach could learn image features that are difficult to model using standard Gaussian priors. Furthermore, it was shown that the approach can be utilised in compensating for modelling errors due to coarse discretisation enabling computationally efficient solutions. Overall, the approach provided improved computation times compared to a standard Gauss-Newton iteration.

Fluorescence lifetime optical tomography in weakly scattering media in the presence of highly scattering inclusions.

We consider the problem of fluorescence lifetime optical tomographic imaging in a weakly scattering medium in the presence of highly scattering inclusions. We suggest an approximation to the radiative transfer equation, which results from the assumption that the transport coefficient of the scattering media differs by an order of magnitude for weakly and highly scattering regions. The image reconstruction algorithm is based on the variational framework and employs angularly selective intensity measurements. We present numerical simulation of light scattering in a weakly scattering medium that embeds highly scattering objects. Our reconstruction algorithm is verified by recovering optical and fluorescent parameters from numerically simulated datasets.

Förster resonance energy transfer imaging in vivo with approximated radiative transfer equation.

We describe a new light transport model, which was applied to three-dimensional lifetime imaging of Förster resonance energy transfer in mice in vivo. The model is an approximation to the radiative transfer equation and combines light diffusion and ray optics. This approximation is well adopted to wide-field time-gated intensity-based data acquisition. Reconstructed image data are presented and compared with results obtained by using the telegraph equation approximation. The new approach provides improved recovery of absorption and scattering parameters while returning similar values for the fluorescence parameters.

Fast image reconstruction in fluorescence optical tomography using data compression.

We present a method for fast reconstruction in fluorescence optical tomography with very large data sets. In recent reports, CCD cameras at multiple positions have been used to collect optical measurements, producing more than 10(7) data samples. This makes storage of the full system Jacobian infeasible, and so data are usually subsampled. The method reported here allows use of the full data set, via image compression methods, and explicit construction of the (small) Jacobian, meaning optimal inversion methods can be applied, and thus leading to very fast reconstruction.

Dual wavelength spread-spectrum time-resolved diffuse optical instrument for the measurement of human brain functional responses.

Near-infrared spectroscopy has proven to be a valuable method to monitor tissue oxygenation and haemodynamics non-invasively and in real-time. Quantification of such parameters requires measurements of the time-of-flight of light through tissue, typically achieved using picosecond pulsed lasers, with their associated cost, complexity, and size. In this work, we present an alternative approach that employs spread-spectrum excitation to enable the development of a small, low-cost, dual-wavelength system using vertical-cavity surface-emitting lasers. Since the optimal wavelengths and drive parameters for optical spectroscopy are not served by commercially available modules as used in our previous single-wavelength demonstration platform, we detail the design of a custom instrument and demonstrate its performance in resolving haemodynamic changes in human subjects during apnoea and cognitive task experiments.

A matrix-free algorithm for multiple wavelength fluorescence tomography.

In the recent years, there has been an increase in applications of non-contact diffusion optical tomography. Especially when the objective is the recovery of fluorescence targets. The non-contact acquisition systems with the use of a CCD-camera produce much denser sampled boundary data sets than fibre-based systems. When model-based reconstruction methods are used, that rely on the inversion of a derivative operator, the large number of measurements poses a challenge since the explicit formulation and storage of the Jacobian matrix could be in general not feasible. This problem is aggravated further in applications, where measurements at multiple wavelengths are used. We present a matrix-free model-based reconstruction method, that addresses the problems of large data sets and reduces the computational cost and memory requirements for the reconstruction. The idea behind the matrix-free method is that information about the Jacobian matrix could be available through matrix times vector products so that the creation and storage of big matrices can be avoided. We tested the method for multiple wavelength fluorescence tomography with simulated and experimental data from phantom experiments, and we found substantial benefits in computational times and memory requirements.

Three-dimensional imaging of Förster resonance energy transfer in heterogeneous turbid media by tomographic fluorescent lifetime imaging.

We report a three-dimensional time-resolved tomographic imaging technique for localizing protein-protein interaction and protein conformational changes in turbid media based on Förster resonant energy-transfer read out using fluorescence lifetime. This application of "tomoFRET" employs an inverse scattering algorithm utilizing the diffusion approximation to the radiative-transfer equation applied to a large tomographic data set of time-gated images. The approach is demonstrated by imaging a highly scattering cylindrical phantom within which are two thin wells containing cytosol preparations of HEK293 cells expressing TN-L15, a cytosolic genetically encoded calcium Förster resonant energy-transfer sensor. A 10 mM calcium chloride solution was added to one of the wells, inducing a protein conformation change upon binding to TN-L15, resulting in Förster resonant energy transfer and a corresponding decrease in the donor fluorescence lifetime. We successfully reconstruct spatially resolved maps of the resulting fluorescence lifetime distribution as well as of the quantum efficiency, absorption, and scattering coefficients.

Differentiation of benign and malignant breast tumors by in-vivo three-dimensional parallel-plate diffuse optical tomography.

We have developed a novel parallel-plate diffuse optical tomography (DOT) system for three-dimensional in vivo imaging of human breast tumor based on large optical data sets. Images of oxy-, deoxy-, and total hemoglobin concentration as well as blood oxygen saturation and tissue scattering were reconstructed. Tumor margins were derived using the optical data with guidance from radiology reports and magnetic resonance imaging. Tumor-to-normal ratios of these endogenous physiological parameters and an optical index were computed for 51 biopsy-proven lesions from 47 subjects. Malignant cancers (N=41) showed statistically significant higher total hemoglobin, oxy-hemoglobin concentration, and scattering compared to normal tissue. Furthermore, malignant lesions exhibited a twofold average increase in optical index. The influence of core biopsy on DOT results was also explored; the difference between the malignant group measured before core biopsy and the group measured more than 1 week after core biopsy was not significant. Benign tumors (N=10) did not exhibit statistical significance in the tumor-to-normal ratios of any parameter. Optical index and tumor-to-normal ratios of total hemoglobin, oxy-hemoglobin concentration, and scattering exhibited high area under the receiver operating characteristic curve values from 0.90 to 0.99, suggesting good discriminatory power. The data demonstrate that benign and malignant lesions can be distinguished by quantitative three-dimensional DOT.

Approximation errors and model reduction in three-dimensional diffuse optical tomography.

Model reduction is often required in diffuse optical tomography (DOT), typically because of limited available computation time or computer memory. In practice, this means that one is bound to use coarse mesh and truncated computation domain in the model for the forward problem. We apply the (Bayesian) approximation error model for the compensation of modeling errors caused by domain truncation and a coarse computation mesh in DOT. The approach is tested with a three-dimensional example using experimental data. The results show that when the approximation error model is employed, it is possible to use mesh densities and computation domains that would be unacceptable with a conventional measurement error model.

Combined reconstruction of fluorescent and optical parameters using time-resolved data.

We present an algorithm for simultaneous reconstruction of optical parameters, quantum yield, and lifetime in turbid media with embedded fluorescent inclusions. This algorithm is designed in the Fourier domain as an iterative solution of a system of differential equations of the Helmholtz type and does not involve full ill-conditioned matrix computations. The approach is based on allowing the unknown optical parameters, quantum yield, and lifetime to depend on the Fourier spectral parameter. The algorithm was applied to a time-gated experimental data set acquired by imaging a highly scattering cylindrical phantom concealing small fluorescent tubes. Relatively accurate reconstruction demonstrates the potential of the method.

Single-pixel camera photoacoustic tomography.

Since it was first demonstrated more than a decade ago, the single-pixel camera concept has been used in numerous applications in which it is necessary or advantageous to reduce the channel count, cost, or data volume. Here, three-dimensional (3-D), compressed-sensing photoacoustic tomography (PAT) is demonstrated experimentally using a single-pixel camera. A large area collimated laser beam is reflected from a planar Fabry­Pérot ultrasound sensor onto a digital micromirror device, which patterns the light using a scrambled Hadamard basis before it is collected into a single photodetector. In this way, inner products of the Hadamard patterns and the distribution of thickness changes of the FP sensor­induced by the photoacoustic waves­are recorded. The initial distribution of acoustic pressure giving rise to those photoacoustic waves is recovered directly from the measured signals using an accelerated proximal gradient-type algorithm to solve a model-based minimization with total variation regularization. Using this approach, it is shown that 3-D PAT of imaging phantoms can be obtained with compression rates as low as 10%. Compressed sensing approaches to photoacoustic imaging, such as this, have the potential to reduce the data acquisition time as well as the volume of data it is necessary to acquire, both of which are becoming increasingly important in the drive for faster imaging systems giving higher resolution images with larger fields of view.