A sparsification and reconstruction strategy for compressed sensing photoacoustic tomography.

Compressed sensing (CS) is a promising approach to reduce the number of measurements in photoacoustic tomography (PAT) while preserving high spatial resolution. This allows to increase the measurement speed and reduce system costs. Instead of collecting point-wise measurements, in CS one uses various combinations of pressure values at different sensor locations. Sparsity is the main condition allowing to recover the photoacoustic (PA) source from compressive measurements. In this paper, a different concept enabling sparse recovery in CS PAT is introduced. This approach is based on the fact that the second time derivative applied to the measured pressure data corresponds to the application of the Laplacian to the original PA source. As typical PA sources consist of smooth parts and singularities along interfaces, the Laplacian of the source is sparse (or at least compressible). To efficiently exploit the induced sparsity, a reconstruction framework is developed to jointly recover the initial and modified sparse sources. Reconstruction results with simulated as well as experimental data are given.

Stochastic Proximal Gradient Algorithms for Multi-Source Quantitative Photoacoustic Tomography.

The development of accurate and efficient image reconstruction algorithms is a central aspect of quantitative photoacoustic tomography (QPAT). In this paper, we address this issues for multi-source QPAT using the radiative transfer equation (RTE) as accurate model for light transport. The tissue parameters are jointly reconstructed from the acoustical data measured for each of the applied sources. We develop stochastic proximal gradient methods for multi-source QPAT, which are more efficient than standard proximal gradient methods in which a single iterative update has complexity proportional to the number applies sources. Additionally, we introduce a completely new formulation of QPAT as multilinear (MULL) inverse problem which avoids explicitly solving the RTE. The MULL formulation of QPAT is again addressed with stochastic proximal gradient methods. Numerical results for both approaches are presented. Besides the introduction of stochastic proximal gradient algorithms to QPAT, we consider the new MULL formulation of QPAT as main contribution of this paper.

Aggregated motion estimation for real-time MRI reconstruction.

PURPOSE: In real-time MRI serial images are generally reconstructed from highly undersampled datasets as the iterative solutions of an inverse problem. While practical realizations based on regularized nonlinear inversion (NLINV) have hitherto been surprisingly successful, strong assumptions about the continuity of image features may affect the temporal fidelity of the estimated reconstructions. THEORY AND METHODS: The proposed method for real-time image reconstruction integrates the deformations between nearby frames into the data consistency term of the inverse problem. The aggregated motion estimation (AME) is not required to be affine or rigid and does not need additional measurements. Moreover, it handles multi-channel MRI data by simultaneously determining the image and its coil sensitivity profiles in a nonlinear formulation which also adapts to non-Cartesian (e.g., radial) sampling schemes. The new method was evaluated for real-time MRI studies using highly undersampled radial gradient-echo sequences. RESULTS: AME reconstructions for a motion phantom with controlled speed as well as for measurements of human heart and tongue movements demonstrate improved temporal fidelity and reduced residual undersampling artifacts when compared with NLINV reconstructions without motion estimation. CONCLUSION: Nonlinear inverse reconstructions with aggregated motion estimation offer improved image quality and temporal acuity for visualizing rapid dynamic processes by real-time MRI.

Plug-and-Play Image Reconstruction Is a Convergent Regularization Method.

Non-uniqueness and instability are characteristic features of image reconstruction methods. As a result, it is necessary to develop regularization methods that can be used to compute reliable approximate solutions. A regularization method provides a family of stable reconstructions that converge to a specific solution of the noise-free problem as the noise level tends to zero. The standard regularization technique is defined by a variational image reconstruction that minimizes a data discrepancy augmented by a regularizer. The actual numerical implementation makes use of iterative methods, often involving proximal mappings of the regularizer. In recent years, Plug-and-Play (PnP) image reconstruction has been developed as a new powerful generalization of variational methods based on replacing proximal mappings by more general image denoisers. While PnP iterations yield excellent results, neither stability nor convergence in the sense of regularization have been studied so far. In this work, we extend the idea of PnP by considering families of PnP iterations, each accompanied by its own denoiser. As our main theoretical result, we show that such PnP reconstructions lead to stable and convergent regularization methods. This shows for the first time that PnP is as mathematically justified for robust image reconstruction as variational methods.

Derivative-Free Iterative One-Step Reconstruction for Multispectral CT.

Image reconstruction in multispectral computed tomography (MSCT) requires solving a challenging nonlinear inverse problem, commonly tackled via iterative optimization algorithms. Existing methods necessitate computing the derivative of the forward map and potentially its regularized inverse. In this work, we present a simple yet highly effective algorithm for MSCT image reconstruction, utilizing iterative update mechanisms that leverage the full forward model in the forward step and a derivative-free adjoint problem. Our approach demonstrates both fast convergence and superior performance compared to existing algorithms, making it an interesting candidate for future work. We also discuss further generalizations of our method and its combination with additional regularization and other data discrepancy terms.

Design, implementation, and analysis of a compressed sensing photoacoustic projection imaging system.

Significance: Compressed sensing (CS) uses special measurement designs combined with powerful mathematical algorithms to reduce the amount of data to be collected while maintaining image quality. This is relevant to almost any imaging modality, and in this paper we focus on CS in photoacoustic projection imaging (PAPI) with integrating line detectors (ILDs). Aim: Our previous research involved rather general CS measurements, where each ILD can contribute to any measurement. In the real world, however, the design of CS measurements is subject to practical constraints. In this research, we aim at a CS-PAPI system where each measurement involves only a subset of ILDs, and which can be implemented in a cost-effective manner. Approach: We extend the existing PAPI with a self-developed CS unit. The system provides structured CS matrices for which the existing recovery theory cannot be applied directly. A random search strategy is applied to select the CS measurement matrix within this class for which we obtain exact sparse recovery. Results: We implement a CS PAPI system for a compression factor of 4:3, where specific measurements are made on separate groups of 16 ILDs. We algorithmically design optimal CS measurements that have proven sparse CS capabilities. Numerical experiments are used to support our results. Conclusions: CS with proven sparse recovery capabilities can be integrated into PAPI, and numerical results support this setup. Future work will focus on applying it to experimental data and utilizing data-driven approaches to enhance the compression factor and generalize the signal class.

Analysis for Full-Field Photoacoustic Tomography with Variable Sound Speed.

Photoacoustic tomography (PAT) is a non-invasive imaging modality that requires recovering the initial data of the wave equation from certain measurements of the solution outside the object. In the standard PAT measurement setup, the used data consist of time-dependent signals measured on an observation surface. In contrast, the measured data from the recently invented full-field detection technique provide the solution of the wave equation on a spatial domain at a single instant in time. While reconstruction using classical PAT data has been extensively studied, not much is known for the full field PAT problem. In this paper, we build mathematical foundations of the latter problem for variable sound speed and settle its uniqueness and stability. Moreover, we introduce an exact inversion method using time-reversal and study its convergence. Our results demonstrate the suitability of both the full field approach and the proposed time-reversal technique for high resolution photoacoustic imaging.

A deep learning pipeline for the automated segmentation of posterior limb of internal capsule in preterm neonates.

Segmentation of specific brain tissue from MRI volumes is of great significance for brain disease diagnosis, progression assessment, and monitoring of neurological conditions. Manual segmentation is time-consuming, laborious, and subjective, which significantly amplifies the need for automated processes. Over the last decades, the active development in the field of deep learning, especially convolutional neural networks (CNNs), and the associated performance improvements have increased the demand for the application of CNN-based methods to provide consistent measurements and quantitative analyses. In this paper, we present an efficient deep learning approach for the segmentation of brain tissue. More specifically, we address the problem of segmentation of the posterior limb of the internal capsule (PLIC) in preterm neonates. To this end, we propose a CNN-based pipeline comprised of slice-selection modules and a multi-view segmentation model, which exploits the 3D information contained in the MRI volumes to improve segmentation performance. One special feature of the proposed method is its ability to identify one desired slice out of the whole image volume, which is relevant for pediatricians in terms of prognosis. To increase computational efficiency, we apply a strategy that automatically reduces the information contained in the MRI volumes to its relevant parts. Finally, we conduct an expert rating alongside standard evaluation metrics, such as dice score, to evaluate the performance of the proposed framework. We demonstrate the benefit of the multi-view technique by comparing it with its single-view counterparts, which reveals that the proposed method strikes a good balance between exploiting the available image information and reducing the required computing power compared to 3D segmentation networks. Standard evaluation metrics as, well as expert-based assessment, confirm the good performance of the proposed framework, with the latter being more relevant in terms of clinical applicability. We demonstrate that the proposed deep learning pipeline can compete with the experts in terms of accuracy. To prove the generalisability of the proposed method, we additionally assess our deep learning pipeline to data from the Developing Human Connectome Project (dHCP).

Discretization of Learned NETT Regularization for Solving Inverse Problems.

Deep learning based reconstruction methods deliver outstanding results for solving inverse problems and are therefore becoming increasingly important. A recently invented class of learning-based reconstruction methods is the so-called NETT (for Network Tikhonov Regularization), which contains a trained neural network as regularizer in generalized Tikhonov regularization. The existing analysis of NETT considers fixed operators and fixed regularizers and analyzes the convergence as the noise level in the data approaches zero. In this paper, we extend the frameworks and analysis considerably to reflect various practical aspects and take into account discretization of the data space, the solution space, the forward operator and the neural network defining the regularizer. We show the asymptotic convergence of the discretized NETT approach for decreasing noise levels and discretization errors. Additionally, we derive convergence rates and present numerical results for a limited data problem in photoacoustic tomography.

An end-to-end-trainable iterative network architecture for accelerated radial multi-coil 2D cine MR image reconstruction.

PURPOSE: Iterative convolutional neural networks (CNNs) which resemble unrolled learned iterative schemes have shown to consistently deliver state-of-the-art results for image reconstruction problems across different imaging modalities. However, because these methods include the forward model in the architecture, their applicability is often restricted to either relatively small reconstruction problems or to problems with operators which are computationally cheap to compute. As a consequence, they have not been applied to dynamic non-Cartesian multi-coil reconstruction problems so far. METHODS: In this work, we propose a CNN architecture for image reconstruction of accelerated 2D radial cine MRI with multiple receiver coils. The network is based on a computationally light CNN component and a subsequent conjugate gradient (CG) method which can be jointly trained end-to-end using an efficient training strategy. We investigate the proposed training strategy and compare our method with other well-known reconstruction techniques with learned and non-learned regularization methods. RESULTS: Our proposed method outperforms all other methods based on non-learned regularization. Further, it performs similar or better than a CNN-based method employing a 3D U-Net and a method using adaptive dictionary learning. In addition, we empirically demonstrate that even by training the network with only iteration, it is possible to increase the length of the network at test time and further improve the results. CONCLUSIONS: End-to-end training allows to highly reduce the number of trainable parameters of and stabilize the reconstruction network. Further, because it is possible to change the length of the network at the test time, the need to find a compromise between the complexity of the CNN-block and the number of iterations in each CG-block becomes irrelevant.

Full field detection in photoacoustic tomography.

Imaging the full acoustic field around an object by use of an optical phase contrast method is used to accelerate the data acquisition in photoacoustic tomography. Images obtained with a CCD-camera at a certain time show a projection of the instantaneous pressure field in a given direction. In this work a reconstruction method is presented to obtain the two-dimensional initial pressure distribution by back propagating the observed wave pattern in Radon space. Numerical simulations are used to prove the accuracy of the reconstruction algorithm and to demonstrate a method for correcting limited data artifacts. Finally, the overall performance is shown with experimentally obtained data.

Big in Japan: Regularizing Networks for Solving Inverse Problems.

Deep learning and (deep) neural networks are emerging tools to address inverse problems and image reconstruction tasks. Despite outstanding performance, the mathematical analysis for solving inverse problems by neural networks is mostly missing. In this paper, we introduce and rigorously analyze families of deep regularizing neural networks (RegNets) of the form B α + N θ ( α ) B α , where B α is a classical regularization and the network N θ ( α ) B α is trained to recover the missing part Id X - B α not found by the classical regularization. We show that these regularizing networks yield a convergent regularization method for solving inverse problems. Additionally, we derive convergence rates (quantitative error estimates) assuming a sufficient decay of the associated distance function. We demonstrate that our results recover existing convergence and convergence rates results for filter-based regularization methods as well as the recently introduced null space network as special cases. Numerical results are presented for a tomographic sparse data problem, which clearly demonstrate that the proposed RegNets improve classical regularization as well as the null space network.

Phospholipid Acyl Chain Diversity Controls the Tissue-Specific Assembly of Mitochondrial Cardiolipins.

Cardiolipin (CL) is a phospholipid specific for mitochondrial membranes and crucial for many core tasks of this organelle. Its acyl chain configurations are tissue specific, functionally important, and generated via post-biosynthetic remodeling. However, this process lacks the necessary specificity to explain CL diversity, which is especially evident for highly specific CL compositions in mammalian tissues. To investigate the so far elusive regulatory origin of CL homeostasis in mice, we combine lipidomics, integrative transcriptomics, and data-driven machine learning. We demonstrate that not transcriptional regulation, but cellular phospholipid compositions are closely linked to the tissue specificity of CL patterns allowing artificial neural networks to precisely predict cross-tissue CL compositions in a consistent mechanistic specificity rationale. This is especially relevant for the interpretation of disease-related perturbations of CL homeostasis, by allowing differentiation between specific aberrations in CL metabolism and changes caused by global alterations in cellular (phospho-)lipid metabolism.

Deep learning for photoacoustic tomography from sparse data.

The development of fast and accurate image reconstruction algorithms is a central aspect of computed tomography. In this paper, we investigate this issue for the sparse data problem in photoacoustic tomography (PAT). We develop a direct and highly efficient reconstruction algorithm based on deep learning. In our approach, image reconstruction is performed with a deep convolutional neural network (CNN), whose weights are adjusted prior to the actual image reconstruction based on a set of training data. The proposed reconstruction approach can be interpreted as a network that uses the PAT filtered backprojection algorithm for the first layer, followed by the U-net architecture for the remaining layers. Actual image reconstruction with deep learning consists in one evaluation of the trained CNN, which does not require time-consuming solution of the forward and adjoint problems. At the same time, our numerical results demonstrate that the proposed deep learning approach reconstructs images with a quality comparable to state of the art iterative approaches for PAT from sparse data.

Exact and approximative imaging methods for photoacoustic tomography using an arbitrary detection surface.

Two universal reconstruction methods for photoacoustic (also called optoacoustic or thermoacoustic) computed tomography are derived, applicable to an arbitrarily shaped detection surface. In photoacoustic tomography acoustic pressure waves are induced by illuminating a semitransparent sample with pulsed electromagnetic radiation and are measured on a detection surface outside the sample. The imaging problem consists in reconstructing the initial pressure sources from those measurements. The first solution to this problem is based on the time reversal of the acoustic pressure field with a second order embedded boundary method. The pressure on the arbitrarily shaped detection surface is set to coincide with the measured data in reversed temporal order. In the second approach the reconstruction problem is solved by calculating the far-field approximation, a concept well known in physics, where the generated acoustic wave is approximated by an outgoing spherical wave with the reconstruction point as center. Numerical simulations are used to compare the proposed universal reconstruction methods with existing algorithms.

Sampling Conditions for the Circular Radon Transform.

Recovering a function from circular or spherical mean values is the basis of many modern imaging technologies, such as photo and thermoacoustic computed tomography or ultrasound reflection tomography. Recently, much progress has been made concerning the problem of recovering a function from its circular mean values (its circular Radon transform). In particular, theoretically exact inversion formulas of the back-projection type have been discovered using continuously sampled data. In practical applications, however, only a discrete number of circular mean values can be collected. In this paper, we address this issue in the context of the Shannon sampling theory. We derive sharp sampling conditions for the number of angular and radial samples, such that any essentially b0-bandlimited function can be recovered from a finite number of such circular mean values.

Thermoacoustic tomography with integrating area and line detectors.

Thermoacoustic (optoacoustic, photoacoustic) tomography is based on the generation of acoustic waves by illumination of a sample with a short electromagnetic pulse. The absorption density inside the sample is reconstructed from the acoustic pressure measured outside the illuminated sample. So far measurement data have been collected with small detectors as approximations of point detectors. Here, a novel measurement setup applying integrating detectors (e.g., lines or planes made of piezoelectric films) is presented. That way, the pressure is integrated along one or two dimensions, enabling the use of numerically efficient algorithms, such as algorithms for the inverse radon transformation, for thermoacoustic tomography. To reconstruct a three-dimensional sample, either an area detector has to be moved tangential around a sphere that encloses the sample or an array of line detectors is rotated around a single axis. The line detectors can be focused on cross sections perpendicular to the rotation axis using a synthetic aperture (SAFT) or by scanning with a cylindrical lens detector. Measurements were made with piezoelectric polyvinylidene fluoride film detectors and evaluated by comparison with numerical simulations. The resolution achieved in the resulting tomography images is demonstrated on the example of the reconstructed cross section of a grape.

Mapping molecules in scanning far-field fluorescence nanoscopy.

In fluorescence microscopy, the distribution of the emitting molecule number in space is usually obtained by dividing the measured fluorescence by that of a single emitter. However, the brightness of individual emitters may vary strongly in the sample or be inaccessible. Moreover, with increasing (super-) resolution, fewer molecules are found per pixel, making this approach unreliable. Here we map the distribution of molecules by exploiting the fact that a single molecule emits only a single photon at a time. Thus, by analysing the simultaneous arrival of multiple photons during confocal imaging, we can establish the number and local brightness of typically up to 20 molecules per confocal (diffraction sized) recording volume. Subsequent recording by stimulated emission depletion microscopy provides the distribution of the number of molecules with subdiffraction resolution. The method is applied to mapping the three-dimensional nanoscale organization of internalized transferrin receptors on human HEK293 cells.

Deblurring algorithms accounting for the finite detector size in photoacoustic tomography.

Most reconstruction algorithms for photoacoustic tomography, like back projection or time reversal, work ideally for point-like detectors. For real detectors, which integrate the pressure over their finite size, images reconstructed by these algorithms show some blurring. Iterative reconstruction algorithms using an imaging matrix can take the finite size of real detectors directly into account, but the numerical effort is significantly higher compared to the use of direct algorithms. For spherical or cylindrical detection surfaces, the blurring caused by a finite detector size is proportional to the distance from the rotation center (spin blur) and is equal to the detector size at the detection surface. In this work, we apply deconvolution algorithms to reduce this type of blurring on simulated and on experimental data. Two particular deconvolution methods are compared, which both utilize the fact that a representation of the blurred image in polar coordinates decouples pixels at different radii from the rotation center. Experimental data have been obtained with a flat, rectangular piezoelectric detector measuring signals around a plastisol cylinder containing various small photoacoustic sources with variable distance from the center. Both simulated and experimental results demonstrate a nearly complete elimination of spin blur.