Score-based generative model-assisted information compensation for high-quality limited-view reconstruction in photoacoustic tomography.

Photoacoustic tomography (PAT) regularly operates in limited-view cases owing to data acquisition limitations. The results using traditional methods in limited-view PAT exhibit distortions and numerous artifacts. Here, a novel limited-view PAT reconstruction strategy that combines model-based iteration with score-based generative model was proposed. By incrementally adding noise to the training samples, prior knowledge can be learned from the complex probability distribution. The acquired prior is then utilized as constraint in model-based iteration. The information of missing views can be gradually compensated by cyclic iteration to achieve high-quality reconstruction. The performance of the proposed method was evaluated with the circular phantom and in vivo experimental data. Experimental results demonstrate the outstanding effectiveness of the proposed method in limited-view cases. Notably, the proposed method exhibits excellent performance in limited-view case of 70° compared with traditional method. It achieves a remarkable improvement of 203% in PSNR and 48% in SSIM for the circular phantom experimental data, and an enhancement of 81% in PSNR and 65% in SSIM for in vivo experimental data, respectively. The proposed method has capability of reconstructing PAT images in extremely limited-view cases, which will further expand the application in clinical scenarios.

Dual-domain Collaborative Diffusion Sampling for Multi-Source Stationary Computed Tomography Reconstruction.

The multi-source stationary CT, where both the detector and X-ray source are fixed, represents a novel imaging system with high temporal resolution that has garnered significant interest. Limited space within the system restricts the number of X-ray sources, leading to sparse-view CT imaging challenges. Recent diffusion models for reconstructing sparse-view CT have generally focused separately on sinogram or image domains. Sinogram-centric models effectively estimate missing projections but may introduce artifacts, lacking mechanisms to ensure image correctness. Conversely, image-domain models, while capturing detailed image features, often struggle with complex data distribution, leading to inaccuracies in projections. Addressing these issues, the Dual-domain Collaborative Diffusion Sampling (DCDS) model integrates sinogram and image domain diffusion processes for enhanced sparse-view reconstruction. This model combines the strengths of both domains in an optimized mathematical framework. A collaborative diffusion mechanism underpins this model, improving sinogram recovery and image generative capabilities. This mechanism facilitates feedback-driven image generation from the sinogram domain and uses image domain results to complete missing projections. Optimization of the DCDS model is further achieved through the alternative direction iteration method, focusing on data consistency updates. Extensive testing, including numerical simulations, real phantoms, and clinical cardiac datasets, demonstrates the DCDS model's effectiveness. It consistently outperforms various state-of-the-art benchmarks, delivering exceptional reconstruction quality and precise sinogram.

Dual-domain mean-reverting diffusion model-enhanced temporal compressive coherent diffraction imaging.

Temporal compressive coherent diffraction imaging is a lensless imaging technique with the capability to capture fast-moving small objects. However, the accuracy of imaging reconstruction is often hindered by the loss of frequency domain information, a critical factor limiting the quality of the reconstructed images. To improve the quality of these reconstructed images, a method dual-domain mean-reverting diffusion model-enhanced temporal compressive coherent diffraction imaging (DMDTC) has been introduced. DMDTC leverages the mean-reverting diffusion model to acquire prior information in both frequency and spatial domain through sample learning. The frequency domain mean-reverting diffusion model is employed to recover missing information, while hybrid input-output algorithm is carried out to reconstruct the spatial domain image. The spatial domain mean-reverting diffusion model is utilized for denoising and image restoration. DMDTC has demonstrated a significant enhancement in the quality of the reconstructed images. The results indicate that the structural similarity and peak signal-to-noise ratio of images reconstructed by DMDTC surpass those obtained through conventional methods. DMDTC enables high temporal frame rates and high spatial resolution in coherent diffraction imaging.

Unsupervised disentanglement strategy for mitigating artifact in photoacoustic tomography under extremely sparse view.

Traditional methods under sparse view for reconstruction of photoacoustic tomography (PAT) often result in significant artifacts. Here, a novel image to image transformation method based on unsupervised learning artifact disentanglement network (ADN), named PAT-ADN, was proposed to address the issue. This network is equipped with specialized encoders and decoders that are responsible for encoding and decoding the artifacts and content components of unpaired images, respectively. The performance of the proposed PAT-ADN was evaluated using circular phantom data and the animal in vivo experimental data. The results demonstrate that PAT-ADN exhibits excellent performance in effectively removing artifacts. In particular, under extremely sparse view (e.g., 16 projections), structural similarity index and peak signal-to-noise ratio are improved by ∼188 % and ∼85 % in in vivo experimental data using the proposed method compared to traditional reconstruction methods. PAT-ADN improves the imaging performance of PAT, opening up possibilities for its application in multiple domains.

Knowledge-driven deep learning for fast MR imaging: Undersampled MR image reconstruction from supervised to un-supervised learning.

Deep learning (DL) has emerged as a leading approach in accelerating MRI. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MRI involves physics-based imaging processes, unique data properties, and diverse imaging tasks. This domain knowledge needs to be integrated with data-driven approaches. Our review will introduce the significant challenges faced by such knowledge-driven DL approaches in the context of fast MRI along with several notable solutions, which include learning neural networks and addressing different imaging application scenarios. The traits and trends of these techniques have also been given which have shifted from supervised learning to semi-supervised learning, and finally, to unsupervised learning methods. In addition, MR vendors' choices of DL reconstruction have been provided along with some discussions on open questions and future directions, which are critical for the reliable imaging systems.

Evaluation of low-dose computed tomography reconstruction using spatial-radon domain total generalized variation regularization.

The x-ray radiation dose in computed tomography (CT) examination has been a major concern for patients. Lowing the tube current and exposure time in data acquisition is a straightforward and cost-effective strategy to reduce the x-ray radiation dose. However, this will inevitably increase the noise fluctuations in measured projection data, and the corresponding CT image quality will be severely degraded if noise suppression is not performed during image reconstruction. To reconstruct high-quality low-dose CT image, we present a spatial-radon domain total generalized variation (SRDTGV) regularization for statistical iterative reconstruction (SIR) based on penalized weighted least-squares (PWLS) principle, which is called PWLS-SRDTGV for simplicity. The presented PWLS-SRDTGV model can simultaneously reconstruct high-quality CT image in space domain and its corresponding projection in radon domain. An efficient split Bregman algorithm was applied to minimize the cost function of the proposed reconstruction model. Qualitative and quantitative studies were performed to evaluate the effectiveness of the PWLS-SRDTGV image reconstruction algorithm using a digital 3D XCAT phantom and an anthropomorphic torso phantom. The experimental results demonstrate that PWLS-SRDTGV algorithm achieves notable gains in noise reduction, streak artifact suppression, and edge preservation compared with competing reconstruction approaches.

Correlated and Multi-frequency Diffusion Modeling for Highly Under-sampled MRI Reconstruction.

Given the obstacle in accentuating the reconstruction accuracy for diagnostically significant tissues, most existing MRI reconstruction methods perform targeted reconstruction of the entire MR image without considering fine details, especially when dealing with highly under-sampled images. Therefore, a considerable volume of efforts has been directed towards surmounting this challenge, as evidenced by the emergence of numerous methods dedicated to preserving high-frequency content as well as fine textural details in the reconstructed image. In this case, exploring the merits associated with each method of mining high-frequency information and formulating a reasonable principle to maximize the joint utilization of these approaches will be a more effective solution to achieve accurate reconstruction. Specifically, this work constructs an innovative principle named Correlated and Multi-frequency Diffusion Model (CM-DM) for highly under-sampled MRI reconstruction. In essence, the rationale underlying the establishment of such principle lies not in assembling arbitrary models, but in pursuing the effective combinations and replacement of components. It also means that the novel principle focuses on forming a correlated and multi-frequency prior through different high-frequency operators in the diffusion process. Moreover, multi-frequency prior further constraints the noise term closer to the target distribution in the frequency domain, thereby making the diffusion process converge faster. Experimental results verify that the proposed method achieved superior reconstruction accuracy, with a notable enhancement of approximately 2dB in PSNR compared to state-of-the-art methods.

High-resolution iterative reconstruction at extremely low sampling rate for Fourier single-pixel imaging via diffusion model.

The trade-off between imaging efficiency and imaging quality has always been encountered by Fourier single-pixel imaging (FSPI). To achieve high-resolution imaging, the increase in the number of measurements is necessitated, resulting in a reduction of imaging efficiency. Here, a novel high-quality reconstruction method for FSPI imaging via diffusion model was proposed. A score-based diffusion model is designed to learn prior information of the data distribution. The real-sampled low-frequency Fourier spectrum of the target is employed as a consistency term to iteratively constrain the model in conjunction with the learned prior information, achieving high-resolution reconstruction at extremely low sampling rates. The performance of the proposed method is evaluated by simulations and experiments. The results show that the proposed method has achieved superior quality compared with the traditional FSPI method and the U-Net method. Especially at the extremely low sampling rate (e.g., 1%), an approximately 241% improvement in edge intensity-based score was achieved by the proposed method for the coin experiment, compared with the traditional FSPI method. The method has the potential to achieve high-resolution imaging without compromising imaging speed, which will further expanding the application scope of FSPI in practical scenarios.

Invertible and Variable Augmented Network for Pretreatment Patient-Specific Quality Assurance Dose Prediction.

Pretreatment patient-specific quality assurance (prePSQA) is conducted to confirm the accuracy of the radiotherapy dose delivered. However, the process of prePSQA measurement is time consuming and exacerbates the workload for medical physicists. The purpose of this work is to propose a novel deep learning (DL) network to improve the accuracy and efficiency of prePSQA. A modified invertible and variable augmented network was developed to predict the three-dimensional (3D) measurement-guided dose (MDose) distribution of 300 cancer patients who underwent volumetric modulated arc therapy (VMAT) between 2018 and 2021, in which 240 cases were randomly selected for training, and 60 for testing. For simplicity, the present approach was termed as "IVPSQA." The input data include CT images, radiotherapy dose exported from the treatment planning system, and MDose distribution extracted from the verification system. Adam algorithm was used for first-order gradient-based optimization of stochastic objective functions. The IVPSQA model obtained high-quality 3D prePSQA dose distribution maps in head and neck, chest, and abdomen cases, and outperformed the existing U-Net-based prediction approaches in terms of dose difference maps and horizontal profiles comparison. Moreover, quantitative evaluation metrics including SSIM, MSE, and MAE demonstrated that the proposed approach achieved a good agreement with ground truth and yield promising gains over other advanced methods. This study presented the first work on predicting 3D prePSQA dose distribution by using the IVPSQA model. The proposed method could be taken as a clinical guidance tool and help medical physicists to reduce the measurement work of prePSQA.

Variable augmentation network for invertible MR coil compression.

To improve the efficiency of multi-coil data compression and recover the compressed image reversibly, increasing the possibility of applying the proposed method to medical scenarios. A deep learning algorithm is employed for MR coil compression in the presented work. The approach introduces a variable augmentation network for invertible coil compression (VAN-ICC). This network utilizes the inherent reversibility of normalizing flow-based models. The aim is to enhance the readability of the sentence and clearly convey the key components of the algorithm. By applying the variable augmentation technology to image/k-space variables from multi-coils, VAN-ICC trains the invertible network by finding an invertible and bijective function, which can map the original data to the compressed counterpart and vice versa. Experiments conducted on both fully-sampled and under-sampled data verified the effectiveness and flexibility of VAN-ICC. Quantitative and qualitative comparisons with traditional non-deep learning-based approaches demonstrated that VAN-ICC carries much higher compression effects. The proposed method trains the invertible network by finding an invertible and bijective function, which improves the defects of traditional coil compression method by utilizing inherent reversibility of normalizing flow-based models. In addition, the application of variable augmentation technology ensures the implementation of reversible networks. In short, VAN-ICC offered a competitive advantage over other traditional coil compression algorithms.

Stage-by-stage Wavelet Optimization Refinement Diffusion Model for Sparse-View CT Reconstruction.

Diffusion model has emerged as a potential tool to tackle the challenge of sparse-view CT reconstruction, displaying superior performance compared to conventional methods. Nevertheless, these prevailing diffusion models predominantly focus on the sinogram or image domains, which can lead to instability during model training, potentially culminating in convergence towards local minimal solutions. The wavelet transform serves to disentangle image contents and features into distinct frequency-component bands at varying scales, adeptly capturing diverse directional structures. Employing the wavelet transform as a guiding sparsity prior significantly enhances the robustness of diffusion models. In this study, we present an innovative approach named the Stage-by-stage Wavelet Optimization Refinement Diffusion (SWORD) model for sparse-view CT reconstruction. Specifically, we establish a unified mathematical model integrating low-frequency and high-frequency generative models, achieving the solution with an optimization procedure. Furthermore, we perform the low-frequency and high-frequency generative models on wavelet's decomposed components rather than the original sinogram, ensuring the stability of model training. Our method is rooted in established optimization theory, comprising three distinct stages, including low-frequency generation, high-frequency refinement and domain transform. The experimental results demonstrated that the proposed method outperformed existing state-of-the-art methods both quantitatively and qualitatively.

Accelerated model-based iterative reconstruction strategy for sparse-view photoacoustic tomography aided by multi-channel autoencoder priors.

Photoacoustic tomography (PAT) commonly works in sparse view due to data acquisition limitations. However, reconstruction suffers from serious deterioration (e.g., severe artifacts) using traditional algorithms under sparse view. Here, a novel accelerated model-based iterative reconstruction strategy for sparse-view PAT aided by multi-channel autoencoder priors was proposed. A multi-channel denoising autoencoder network was designed to learn prior information, which provides constraints for model-based iterative reconstruction. This integration accelerates the iteration process, leading to optimal reconstruction outcomes. The performance of the proposed method was evaluated using blood vessel simulation data and experimental data. The results show that the proposed method can achieve superior sparse-view reconstruction with a significant acceleration of iteration. Notably, the proposed method exhibits excellent performance under extremely sparse condition (e.g., 32 projections) compared with the U-Net method, with an improvement of 48% in PSNR and 12% in SSIM for in vivo experimental data.

Wavelet-Improved Score-Based Generative Model for Medical Imaging.

The score-based generative model (SGM) has demonstrated remarkable performance in addressing challenging under-determined inverse problems in medical imaging. However, acquiring high-quality training datasets for these models remains a formidable task, especially in medical image reconstructions. Prevalent noise perturbations or artifacts in low-dose Computed Tomography (CT) or under-sampled Magnetic Resonance Imaging (MRI) hinder the accurate estimation of data distribution gradients, thereby compromising the overall performance of SGMs when trained with these data. To alleviate this issue, we propose a wavelet-improved denoising technique to cooperate with the SGMs, ensuring effective and stable training. Specifically, the proposed method integrates a wavelet sub-network and the standard SGM sub-network into a unified framework, effectively alleviating inaccurate distribution of the data distribution gradient and enhancing the overall stability. The mutual feedback mechanism between the wavelet sub-network and the SGM sub-network empowers the neural network to learn accurate scores even when handling noisy samples. This combination results in a framework that exhibits superior stability during the learning process, leading to the generation of more precise and reliable reconstructed images. During the reconstruction process, we further enhance the robustness and quality of the reconstructed images by incorporating regularization constraint. Our experiments, which encompass various scenarios of low-dose and sparse-view CT, as well as MRI with varying under-sampling rates and masks, demonstrate the effectiveness of the proposed method by significantly enhanced the quality of the reconstructed images. Especially, our method with noisy training samples achieves comparable results to those obtained using clean data. Our code at https://zenodo.org/record/8266123.

Sparse-view reconstruction for photoacoustic tomography combining diffusion model with model-based iteration.

As a non-invasive hybrid biomedical imaging technology, photoacoustic tomography combines high contrast of optical imaging and high penetration of acoustic imaging. However, the conventional standard reconstruction under sparse view could result in low-quality image in photoacoustic tomography. Here, a novel model-based sparse reconstruction method for photoacoustic tomography via diffusion model was proposed. A score-based diffusion model is designed for learning the prior information of the data distribution. The learned prior information is utilized as a constraint for the data consistency term of an optimization problem based on the least-square method in the model-based iterative reconstruction, aiming to achieve the optimal solution. Blood vessels simulation data and the animal in vivo experimental data were used to evaluate the performance of the proposed method. The results demonstrate that the proposed method achieves higher-quality sparse reconstruction compared with conventional reconstruction methods and U-Net. In particular, under the extreme sparse projection (e.g., 32 projections), the proposed method achieves an improvement of ∼ 260 % in structural similarity and ∼ 30 % in peak signal-to-noise ratio for in vivo data, compared with the conventional delay-and-sum method. This method has the potential to reduce the acquisition time and cost of photoacoustic tomography, which will further expand the application range.

TransQA: deep hybrid transformer network for measurement-guided volumetric dose prediction of pre-treatment patient-specific quality assurance.

Objective. Performing pre-treatment patient-specific quality assurance (prePSQA) is considered an essential, time-consuming, and resource-intensive task for volumetric modulated arc radiotherapy (VMAT) which confirms the dose accuracy and ensure patient safety. Most current machine learning and deep learning approaches stack excessive convolutional/pooling operations (CPs) to predict prePSQA with two-dimensional or one-dimensional information input. However, these models generally present limitations in explicitly modeling long-range dependency for volumetric dose prediction due to the loss of spatial dose features and the inherent locality of CPs. The purpose of this work is to construct a deep hybrid network by combining the self-attention mechanism-based Transformer with modified U-Net for predicting measurement-guided volumetric dose (MDose) of prePSQA.Approach. The enrolled 307 cancer patients underwent VMAT were randomly divided into 246 and 61 cases for training and testing the model. The input data included computed tomography images, radiotherapy dose images exported from the treatment planning system, as well as the MDose distribution from the verification system. The output was the predicted high-quality voxel-wise prePSQA dose distribution.Main results: qualitative and quantitative experimental results show that the proposed prediction method could achieve comparable or better performance on MDose prediction over other approaches in terms of spatial dose distribution, dose-volume histogram metrics, gamma passing rates, mean absolute error, root mean square error, and structural similarity.Significance. The preliminary results on multiple cancer sites show that our approach can be taken as a clinical guidance tool and help medical physicists to reduce the measurement work of prePSQA.

Universal generative modeling in dual domains for dynamic MRI.

Dynamic magnetic resonance image reconstruction from incomplete k-space data has generated great research interest due to its ability to reduce scan time. Nevertheless, the reconstruction problem remains a thorny issue due to its ill posed nature. Recently, diffusion models, especially score-based generative models, have demonstrated great potential in terms of algorithmic robustness and flexibility of utilization. Moreover, a unified framework through the variance exploding stochastic differential equation is proposed to enable new sampling methods and further extend the capabilities of score-based generative models. Therefore, by taking advantage of the unified framework, we propose a k-space and image dual-domain collaborative universal generative model (DD-UGM), which combines the score-based prior with a low-rank regularization penalty to reconstruct highly under-sampled measurements. More precisely, we extract prior components from both image and k-space domains via a universal generative model and adaptively handle these prior components for faster processing while maintaining good generation quality. Experimental comparisons demonstrate the noise reduction and detail preservation abilities of the proposed method. Moreover, DD-UGM can reconstruct data of different frames by only training a single frame image, which reflects the flexibility of the proposed model.

WKGM: weighted k-space generative model for parallel imaging reconstruction.

Deep learning based parallel imaging (PI) has made great progress in recent years to accelerate MRI. Nevertheless, it still has some limitations: for example, the robustness and flexibility of existing methods are greatly deficient. In this work, we propose a method to explore the k-space domain learning via robust generative modeling for flexible calibrationless PI reconstruction, coined the weighted k-space generative model (WKGM). Specifically, WKGM is a generalized k-space domain model, where the k-space weighting technology and high-dimensional space augmentation design are efficiently incorporated for score-based generative model training, resulting in good and robust reconstructions. In addition, WKGM is flexible and thus can be synergistically combined with various traditional k-space PI models, which can make full use of the correlation between multi-coil data and realize calibrationless PI. Even though our model was trained on only 500 images, experimental results with varying sampling patterns and acceleration factors demonstrate that WKGM can attain state-of-the-art reconstruction results with the well learned k-space generative prior.

Noise-insensitive defocused signal and resolution enhancement for optical-resolution photoacoustic microscopy via deep learning.

Optical-resolution photoacoustic microscopy suffers from narrow depth of field and a significant deterioration in defocused signal intensity and spatial resolution. Here, a method based on deep learning was proposed to enhance the defocused resolution and signal-to-noise ratio. A virtual optical-resolution photoacoustic microscopy based on k-wave was used to obtain the datasets of deep learning with different noise levels. A fully dense U-Net was trained with randomly distributed sources to improve the quality of photoacoustic images. The results show that the PSNR of defocused signal was enhanced by more than 1.2 times. An over 2.6-fold enhancement in lateral resolution and an over 3.4-fold enhancement in axial resolution of defocused regions were achieved. The large volumetric and high-resolution imaging of blood vessels further verified that the proposed method can effectively overcome the deterioration of the signal and the spatial resolution due to the narrow depth of field of optical-resolution photoacoustic microscopy.

Low-rank tensor assisted K-space generative model for parallel imaging reconstruction.

Although recent deep learning methods, especially generative models, have shown good performance in magnetic resonance imaging, there is still much room for improvement. Considering that the sample number and internal dimension in score-based generative model have key influence on estimating the gradients of data distribution, we present a new idea for parallel imaging reconstruction, named low-rank tensor assisted k-space generative model (LR-KGM). It means that we transform low-rank information into high-dimensional prior information for learning. More specifically, the multi-channel data is constructed into a large Hankel matrix to reduce the number of training samples, which is subsequently collapsed into a tensor for the stage of prior learning. In the testing phase, the low-rank rotation strategy is utilized to impose low-rank constraints on the output tensors of the generative network. Furthermore, we alternate the reconstruction between traditional generative iterations and low-rank high-dimensional tensor iterations. Experimental comparisons with the state-of-the-arts demonstrated that the proposed LR-KGM method achieved better performance.

High-speed real 3D scene acquisition and 3D holographic reconstruction system based on ultrafast optical axial scanning.

The lack of three-dimensional (3D) content is one of the challenges that have been faced by holographic 3D display. Here, we proposed a real 3D scene acquisition and 3D holographic reconstruction system based on ultrafast optical axial scanning. An electrically tunable lens (ETL) was used for high-speed focus shift (up to 2.5 ms). A CCD camera was synchronized with the ETL to acquire multi-focused image sequence of real scene. Then, the focusing area of each multi-focused image was extracted by using Tenengrad operator, and the 3D image were obtained. Finally, 3D holographic reconstruction visible to the naked eye can be achieved by the layer-based diffraction algorithm. The feasibility and effectiveness of the proposed method have been demonstrated by simulation and experiment, and the experimental results agree well with the simulation results. This method will further expand the application of holographic 3D display in the field of education, advertising, entertainment, and other fields.