Quantitative susceptibility mapping through model-based deep image prior (MoDIP).

The data-driven approach of supervised learning methods has limited applicability in solving dipole inversion in Quantitative Susceptibility Mapping (QSM) with varying scan parameters across different objects. To address this generalization issue in supervised QSM methods, we propose a novel training-free model-based unsupervised method called MoDIP (Model-based Deep Image Prior). MoDIP comprises a small, untrained network and a Data Fidelity Optimization (DFO) module. The network converges to an interim state, acting as an implicit prior for image regularization, while the optimization process enforces the physical model of QSM dipole inversion. Experimental results demonstrate MoDIP's excellent generalizability in solving QSM dipole inversion across different scan parameters. It exhibits robustness against pathological brain QSM, achieving over 32 % accuracy improvement than supervised deep learning methods. It is also 33 % more computationally efficient and runs 4 times faster than conventional DIP-based approaches, enabling 3D high-resolution image reconstruction in under 4.5 min.

Unsupervised motion artifact correction of turbo spin-echo MRI using deep image prior.

PURPOSE: In MRI, motion artifacts can significantly degrade image quality. Motion artifact correction methods using deep neural networks usually required extensive training on large datasets, making them time-consuming and resource-intensive. In this paper, an unsupervised deep learning-based motion artifact correction method for turbo-spin echo MRI is proposed using the deep image prior framework. THEORY AND METHODS: The proposed approach takes advantage of the high impedance to motion artifacts offered by the neural network parameterization to remove motion artifacts in MR images. The framework consists of parameterization of MR image, automatic spatial transformation, and motion simulation model. The proposed method synthesizes motion-corrupted images from the motion-corrected images generated by the convolutional neural network, where an optimization process minimizes the objective function between the synthesized images and the acquired images. RESULTS: In the simulation study of 280 slices from 14 subjects, the proposed method showed a significant increase in the averaged structural similarity index measure by 0.2737 in individual coil images and by 0.4550 in the root-sum-of-square images. In addition, the ablation study demonstrated the effectiveness of each proposed component in correcting motion artifacts compared to the corrected images produced by the baseline method. The experiments on real motion dataset has shown its clinical potential. CONCLUSION: The proposed method exhibited significant quantitative and qualitative improvements in correcting rigid and in-plane motion artifacts in MR images acquired using turbo spin-echo sequence.

Enhancing quality and speed in database-free neural network reconstructions of undersampled MRI with SCAMPI.

PURPOSE: We present SCAMPI (Sparsity Constrained Application of deep Magnetic resonance Priors for Image reconstruction), an untrained deep Neural Network for MRI reconstruction without previous training on datasets. It expands the Deep Image Prior approach with a multidomain, sparsity-enforcing loss function to achieve higher image quality at a faster convergence speed than previously reported methods. METHODS: Two-dimensional MRI data from the FastMRI dataset with Cartesian undersampling in phase-encoding direction were reconstructed for different acceleration rates for single coil and multicoil data. RESULTS: The performance of our architecture was compared to state-of-the-art Compressed Sensing methods and ConvDecoder, another untrained Neural Network for two-dimensional MRI reconstruction. SCAMPI outperforms these by better reducing undersampling artifacts and yielding lower error metrics in multicoil imaging. In comparison to ConvDecoder, the U-Net architecture combined with an elaborated loss-function allows for much faster convergence at higher image quality. SCAMPI can reconstruct multicoil data without explicit knowledge of coil sensitivity profiles. Moreover, it is a novel tool for reconstructing undersampled single coil k-space data. CONCLUSION: Our approach avoids overfitting to dataset features, that can occur in Neural Networks trained on databases, because the network parameters are tuned only on the reconstruction data. It allows better results and faster reconstruction than the baseline untrained Neural Network approach.

Deep image prior cine MR fingerprinting with B1 + spin history correction.

PURPOSE: To develop a deep image prior (DIP) reconstruction for B1 + -corrected 2D cine MR fingerprinting (MRF). METHODS: The proposed method combines low-rank (LR) modeling with a DIP to generate cardiac phase-resolved parameter maps without motion correction, employing self-supervised training to enforce consistency with undersampled spiral k-space data. Two implementations were tested: one approach (DIP) for cine T1 , T2 , and M0 mapping, and a second approach (DIP with effective B1 + estimation [DIP-B1]) that also generated an effective B1 + map to correct for errors due to RF transmit inhomogeneities, through-plane motion, and blood flow. Cine MRF data were acquired in 14 healthy subjects and four reconstructions were compared: LR, low-rank motion-corrected (LRMC), DIP, and DIP-B1. Results were compared to diastolic ECG-triggered MRF, MOLLI, and T2 -prep bSSFP. Additionally, bright-blood and dark-blood images calculated from cine MRF maps were used to quantify ventricular function and compared to reference cine measurements. RESULTS: DIP and DIP-B1 outperformed other cine MRF reconstructions with improved noise suppression and delineation of high-resolution details. Within-segment variability in the myocardium (reported as the coefficient of variation for T1 /T2 ) was lowest for DIP-B1 (2.3/8.3%) followed by DIP (2.7/8.7%), LRMC (3.5/10.5%), and LR (15.3/39.6%). Spatial homogeneity improved with DIP-B1 having the lowest intersegment variability (2.6/4.1%). The mean bias in ejection fraction was -1.1% compared to reference cine scans. CONCLUSION: A DIP reconstruction for 2D cine MRF enabled cardiac phase-resolved mapping of T1 , T2 , M0 , and the effective B1 + with improved noise suppression and precision compared to LR and LRMC.

High-resolution extracellular pH imaging of liver cancer with multiparametric MR using Deep Image Prior.

Noninvasive extracellular pH (pHe) mapping with Biosensor Imaging of Redundant Deviation in Shifts (BIRDS) using MR spectroscopic imaging (MRSI) has been demonstrated on 3T clinical MR scanners at 8 × 8 × 10 mm3 spatial resolution and applied to study various liver cancer treatments. Although pHe imaging at higher resolution can be achieved by extending the acquisition time, a postprocessing method to increase the resolution is preferable, to minimize the duration spent by the subject in the MR scanner. In this work, we propose to improve the spatial resolution of pHe mapping with BIRDS by incorporating anatomical information in the form of multiparametric MRI and using an unsupervised deep-learning technique, Deep Image Prior (DIP). Specifically, we used high-resolution T 1 , T 2 , and diffusion-weighted imaging (DWI) MR images of rabbits with VX2 liver tumors as inputs to a U-Net architecture to provide anatomical information. U-Net parameters were optimized to minimize the difference between the output super-resolution image and the experimentally acquired low-resolution pHe image using the mean-absolute error. In this way, the super-resolution pHe image would be consistent with both anatomical MR images and the low-resolution pHe measurement from the scanner. The method was developed based on data from 49 rabbits implanted with VX2 liver tumors. For evaluation, we also acquired high-resolution pHe images from two rabbits, which were used as ground truth. The results indicate a good match between the spatial characteristics of the super-resolution images and the high-resolution ground truth, supported by the low pixelwise absolute error.

Denoising and uncertainty estimation in parameter mapping with approximate Bayesian deep image priors.

PURPOSE: To mitigate the problem of noisy parameter maps with high uncertainties by casting parameter mapping as a denoising task based on Deep Image Priors. METHODS: We extend the concept of denoising with Deep Image Prior (DIP) into parameter mapping by treating the output of an image-generating network as a parametrization of tissue parameter maps. The method implicitly denoises the parameter mapping process by filtering low-level image features with an untrained convolutional neural network (CNN). Our implementation includes uncertainty estimation from Bernoulli approximate variational inference, implemented with MC dropout, which provides model uncertainty in each voxel of the denoised parameter maps. The method is modular, so the specifics of different applications (e.g., T1 mapping) separate into application-specific signal equation blocks. We evaluate the method on variable flip angle T1 mapping, multi-echo T2 mapping, and apparent diffusion coefficient mapping. RESULTS: We found that deep image prior adapts successfully to several applications in parameter mapping. In all evaluations, the method produces noise-reduced parameter maps with decreased uncertainty compared to conventional methods. The downsides of the proposed method are the long computational time and the introduction of some bias from the denoising prior. CONCLUSION: DIP successfully denoise the parameter mapping process and applies to several applications with limited hyperparameter tuning. Further, it is easy to implement since DIP methods do not use network training data. Although time-consuming, uncertainty information from MC dropout makes the method more robust and provides useful information when properly calibrated.

Personalized synthetic MR imaging with deep learning enhancements.

PURPOSE: Personalized synthetic MRI (syn-MRI) uses MR images of an individual subject acquired at a few design parameters (echo time, repetition time, flip angle) to obtain underlying parametric ( ρ , T 1 , T 2 ) $$ \left(\rho, {\mathrm{T}}_1,{\mathrm{T}}_2\right) $$ maps, from where MR images of that individual at other design parameter settings are synthesized. However, classical methods that use least-squares (LS) or maximum likelihood estimators (MLE) are unsatisfactory at higher noise levels because the underlying inverse problem is ill-posed. This article provides a pipeline to enhance the synthesis of such images in three-dimensional (3D) using a deep learning (DL) neural network architecture for spatial regularization in a personalized setting where having more than a few training images is impractical. METHODS: Our DL enhancements employ a Deep Image Prior (DIP) with a U-net type denoising architecture that includes situations with minimal training data, such as personalized syn-MRI. We provide a general workflow for syn-MRI from three or more training images. Our workflow, called DIPsyn-MRI, uses DIP to enhance training images, then obtains parametric images using LS or MLE before synthesizing images at desired design parameter settings. DIPsyn-MRI is implemented in our publicly available Python package DeepSynMRI available at: https://github.com/StatPal/DeepSynMRI. RESULTS: We demonstrate feasibility and improved performance of DIPsyn-MRI on 3D datasets acquired using the Brainweb interface for spin-echo and FLASH imaging sequences, at different noise levels. Our DL enhancements improve syn-MRI in the presence of different intensity nonuniformity levels of the magnetic field, for all but very low noise levels. CONCLUSION: This article provides recipes and software to realistically facilitate DL-enhanced personalized syn-MRI.

A Physics-Inspired Deep Learning Framework for an Efficient Fourier Ptychographic Microscopy Reconstruction under Low Overlap Conditions.

Two-dimensional observation of biological samples at hundreds of nanometers resolution or even below is of high interest for many sensitive medical applications. Recent advances have been obtained over the last ten years with computational imaging. Among them, Fourier Ptychographic Microscopy is of particular interest because of its important super-resolution factor. In complement to traditional intensity images, phase images are also produced. A large set of N raw images (with typically N = 225) is, however, required because of the reconstruction process that is involved. In this paper, we address the problem of FPM image reconstruction using a few raw images only (here, N = 37) as is highly desirable to increase microscope throughput. In contrast to previous approaches, we develop an algorithmic approach based on a physics-informed optimization deep neural network and statistical reconstruction learning. We demonstrate its efficiency with the help of simulations. The forward microscope image formation model is explicitly introduced in the deep neural network model to optimize its weights starting from an initialization that is based on statistical learning. The simulation results that are presented demonstrate the conceptual benefits of the approach. We show that high-quality images are effectively reconstructed without any appreciable resolution degradation. The learning step is also shown to be mandatory.

A Denoising Method Using Deep Image Prior to Human-Target Detection Using MIMO FMCW Radar.

A Multiple-Input Multiple-Output (MIMO) Frequency-Modulated Continuous Wave (FMCW) radar can provide a range-angle map that expresses the signal power against each range and angle. It is possible to estimate object locations by detecting the signal power that exceeds a threshold using an algorithm, such as Constant False Alarm Rate (CFAR). However, noise and multipath components often exist over the range-angle map, which could produce false alarms for an undesired location depending on the threshold setting. In other words, the threshold setting is sensitive in noisy range-angle maps. Therefore, if the noise is reduced, the threshold can be easily set to reduce the number of false alarms. In this paper, we propose a method that improves the CFAR threshold tolerance by denoising a range-angle map using Deep Image Prior (DIP). DIP is an unsupervised deep-learning technique that enables image denoising. In the proposed method, DIP is applied to the range-angle map calculated by the Curve-Length (CL) method, and then the object location is detected over the denoised range-angle map based on Cell-Averaging CFAR (CA-CFAR), which is a typical threshold setting algorithm. Through the experiments to estimate human locations in indoor environments, we confirmed that the proposed method with DIP reduced the number of false alarms and estimated the human location accurately while improving the tolerance of the threshold setting, compared to the method without DIP.

Low-Light Image Enhancement via Retinex-Style Decomposition of Denoised Deep Image Prior.

Low-light images are a common phenomenon when taking photos in low-light environments with inappropriate camera equipment, leading to shortcomings such as low contrast, color distortion, uneven brightness, and high loss of detail. These shortcomings are not only subjectively annoying but also affect the performance of many computer vision systems. Enhanced low-light images can be better applied to image recognition, object detection and image segmentation. This paper proposes a novel RetinexDIP method to enhance images. Noise is considered as a factor in image decomposition using deep learning generative strategies. The involvement of noise makes the image more real, weakens the coupling relationship between the three components, avoids overfitting, and improves generalization. Extensive experiments demonstrate that our method outperforms existing methods qualitatively and quantitatively.

A Single Image Deep Learning Approach to Restoration of Corrupted Landsat-7 Satellite Images.

Remote sensing is increasingly recognized as a convenient tool with a wide variety of uses in agriculture. Landsat-7 has supplied multi-spectral imagery of the Earth's surface for more than 4 years and has become an important data source for a large number of research and policy-making initiatives. Unfortunately, a scan line corrector (SLC) on Landsat-7 broke down in May 2003, which caused the loss of up to 22 percent of any given scene. We present a single-image approach based on leveraging the abilities of the deep image prior method to fill in gaps using only the corrupt image. We test the ability of deep image prior to reconstruct remote sensing scenes with different levels of corruption in them. Additionally, we compare the performance of our approach with the performance of classical single-image gap-filling methods. We demonstrate a quantitative advantage of the proposed approach compared with classical gap-filling methods. The lowest-performing restoration made by the deep image prior approach reaches 0.812 in r2, while the best value for the classical approaches is 0.685. We also present the robustness of deep image prior in comparing the influence of the number of corrupted pixels on the restoration results. The usage of this approach could expand the possibilities for a wide variety of agricultural studies and applications.

Noise-Resistant Demosaicing with Deep Image Prior Network and Random RGBW Color Filter Array.

In this paper, we propose a deep-image-prior-based demosaicing method for a random RGBW color filter array (CFA). The color reconstruction from the random RGBW CFA is performed by the deep image prior network, which uses only the RGBW CFA image as the training data. To our knowledge, this work is a first attempt to reconstruct the color image with a neural network using only a single RGBW CFA in the training. Due to the White pixels in the RGBW CFA, more light is transmitted through the CFA than in the case with the conventional RGB CFA. As the image sensor can detect more light, the signal-to-noise-ratio (SNR) increases and the proposed demosaicing method can reconstruct the color image with a higher visual quality than other existing demosaicking methods, especially in the presence of noise. We propose a loss function that can train the deep image prior (DIP) network to reconstruct the colors from the White pixels as well as from the red, green, and blue pixels in the RGBW CFA. Apart from using the DIP network, no additional complex reconstruction algorithms are required for the demosaicing. The proposed demosaicing method becomes useful in situations when the noise becomes a major problem, for example, in low light conditions. Experimental results show the validity of the proposed method for joint demosaicing and denoising.

Simultaneous Denoising of Dynamic PET Images Based on Deep Image Prior.

Parametric imaging obtained from kinetic modeling analysis of dynamic positron emission tomography (PET) data is a useful tool for quantifying tracer kinetics. However, pixel-wise time-activity curves have high noise levels which lead to poor quality of parametric images. To solve this limitation, we proposed a new image denoising method based on deep image prior (DIP). Like the original DIP method, the proposed DIP method is an unsupervised method, in which no training dataset is required. However, the difference is that our method can simultaneously denoise all dynamic PET images. Moreover, we propose a modified version of the DIP method called double DIP (DDIP), which has two DIP architectures. The additional DIP model is used to generate high-quality input data for the second DIP model. Computer simulations were performed to evaluate the performance of the proposed DIP-based methods. Our simulation results showed that the DDIP method outperformed the single DIP method. In addition, the DDIP method combined with data augmentation could generate PET parametric images with superior image quality compared to the spatiotemporal-based non-local means filtering and high constrained backprojection. Our preliminary results show that our proposed DDIP method is a novel and effective unsupervised method for simultaneously denoising dynamic PET images.

An Advanced Noise Reduction and Edge Enhancement Algorithm.

Complementary metal-oxide-semiconductor (CMOS) image sensors can cause noise in images collected or transmitted in unfavorable environments, especially low-illumination scenarios. Numerous approaches have been developed to solve the problem of image noise removal. However, producing natural and high-quality denoised images remains a crucial challenge. To meet this challenge, we introduce a novel approach for image denoising with the following three main contributions. First, we devise a deep image prior-based module that can produce a noise-reduced image as well as a contrast-enhanced denoised one from a noisy input image. Second, the produced images are passed through a proposed image fusion (IF) module based on Laplacian pyramid decomposition to combine them and prevent noise amplification and color shift. Finally, we introduce a progressive refinement (PR) module, which adopts the summed-area tables to take advantage of spatially correlated information for edge and image quality enhancement. Qualitative and quantitative evaluations demonstrate the efficiency, superiority, and robustness of our proposed method.

Joint Demosaicing and Denoising Based on a Variational Deep Image Prior Neural Network.

A joint demosaicing and denoising task refers to the task of simultaneously reconstructing and denoising a color image from a patterned image obtained by a monochrome image sensor with a color filter array. Recently, inspired by the success of deep learning in many image processing tasks, there has been research to apply convolutional neural networks (CNNs) to the task of joint demosaicing and denoising. However, such CNNs need many training data to be trained, and work well only for patterned images which have the same amount of noise they have been trained on. In this paper, we propose a variational deep image prior network for joint demosaicing and denoising which can be trained on a single patterned image and works for patterned images with different levels of noise. We also propose a new RGB color filter array (CFA) which works better with the proposed network than the conventional Bayer CFA. Mathematical justifications of why the variational deep image prior network suits the task of joint demosaicing and denoising are also given, and experimental results verify the performance of the proposed method.

Reference-Driven Compressed Sensing MR Image Reconstruction Using Deep Convolutional Neural Networks without Pre-Training.

Deep learning has proven itself to be able to reduce the scanning time of Magnetic Resonance Imaging (MRI) and to improve the image reconstruction quality since it was introduced into Compressed Sensing MRI (CS-MRI). However, the requirement of using large, high-quality, and patient-based datasets for network training procedures is always a challenge in clinical applications. In this paper, we propose a novel deep learning based compressed sensing MR image reconstruction method that does not require any pre-training procedure or training dataset, thereby largely reducing clinician dependence on patient-based datasets. The proposed method is based on the Deep Image Prior (DIP) framework and uses a high-resolution reference MR image as the input of the convolutional neural network in order to induce the structural prior in the learning procedure. This reference-driven strategy improves the efficiency and effect of network learning. We then add the k-space data correction step to enforce the consistency of the k-space data with the measurements, which further improve the image reconstruction accuracy. Experiments on in vivo MR datasets showed that the proposed method can achieve more accurate reconstruction results from undersampled k-space data.

Population-based deep image prior for dynamic PET denoising: A data-driven approach to improve parametric quantification.

The high noise level of dynamic Positron Emission Tomography (PET) images degrades the quality of parametric images. In this study, we aim to improve the quality and quantitative accuracy of Ki images by utilizing deep learning techniques to reduce the noise in dynamic PET images. We propose a novel denoising technique, Population-based Deep Image Prior (PDIP), which integrates population-based prior information into the optimization process of Deep Image Prior (DIP). Specifically, the population-based prior image is generated from a supervised denoising model that is trained on a prompts-matched static PET dataset comprising 100 clinical studies. The 3D U-Net architecture is employed for both the supervised model and the following DIP optimization process. We evaluated the efficacy of PDIP for noise reduction in 25%-count and 100%-count dynamic PET images from 23 patients by comparing with two other baseline techniques: the Prompts-matched Supervised model (PS) and a conditional DIP (CDIP) model that employs the mean static PET image as the prior. Both the PS and CDIP models show effective noise reduction but result in smoothing and removal of small lesions. In addition, the utilization of a single static image as the prior in the CDIP model also introduces a similar tracer distribution to the denoised dynamic frames, leading to lower Ki in general as well as incorrect Ki in the descending aorta. By contrast, as the proposed PDIP model utilizes intrinsic image features from the dynamic dataset and a large clinical static dataset, it not only achieves comparable noise reduction as the supervised and CDIP models but also improves lesion Ki predictions.

RBP-DIP: Residual back projection with deep image prior for ill-posed CT reconstruction.

The success of deep image prior (DIP) in a number of image processing tasks has motivated their application in image reconstruction problems in computed tomography (CT). In this paper, we introduce a residual back projection technique (RBP) that improves the performance of deep image prior framework in iterative CT reconstruction, especially when the reconstruction problem is highly ill-posed. The RBP-DIP framework uses an untrained U-net in conjunction with a novel residual back projection connection to minimize the objective function while improving reconstruction accuracy. In each iteration, the weights of the untrained U-net are optimized, and the output of the U-net in the current iteration is used to update the input of the U-net in the next iteration through the proposed RBP connection. The introduction of the RBP connection strengthens the regularization effects of the DIP framework in the context of iterative CT reconstruction leading to improvements in accuracy. Our experiments demonstrate that the RBP-DIP framework offers improvements over other state-of-the-art conventional IR methods, as well as pre-trained and untrained models with similar network structures under multiple conditions. These improvements are particularly significant in the few-view and limited-angle CT reconstructions, where the corresponding inverse problems are highly ill-posed and the training data is limited. Furthermore, RBP-DIP has the potential for further improvement. Most existing IR algorithms, pre-trained models, and enhancements applicable to the original DIP algorithm can also be integrated into the RBP-DIP framework.

An iterative reconstruction algorithm for unsupervised PET image.

Objective.In recent years, convolutional neural networks (CNNs) have shown great potential in positron emission tomography (PET) image reconstruction. However, most of them rely on many low-quality and high-quality reference PET image pairs for training, which are not always feasible in clinical practice. On the other hand, many works improve the quality of PET image reconstruction by adding explicit regularization or optimizing the network structure, which may lead to complex optimization problems.Approach.In this paper, we develop a novel iterative reconstruction algorithm by integrating the deep image prior (DIP) framework, which only needs the prior information (e.g. MRI) and sinogram data of patients. To be specific, we construct the objective function as a constrained optimization problem and utilize the existing PET image reconstruction packages to streamline calculations. Moreover, to further improve both the reconstruction quality and speed, we introduce the Nesterov's acceleration part and the restart mechanism in each iteration.Main results.2D experiments on PET data sets based on computer simulations and real patients demonstrate that our proposed algorithm can outperform existing MLEM-GF, KEM and DIPRecon methods.Significance.Unlike traditional CNN methods, the proposed algorithm does not rely on large data sets, but only leverages inter-patient information. Furthermore, we enhance reconstruction performance by optimizing the iterative algorithm. Notably, the proposed method does not require much modification of the basic algorithm, allowing for easy integration into standard implementations.

A hybrid deep image prior and compressed sensing reconstruction method for highly accelerated 3D coronary magnetic resonance angiography.

Introduction: High-resolution whole-heart coronary magnetic resonance angiography (CMRA) often suffers from unreasonably long scan times, rendering imaging acceleration highly desirable. Traditional reconstruction methods used in CMRA rely on either hand-crafted priors or supervised learning models. Although the latter often yield superior reconstruction quality, they require a large amount of training data and memory resources, and may encounter generalization issues when dealing with out-of-distribution datasets. Methods: To address these challenges, we introduce an unsupervised reconstruction method that combines deep image prior (DIP) with compressed sensing (CS) to accelerate 3D CMRA. This method incorporates a slice-by-slice DIP reconstruction and 3D total variation (TV) regularization, enabling high-quality reconstruction under a significant acceleration while enforcing continuity in the slice direction. We evaluated our method by comparing it to iterative SENSE, CS-TV, CS-wavelet, and other DIP-based variants, using both retrospectively and prospectively undersampled datasets. Results: The results demonstrate the superiority of our 3D DIP-CS approach, which improved the reconstruction accuracy relative to the other approaches across both datasets. Ablation studies further reveal the benefits of combining DIP with 3D TV regularization, which leads to significant improvements of image quality over pure DIP-based methods. Evaluation of vessel sharpness and image quality scores shows that DIP-CS improves the quality of reformatted coronary arteries. Discussion: The proposed method enables scan-specific reconstruction of high-quality 3D CMRA from a five-minute acquisition, without relying on fully-sampled training data or placing a heavy burden on memory resources.