Missing Wedge Completion via Unsupervised Learning with Coordinate Networks.

Cryogenic electron tomography (cryoET) is a powerful tool in structural biology, enabling detailed 3D imaging of biological specimens at a resolution of nanometers. Despite its potential, cryoET faces challenges such as the missing wedge problem, which limits reconstruction quality due to incomplete data collection angles. Recently, supervised deep learning methods leveraging convolutional neural networks (CNNs) have considerably addressed this issue; however, their pretraining requirements render them susceptible to inaccuracies and artifacts, particularly when representative training data is scarce. To overcome these limitations, we introduce a proof-of-concept unsupervised learning approach using coordinate networks (CNs) that optimizes network weights directly against input projections. This eliminates the need for pretraining, reducing reconstruction runtime by 3-20× compared to supervised methods. Our in silico results show improved shape completion and reduction of missing wedge artifacts, assessed through several voxel-based image quality metrics in real space and a novel directional Fourier Shell Correlation (FSC) metric. Our study illuminates benefits and considerations of both supervised and unsupervised approaches, guiding the development of improved reconstruction strategies.

Missing Wedge Completion via Unsupervised Learning with Coordinate Networks.

Cryogenic electron tomography (cryoET) is a powerful tool in structural biology, enabling detailed 3D imaging of biological specimens at a resolution of nanometers. Despite its potential, cryoET faces challenges such as the missing wedge problem, which limits reconstruction quality due to incomplete data collection angles. Recently, supervised deep learning methods leveraging convolutional neural networks (CNNs) have considerably addressed this issue; however, their pretraining requirements render them susceptible to inaccuracies and artifacts, particularly when representative training data is scarce. To overcome these limitations, we introduce a proof-of-concept unsupervised learning approach using coordinate networks (CNs) that optimizes network weights directly against input projections. This eliminates the need for pretraining, reducing reconstruction runtime by 3 - 20× compared to supervised methods. Our in silico results show improved shape completion and reduction of missing wedge artifacts, assessed through several voxel-based image quality metrics in real space and a novel directional Fourier Shell Correlation (FSC) metric. Our study illuminates benefits and considerations of both supervised and unsupervised approaches, guiding the development of improved reconstruction strategies.

Adapted large language models can outperform medical experts in clinical text summarization.

Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown promise in natural language processing (NLP) tasks, their effectiveness on a diverse range of clinical summarization tasks remains unproven. Here we applied adaptation methods to eight LLMs, spanning four distinct clinical summarization tasks: radiology reports, patient questions, progress notes and doctor-patient dialogue. Quantitative assessments with syntactic, semantic and conceptual NLP metrics reveal trade-offs between models and adaptation methods. A clinical reader study with 10 physicians evaluated summary completeness, correctness and conciseness; in most cases, summaries from our best-adapted LLMs were deemed either equivalent (45%) or superior (36%) compared with summaries from medical experts. The ensuing safety analysis highlights challenges faced by both LLMs and medical experts, as we connect errors to potential medical harm and categorize types of fabricated information. Our research provides evidence of LLMs outperforming medical experts in clinical text summarization across multiple tasks. This suggests that integrating LLMs into clinical workflows could alleviate documentation burden, allowing clinicians to focus more on patient care.

Clinical Text Summarization: Adapting Large Language Models Can Outperform Human Experts.

Sifting through vast textual data and summarizing key information from electronic health records (EHR) imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown immense promise in natural language processing (NLP) tasks, their efficacy on a diverse range of clinical summarization tasks has not yet been rigorously demonstrated. In this work, we apply domain adaptation methods to eight LLMs, spanning six datasets and four distinct clinical summarization tasks: radiology reports, patient questions, progress notes, and doctor-patient dialogue. Our thorough quantitative assessment reveals trade-offs between models and adaptation methods in addition to instances where recent advances in LLMs may not improve results. Further, in a clinical reader study with ten physicians, we show that summaries from our best-adapted LLMs are preferable to human summaries in terms of completeness and correctness. Our ensuing qualitative analysis highlights challenges faced by both LLMs and human experts. Lastly, we correlate traditional quantitative NLP metrics with reader study scores to enhance our understanding of how these metrics align with physician preferences. Our research marks the first evidence of LLMs outperforming human experts in clinical text summarization across multiple tasks. This implies that integrating LLMs into clinical workflows could alleviate documentation burden, empowering clinicians to focus more on personalized patient care and the inherently human aspects of medicine.

Noise2Recon: Enabling SNR-robust MRI reconstruction with semi-supervised and self-supervised learning.

PURPOSE: To develop a method for building MRI reconstruction neural networks robust to changes in signal-to-noise ratio (SNR) and trainable with a limited number of fully sampled scans. METHODS: We propose Noise2Recon, a consistency training method for SNR-robust accelerated MRI reconstruction that can use both fully sampled (labeled) and undersampled (unlabeled) scans. Noise2Recon uses unlabeled data by enforcing consistency between model reconstructions of undersampled scans and their noise-augmented counterparts. Noise2Recon was compared to compressed sensing and both supervised and self-supervised deep learning baselines. Experiments were conducted using retrospectively accelerated data from the mridata three-dimensional fast-spin-echo knee and two-dimensional fastMRI brain datasets. All methods were evaluated in label-limited settings and among out-of-distribution (OOD) shifts, including changes in SNR, acceleration factors, and datasets. An extensive ablation study was conducted to characterize the sensitivity of Noise2Recon to hyperparameter choices. RESULTS: In label-limited settings, Noise2Recon achieved better structural similarity, peak signal-to-noise ratio, and normalized-RMS error than all baselines and matched performance of supervised models, which were trained with 14 × $$ 14\times $$ more fully sampled scans. Noise2Recon outperformed all baselines, including state-of-the-art fine-tuning and augmentation techniques, among low-SNR scans and when generalizing to OOD acceleration factors. Augmentation extent and loss weighting hyperparameters had negligible impact on Noise2Recon compared to supervised methods, which may indicate increased training stability. CONCLUSION: Noise2Recon is a label-efficient reconstruction method that is robust to distribution shifts, such as changes in SNR, acceleration factors, and others, with limited or no fully sampled training data.

SLfRank: Shinnar-Le-Roux Pulse Design With Reduced Energy and Accurate Phase Profiles Using Rank Factorization.

The Shinnar-Le-Roux (SLR) algorithm is widely used to design frequency selective pulses with large flip angles. We improve its design process to generate pulses with lower energy (by as much as 26%) and more accurate phase profiles. Concretely, the SLR algorithm consists of two steps: (1) an invertible transform between frequency selective pulses and polynomial pairs that represent Cayley-Klein (CK) parameters and (2) the design of the CK polynomial pair to match the desired magnetization profiles. Because the CK polynomial pair is bi-linearly coupled, the original algorithm sequentially solves for each polynomial instead of jointly. This results in sub-optimal pulses. Instead, we leverage a convex relaxation technique, commonly used for low rank matrix recovery, to address the bi-linearity. Our numerical experiments show that the resulting pulses are almost always globally optimal in practice. For slice excitation, the proposed algorithm results in more accurate linear phase profiles. And in general the improved pulses have lower energy than the original SLR pulses.

Optimization in the space domain for density compensation with the nonuniform FFT.

The non-uniform Discrete Fourier Transform algorithm has shown great utility for reconstructing images from non-uniformly spaced Fourier samples in several imaging modalities. Due to the non-uniform spacing, some correction for the variable density of the samples must be made. Common methods for generating density compensation values are either sub-optimal or only consider a finite set of points in the optimization. This manuscript presents an algorithm for generating density compensation values from a set of Fourier samples that takes into account the point spread function over an entire rectangular region in the image domain. We show that the reconstructed images using the density compensation values of this method are of superior quality when compared to other standard methods. Results are shown with a numerical phantom and with magnetic resonance images of the abdomen and the knee.

Automated MRI Field of View Prescription from Region of Interest Prediction by Intra-Stack Attention Neural Network.

Manual prescription of the field of view (FOV) by MRI technologists is variable and prolongs the scanning process. Often, the FOV is too large or crops critical anatomy. We propose a deep learning framework, trained by radiologists' supervision, for automating FOV prescription. An intra-stack shared feature extraction network and an attention network are used to process a stack of 2D image inputs to generate scalars defining the location of a rectangular region of interest (ROI). The attention mechanism is used to make the model focus on a small number of informative slices in a stack. Then, the smallest FOV that makes the neural network predicted ROI free of aliasing is calculated by an algebraic operation derived from MR sampling theory. The framework's performance is examined quantitatively with intersection over union (IoU) and pixel error on position and qualitatively with a reader study. The proposed model achieves an average IoU of 0.867 and an average ROI position error of 9.06 out of 512 pixels on 80 test cases, significantly better than two baseline models and not significantly different from a radiologist. Finally, the FOV given by the proposed framework achieves an acceptance rate of 92% from an experienced radiologist.

Deep Learning-Based Water-Fat Separation from Dual-Echo Chemical Shift-Encoded Imaging.

Conventional water-fat separation approaches suffer long computational times and are prone to water/fat swaps. To solve these problems, we propose a deep learning-based dual-echo water-fat separation method. With IRB approval, raw data from 68 pediatric clinically indicated dual echo scans were analyzed, corresponding to 19382 contrast-enhanced images. A densely connected hierarchical convolutional network was constructed, in which dual-echo images and corresponding echo times were used as input and water/fat images obtained using the projected power method were regarded as references. Models were trained and tested using knee images with 8-fold cross validation and validated on out-of-distribution data from the ankle, foot, and arm. Using the proposed method, the average computational time for a volumetric dataset with ~400 slices was reduced from 10 min to under one minute. High fidelity was achieved (correlation coefficient of 0.9969, l1 error of 0.0381, SSIM of 0.9740, pSNR of 58.6876) and water/fat swaps were mitigated. I is of particular interest that metal artifacts were substantially reduced, even when the training set contained no images with metallic implants. Using the models trained with only contrast-enhanced images, water/fat images were predicted from non-contrast-enhanced images with high fidelity. The proposed water-fat separation method has been demonstrated to be fast, robust, and has the added capability to compensate for metal artifacts.

A Semi-Blind Calibration and Compensation Method for Dynamic Range Recovery of Low-Power Pre-Amplifiers in MRI Receive Chains.

To enable wireless MRI receive arrays, per-channel power consumption must be reduced by a significant factor. To address this, a low-power SiGe alternative to industry standard MRI pre-amplifier blocks has been proposed and its impact on imaging performance evaluated in a benchtop environment. The SiGe amplifier reduces power consumption 28x, but exhibits increased non-linearity and reduced dynamic range relative to industry standard amplifiers. This distorts the images, causing reduced contrast and a blurring of fine features. In conjunction with the amplifier, a semi-blind calibration and compensation framework has been proposed to remove artifacts caused by this non-linearity. Requiring the knowledge of the calibration signal bandwidth, the associated peak transmit powers, and the distorted baseband signals, a second non-linearity is constructed that when cascaded with the receive chain produces a linear response. This method was evaluated for both knee and phantom image datasets of peak input power -20dBm with a -40dBm peak input power image as reference. In the benchtop environment, industry standard amplifiers produced input normalized RMSEs of 0.0199 and 0.0310 for phantom and knee datasets, respectively. The low-power SiGe amplifier resulted in RMSEs of 0.0869 and 0.1130 which were reduced to 0.0158 and 0.0168 following compensation, for phantom and knee images respectively. The ability to effectively compensate for this reduced dynamic range encourages further investigation of low-power SiGe amplifiers for power limited MRI receive arrays.

Rapid fat-water separated T1 mapping using a single-shot radial inversion-recovery spoiled gradient recalled pulse sequence.

T1 mapping is increasingly used in clinical practice and research studies. With limited scan time, existing techniques often have limited spatial resolution, contrast resolution and slice coverage. High fat concentrations yield complex errors in Look-Locker T1 methods. In this study, a dual-echo 2D radial inversion-recovery T1 (DEradIR-T1) technique was developed for fast fat-water separated T1 mapping. The DEradIR-T1 technique was tested in phantoms, 5 volunteers and 28 patients using a 3 T clinical MRI scanner. In our study, simulations were performed to analyze the composite (fat + water) and water-only T1 under different echo times (TE). In standardized phantoms, an inversion-recovery spin echo (IR-SE) sequence with and without fat saturation pulses served as a T1 reference. Parameter mapping with DEradIR-T1 was also assessed in vivo, and values were compared with modified Look-Locker inversion recovery (MOLLI). Bland-Altman analysis and two-tailed paired t-tests were used to compare the parameter maps from DEradIR-T1 with the references. Simulations of the composite and water-only T1 under different TE values and levels of fat matched the in vivo studies. T1 maps from DEradIR-T1 on a NIST phantom (Pcomp = 0.97) and a Calimetrix fat-water phantom (Pwater = 0.56) matched with the references. In vivo T1 was compared with that of MOLLI: R comp 2 = 0.77 ; R water 2 = 0.72 . In this work, intravoxel fat is found to have a variable, echo-time-dependent effect on measured T1 values, and this effect may be mitigated using the proposed DRradIR-T1.

A geometry-informed deep learning framework for ultra-sparse 3D tomographic image reconstruction.

Deep learning affords enormous opportunities to augment the armamentarium of biomedical imaging. However, the pure data-driven nature of deep learning models may limit the model generalizability and application scope. Here we establish a geometry-informed deep learning framework for ultra-sparse 3D tomographic image reconstruction. We introduce a novel mechanism for integrating geometric priors of the imaging system. We demonstrate that the seamless inclusion of known priors is essential to enhance the performance of 3D volumetric computed tomography imaging with ultra-sparse sampling. The study opens new avenues for data-driven biomedical imaging and promises to provide substantially improved imaging tools for various clinical imaging and image-guided interventions.

NeRP: Implicit Neural Representation Learning With Prior Embedding for Sparsely Sampled Image Reconstruction.

Image reconstruction is an inverse problem that solves for a computational image based on sampled sensor measurement. Sparsely sampled image reconstruction poses additional challenges due to limited measurements. In this work, we propose a methodology of implicit Neural Representation learning with Prior embedding (NeRP) to reconstruct a computational image from sparsely sampled measurements. The method differs fundamentally from previous deep learning-based image reconstruction approaches in that NeRP exploits the internal information in an image prior and the physics of the sparsely sampled measurements to produce a representation of the unknown subject. No large-scale data is required to train the NeRP except for a prior image and sparsely sampled measurements. In addition, we demonstrate that NeRP is a general methodology that generalizes to different imaging modalities such as computed tomography (CT) and magnetic resonance imaging (MRI). We also show that NeRP can robustly capture the subtle yet significant image changes required for assessing tumor progression.

Novel-view X-ray projection synthesis through geometry-integrated deep learning.

X-ray imaging is a widely used approach to view the internal structure of a subject for clinical diagnosis, image-guided interventions and decision-making. The X-ray projections acquired at different view angles provide complementary information of patient's anatomy and are required for stereoscopic or volumetric imaging of the subject. In reality, obtaining multiple-view projections inevitably increases radiation dose and complicates clinical workflow. Here we investigate a strategy of obtaining the X-ray projection image at a novel view angle from a given projection image at a specific view angle to alleviate the need for actual projection measurement. Specifically, a Deep Learning-based Geometry-Integrated Projection Synthesis (DL-GIPS) framework is proposed for the generation of novel-view X-ray projections. The proposed deep learning model extracts geometry and texture features from a source-view projection, and then conducts geometry transformation on the geometry features to accommodate the change of view angle. At the final stage, the X-ray projection in the target view is synthesized from the transformed geometry and the shared texture features via an image generator. The feasibility and potential impact of the proposed DL-GIPS model are demonstrated using lung imaging cases. The proposed strategy can be generalized to a general case of multiple projections synthesis from multiple input views and potentially provides a new paradigm for various stereoscopic and volumetric imaging with substantially reduced efforts in data acquisition.

Artifact- and content-specific quality assessment for MRI with image rulers.

In clinical practice MR images are often first seen by radiologists long after the scan. If image quality is inadequate either patients have to return for an additional scan, or a suboptimal interpretation is rendered. An automatic image quality assessment (IQA) would enable real-time remediation. Existing IQA works for MRI give only a general quality score, agnostic to the cause of and solution to low-quality scans. Furthermore, radiologists' image quality requirements vary with the scan type and diagnostic task. Therefore, the same score may have different implications for different scans. We propose a framework with multi-task CNN model trained with calibrated labels and inferenced with image rulers. Labels calibrated by human inputs follow a well-defined and efficient labeling task. Image rulers address varying quality standards and provide a concrete way of interpreting raw scores from the CNN. The model supports assessments of two of the most common artifacts in MRI: noise and motion. It achieves accuracies of around 90%, 6% better than the best previous method examined, and 3% better than human experts on noise assessment. Our experiments show that label calibration, image rulers, and multi-task training improve the model's performance and generalizability.

Attention-guided deep learning for gestational age prediction using fetal brain MRI.

Magnetic resonance imaging offers unrivaled visualization of the fetal brain, forming the basis for establishing age-specific morphologic milestones. However, gauging age-appropriate neural development remains a difficult task due to the constantly changing appearance of the fetal brain, variable image quality, and frequent motion artifacts. Here we present an end-to-end, attention-guided deep learning model that predicts gestational age with R2 score of 0.945, mean absolute error of 6.7 days, and concordance correlation coefficient of 0.970. The convolutional neural network was trained on a heterogeneous dataset of 741 developmentally normal fetal brain images ranging from 19 to 39 weeks in gestational age. We also demonstrate model performance and generalizability using independent datasets from four academic institutions across the U.S. and Turkey with R2 scores of 0.81-0.90 after minimal fine-tuning. The proposed regression algorithm provides an automated machine-enabled tool with the potential to better characterize in utero neurodevelopment and guide real-time gestational age estimation after the first trimester.

Utilizing the Wavelet Transform's Structure in Compressed Sensing.

Compressed sensing has empowered quality image reconstruction with fewer data samples than previously thought possible. These techniques rely on a sparsifying linear transformation. The Daubechies wavelet transform is commonly used for this purpose. In this work, we take advantage of the structure of this wavelet transform and identify an affine transformation that increases the sparsity of the result. After inclusion of this affine transformation, we modify the resulting optimization problem to comply with the form of the Basis Pursuit Denoising problem. Finally, we show theoretically that this yields a lower bound on the error of the reconstruction and present results where solving this modified problem yields images of higher quality for the same sampling patterns using both magnetic resonance and optical images.

Biopsy marker localization with thermo-acoustic ultrasound for lumpectomy guidance.

PURPOSE: Almost one in four lumpectomies fails to fully remove cancerous tissue from the breast, requiring reoperation. This high failure rate suggests that existing lumpectomy guidance methods are inadequate for allowing surgeons to consistently identify the proper volume of tissue for excision. Current guidance techniques either provide little information about the tumor position or require surgeons to frequently switch between making incisions and manually probing for a marker placed at the lesion site. This article explores the feasibility of thermo-acoustic ultrasound (TAUS) to enable hands-free localization of metallic biopsy markers throughout surgery, which would allow for continuous visualization of the lesion site in the breast without the interruption of surgery. In a TAUS-based localization system, microwave excitations would be transmitted into the breast, and the amplification in microwave absorption around the metallic markers would generate acoustic signals from the marker sites through the thermo-acoustic effect. Detection and ranging of these signals by multiple acoustic receivers on the breast could then enable marker localization through acoustic multilateration. METHODS: Physics simulations were used to characterize the TAUS signals generated from different markers by microwave excitations. First, electromagnetic simulations determined the spatial pattern of the amplification in microwave absorption around the markers. Then, acoustic simulations characterized the acoustic fields generated from these markers at various acoustic frequencies. TAUS-based one-dimensional (1D) ranging of two metallic markers-including a biopsy marker that is FDA-approved for clinical use-immersed in saline was also performed using a bench-top setup. To perform TAUS acquisitions, a microwave applicator was driven by 2.66 GHz microwave signals that were amplitude-modulated by chirps at the desired acoustic excitation frequencies, and the resulting TAUS signal from the markers was detected by an ultrasonic transducer. RESULTS: The simulation results show that the geometry of the marker strongly impacts the quantity and spatial pattern of both the microwave absorption around the marker and the resulting TAUS signal generated from the marker. The simulated TAUS signal maps and acoustic frequency responses also make clear that the marker geometry plays an important role in determining the overall system response. Using the bench-top setup, TAUS detection and 1D localization of the markers were successfully demonstrated for multiple different combinations of microwave applicator and metallic marker. These initial results indicate that TAUS-based localization of biopsy markers is feasible. CONCLUSIONS: Through microwave excitations and acoustic detection, TAUS can be used to localize metallic biopsy markers. With further development, TAUS opens new avenues to enable a more intuitive lumpectomy guidance system that could help to achieve better lumpectomy outcomes.

Analysis of deep complex-valued convolutional neural networks for MRI reconstruction and phase-focused applications.

PURPOSE: Deep learning has had success with MRI reconstruction, but previously published works use real-valued networks. The few works which have tried complex-valued networks have not fully assessed their impact on phase. Therefore, the purpose of this work is to fully investigate end-to-end complex-valued convolutional neural networks (CNNs) for accelerated MRI reconstruction and in several phase-based applications in comparison to 2-channel real-valued networks. METHODS: Several complex-valued activation functions for MRI reconstruction were implemented, and their performance was compared. Complex-valued convolution was implemented and tested on an unrolled network architecture and a U-Net-based architecture over a wide range of network widths and depths with knee, body, and phase-contrast datasets. RESULTS: Quantitative and qualitative results demonstrated that complex-valued CNNs with complex-valued convolutions provided superior reconstructions compared to real-valued convolutions with the same number of trainable parameters for both an unrolled network architecture and a U-Net-based architecture, and for 3 different datasets. Complex-valued CNNs consistently had superior normalized RMS error, structural similarity index, and peak SNR compared to real-valued CNNs. CONCLUSION: Complex-valued CNNs can enable superior accelerated MRI reconstruction and phase-based applications such as fat-water separation, and flow quantification compared to real-valued convolutional neural networks.