Neural étendue expander for ultra-wide-angle high-fidelity holographic display.

Holographic displays can generate light fields by dynamically modulating the wavefront of a coherent beam of light using a spatial light modulator, promising rich virtual and augmented reality applications. However, the limited spatial resolution of existing dynamic spatial light modulators imposes a tight bound on the diffraction angle. As a result, modern holographic displays possess low étendue, which is the product of the display area and the maximum solid angle of diffracted light. The low étendue forces a sacrifice of either the field-of-view (FOV) or the display size. In this work, we lift this limitation by presenting neural étendue expanders. This new breed of optical elements, which is learned from a natural image dataset, enables higher diffraction angles for ultra-wide FOV while maintaining both a compact form factor and the fidelity of displayed contents to human viewers. With neural étendue expanders, we experimentally achieve 64 × étendue expansion of natural images in full color, expanding the FOV by an order of magnitude horizontally and vertically, with high-fidelity reconstruction quality (measured in PSNR) over 29 dB on retinal-resolution images.

Usformer: A small network for left atrium segmentation of 3D LGE MRI.

Left atrial (LA) fibrosis plays a vital role as a mediator in the progression of atrial fibrillation. 3D late gadolinium-enhancement (LGE) MRI has been proven effective in identifying LA fibrosis. Image analysis of 3D LA LGE involves manual segmentation of the LA wall, which is both lengthy and challenging. Automated segmentation poses challenges owing to the diverse intensities in data from various vendors, the limited contrast between LA and surrounding tissues, and the intricate anatomical structures of the LA. Current approaches relying on 3D networks are computationally intensive since 3D LGE MRIs and the networks are large. Regarding this issue, most researchers came up with two-stage methods: initially identifying the LA center using a scaled-down version of the MRIs and subsequently cropping the full-resolution MRIs around the LA center for final segmentation. We propose a lightweight transformer-based 3D architecture, Usformer, designed to precisely segment LA volume in a single stage, eliminating error propagation associated with suboptimal two-stage training. The transposed attention facilitates capturing the global context in large 3D volumes without significant computation requirements. Usformer outperforms the state-of-the-art supervised learning methods in terms of accuracy and speed. First, with the smallest Hausdorff Distance (HD) and Average Symmetric Surface Distance (ASSD), it achieved a dice score of 93.1% and 92.0% in the 2018 Atrial Segmentation Challenge and our local institutional dataset, respectively. Second, the number of parameters and computation complexity are largely reduced by 2.8x and 3.8x, respectively. Moreover, Usformer does not require a large dataset. When only 16 labeled MRI scans are used for training, Usformer achieves a 92.1% dice score in the challenge dataset. The proposed Usformer delineates the boundaries of the LA wall relatively accurately, which may assist in the clinical translation of LA LGE for planning catheter ablation of atrial fibrillation.

Accelerating compressed sensing reconstruction of subsampled radial k-space data using geometrically-derived density compensation.

Objective.To accelerate compressed sensing (CS) reconstruction of subsampled radial k-space data using a geometrically-derived density compensation function (gDCF) without significant loss in image quality.Approach.We developed a theoretical framework to calculate a gDCF based on Nyquist distance along the radial and circumferential directions of a discrete polar coordinate system. Our gDCF was compared against standard DCF (e.g. ramp filter) and another commonly used DCF (modified Shepp-Logan (SL) filter). The resulting image quality produced by each DCF was quantified using normalized root-mean-square-error (NRMSE), blur metric (1 = blurriest; 0 = sharpest), and structural similarity index (SSIM; 1 = perfect match; 0 = no match) compared with the reference. For filtered backprojection (FBP) of phantom data obtained at the Nyquist sampling rate, Cartesian k-space sampling was used as the reference. For CS reconstruction of subsampled cardiac magnetic resonance imaging datasets (real-time cardiac cine data with 11 projections per frame,n = 20 patients; cardiac perfusion data with 30 projections per frame,n = 19 patients), CS reconstruction without DCF was used as the reference.Main results.The NRMSE, SSIM, and blur metrics of the phantom data were good for all DCFs, confirming that our gDCF produces uniform densities at the upper limit (Nyquist). For CS reconstruction of subsampled real-time cine and cardiac perfusion datasets, the image quality metrics (SSIM, NRMSE) were significantly (p < 0.05) higher for our gDCF than other DCFs, and the reconstruction time was significantly (p < 0.05) faster for our gDCF (reference) than no DCF (11.9%-52.9% slower), standard DCF (23.9%-57.6% slower), and modified SL filter (13.5%-34.8% slower).Significance.The proposed gDCF accelerates CS reconstruction of subsampled radial k-space data without significant loss in image quality compared with no DCF as the reference.

Intensity interferometry-based 3D imaging.

The development of single-photon counting detectors and arrays has made tremendous steps in recent years, not the least because of various new applications, e.g., LIDAR devices. In this work, a 3D imaging device based on real thermal light intensity interferometry is presented. By using gated SPAD technology, a basic 3D scene is imaged in reasonable measurement time. Compared to conventional approaches, the proposed synchronized photon counting allows the use of more light modes to enhance 3D ranging performance. Advantages like robustness to atmospheric scattering or autonomy by exploiting external light sources can make this ranging approach interesting for future applications.

Rapid reconstruction of highly undersampled, non-Cartesian real-time cine k-space data using a perceptual complex neural network (PCNN).

Highly accelerated real-time cine MRI using compressed sensing (CS) is a promising approach to achieve high spatio-temporal resolution and clinically acceptable image quality in patients with arrhythmia and/or dyspnea. However, its lengthy image reconstruction time may hinder its clinical translation. The purpose of this study was to develop a neural network for reconstruction of non-Cartesian real-time cine MRI k-space data faster (<1 min per slice with 80 frames) than graphics processing unit (GPU)-accelerated CS reconstruction, without significant loss in image quality or accuracy in left ventricular (LV) functional parameters. We introduce a perceptual complex neural network (PCNN) that trains on complex-valued MRI signal and incorporates a perceptual loss term to suppress incoherent image details. This PCNN was trained and tested with multi-slice, multi-phase, cine images from 40 patients (20 for training, 20 for testing), where the zero-filled images were used as input and the corresponding CS reconstructed images were used as practical ground truth. The resulting images were compared using quantitative metrics (structural similarity index (SSIM) and normalized root mean square error (NRMSE)) and visual scores (conspicuity, temporal fidelity, artifacts, and noise scores), individually graded on a five-point scale (1, worst; 3, acceptable; 5, best), and LV ejection fraction (LVEF). The mean processing time per slice with 80 frames for PCNN was 23.7 ± 1.9 s for pre-processing (Step 1, same as CS) and 0.822 ± 0.004 s for dealiasing (Step 2, 166 times faster than CS). Our PCNN produced higher data fidelity metrics (SSIM = 0.88 ± 0.02, NRMSE = 0.014 ± 0.004) compared with CS. While all the visual scores were significantly different (P < 0.05), the median scores were all 4.0 or higher for both CS and PCNN. LVEFs measured from CS and PCNN were strongly correlated (R2 = 0.92) and in good agreement (mean difference = -1.4% [2.3% of mean]; limit of agreement = 10.6% [17.6% of mean]). The proposed PCNN is capable of rapid reconstruction (25 s per slice with 80 frames) of non-Cartesian real-time cine MRI k-space data, without significant loss in image quality or accuracy in LV functional parameters.

Rapid Reconstruction of Four-dimensional MR Angiography of the Thoracic Aorta Using a Convolutional Neural Network.

PURPOSE: To implement an integrated reconstruction pipeline including a graphics processing unit (GPU)-based convolutional neural network (CNN) architecture and test whether it reconstructs four-dimensional non-Cartesian, non-contrast material-enhanced MR angiographic k-space data faster than a central processing unit (CPU)-based compressed sensing (CS) reconstruction pipeline, without significant losses in data fidelity, summed visual score (SVS), or arterial vessel-diameter measurements. MATERIALS AND METHODS: Raw k-space data of 24 patients (18 men and six women; mean age, 56.8 years ± 11.8 [standard deviation]) suspected of having thoracic aortic disease were used to evaluate the proposed reconstruction pipeline derived from an open-source three-dimensional CNN. For training, 4800 zero-filled images and the corresponding CS-reconstructed images from 10 patients were used as input-output pairs. For testing, 6720 zero-filled images from 14 different patients were used as inputs to a trained CNN. Metrics for evaluating the agreement between the CNN and CS images included reconstruction times, structural similarity index (SSIM) and normalized root-mean-square error (NRMSE), SVS (3 = nondiagnostic, 9 = clinically acceptable, 15 = excellent), and vessel diameters. RESULTS: The mean reconstruction time was 65 times and 69 times shorter for the CPU-based and GPU-based CNN pipelines (216.6 seconds ± 40.5 and 204.9 seconds ± 40.5), respectively, than for CS (14 152.3 seconds ± 1708.6) (P < .001). Compared with CS as practical ground truth, CNNs produced high data fidelity (SSIM = 0.94 ± 0.02, NRMSE = 2.8% ± 0.4) and not significantly different (P = .25) SVS and aortic diameters, except at one out of seven locations, where the percentage difference was only 3% (ie, clinically irrelevant). CONCLUSION: The proposed integrated reconstruction pipeline including a CNN architecture is capable of rapidly reconstructing time-resolved volumetric cardiovascular MRI k-space data, without a significant loss in data quality, thereby supporting clinical translation of said non-contrast-enhanced MR angiograms. Supplemental material is available for this article. © RSNA, 2020.

Hand-guided qualitative deflectometry with a mobile device.

We introduce a system that exploits the screen and front-facing camera of a mobile device to perform three-dimensional deflectometry-based surface measurements. In contrast to current mobile deflectometry systems, our method can capture surfaces with large normal variation and wide field of view (FoV). We achieve this by applying automated multi-view panoramic stitching algorithms to produce a large FoV normal map from a hand-guided capture process without the need for external tracking systems, like robot arms or fiducials. The presented work enables 3D surface measurements of specular objects 'in the wild' with a system accessible to users with little to no technical imaging experience. We demonstrate high-quality 3D surface measurements without the need for a calibration procedure. We provide experimental results with our prototype Deflectometry system and discuss applications for computer vision tasks such as object detection and recognition.