BiPMAP: a toolbox for predicting perceived motion artifacts on modern displays.

Viewers of digital displays often experience motion artifacts (e.g., flicker, judder, edge banding, motion blur, color breakup, depth distortion) when presented with dynamic scenes. We developed an interactive software tool for display designers that predicts how a viewer perceives motion artifacts for a variety of stimulus, display, and viewing parameters: the Binocular Perceived Motion Artifact Predictor (BiPMAP). The tool enables the user to specify numerous stimulus, display, and viewing parameters. It implements a model of human spatiotemporal contrast sensitivity in order to determine which artifacts will be seen by a viewer and which will not. The tool visualizes the perceptual effects of discrete space-time sampling on the display by presenting side by side the expected perception when the stimulus is continuous compared to when the same stimulus is presented with the spatial and temporal parameters of a prototype display.

Data-driven modeling and control of an X-ray bimorph adaptive mirror.

Adaptive X-ray mirrors are being adopted on high-coherent-flux synchrotron and X-ray free-electron laser beamlines where dynamic phase control and aberration compensation are necessary to preserve wavefront quality from source to sample, yet challenging to achieve. Additional difficulties arise from the inability to continuously probe the wavefront in this context, which demands methods of control that require little to no feedback. In this work, a data-driven approach to the control of adaptive X-ray optics with piezo-bimorph actuators is demonstrated. This approach approximates the non-linear system dynamics with a discrete-time model using random mirror shapes and interferometric measurements as training data. For mirrors of this type, prior states and voltage inputs affect the shape-change trajectory, and therefore must be included in the model. Without the need for assumed physical models of the mirror's behavior, the generality of the neural network structure accommodates drift, creep and hysteresis, and enables a control algorithm that achieves shape control and stability below 2 nm RMS. Using a prototype mirror and ex situ metrology, it is shown that the accuracy of our trained model enables open-loop shape control across a diverse set of states and that the control algorithm achieves shape error magnitudes that fall within diffraction-limited performance.

Quantitative differential phase contrast imaging in an LED array microscope: errata.

This errata corrects typos found in the derived transfer functions in [Opt. Express23, 11394 (2015)10.1364/OE.23.011394].

Fast non-iterative algorithm for 3D point-cloud holography.

Recently developed iterative and deep learning-based approaches to computer-generated holography (CGH) have been shown to achieve high-quality photorealistic 3D images with spatial light modulators. However, such approaches remain overly cumbersome for patterning sparse collections of target points across a photoresponsive volume in applications including biological microscopy and material processing. Specifically, in addition to requiring heavy computation that cannot accommodate real-time operation in mobile or hardware-light settings, existing sampling-dependent 3D CGH methods preclude the ability to place target points with arbitrary precision, limiting accessible depths to a handful of planes. Accordingly, we present a non-iterative point cloud holography algorithm that employs fast deterministic calculations in order to efficiently allocate patches of SLM pixels to different target points in the 3D volume and spread the patterning of all points across multiple time frames. Compared to a matched-performance implementation of the iterative Gerchberg-Saxton algorithm, our algorithm's relative computation speed advantage was found to increase with SLM pixel count, reaching >100,000x at 512 × 512 array format.

DiffuserSpec: spectroscopy with Scotch tape.

Computational spectroscopy breaks the inherent one-to-one spatial-to-spectral pixel mapping of traditional spectrometers by multiplexing spectral data over a given sensor region. Most computational spectrometers require components that are complex to design, fabricate, or both. DiffuserSpec is a simple computational spectrometer that uses the inherent spectral dispersion of commercially available diffusers to generate speckle patterns that are unique to each wavelength. Using Scotch tape as a diffuser, we demonstrate narrowband and broadband spectral reconstructions with 2-nm spectral resolution over an 85-nm bandwidth in the near-infrared, limited only by the bandwidth of the calibration dataset. We also investigate the effect of spatial sub-sampling of the 2D speckle pattern on resolution performance.

Relationship Between CT Head Findings and Long-term Recovery in Children with Complicated Mild Traumatic Brain Injury.

PRIMARY OBJECTIVE: Complicated mild traumatic brain injury (C-mTBI) refers to CT positive patients with clinically mild TBI. This study investigates the association between CT head findings at time of injury and recovery of paediatric patients with C-mTBI. RESEARCH DESIGN: Retrospective survey and chart review. METHODS: For paediatric patients with C-mTBI (N = 77), CT findings associated with corresponding degree and lengths of recovery from C-mTBI using logistic regression analysis. RESULTS: There was a trend that the odds of incomplete recovery at the time of survey was higher for older children than for younger children (OR = 1.14, 95% CI = 0.98-1.32, p = 0.072). There was a trend that the odds of incomplete recovery (OR = 6.26, 95% CI = 0.97-40.57, p = 0.054) and longer duration for recovery (OR = 8.14, 95% CI = 0.78-84.46, p = 0.079) was higher for children with multiple haemorrhagic contusions than those with single haemorrhagic contusion. No other imaging patterns predicted degree or length of recovery with statistical significance (p > 0.05). CONCLUSIONS: Other than the presence of multiple haemorrhagic contusions, no other pattern of imaging abnormality in paediatric C-mTBI appears to be associated with degree or length of recovery. Further studies with larger cohorts are encouraged.

Geminin is required for Hox gene regulation to pattern the developing limb.

Development of the complex structure of the vertebrate limb requires carefully orchestrated interactions between multiple regulatory pathways and proteins. Among these, precise regulation of 5' Hox transcription factor expression is essential for proper limb bud patterning and elaboration of distinct limb skeletal elements. Here, we identified Geminin (Gmnn) as a novel regulator of this process. A conditional model of Gmnn deficiency resulted in loss or severe reduction of forelimb skeletal elements, while both the forelimb autopod and hindlimb were unaffected. 5' Hox gene expression expanded into more proximal and anterior regions of the embryonic forelimb buds in this Gmnn-deficient model. A second conditional model of Gmnn deficiency instead caused a similar but less severe reduction of hindlimb skeletal elements and hindlimb polydactyly, while not affecting the forelimb. An ectopic posterior SHH signaling center was evident in the anterior hindlimb bud of Gmnn-deficient embryos in this model. This center ectopically expressed Hoxd13, the HOXD13 target Shh, and the SHH target Ptch1, while these mutant hindlimb buds also had reduced levels of the cleaved, repressor form of GLI3, a SHH pathway antagonist. Together, this work delineates a new role for Gmnn in modulating Hox expression to pattern the vertebrate limb.

Untrained networks for compressive lensless photography.

Compressive lensless imagers enable novel applications in an extremely compact device, requiring only a phase or amplitude mask placed close to the sensor. They have been demonstrated for 2D and 3D microscopy, single-shot video, and single-shot hyperspectral imaging; in each case, a compressive-sensing-based inverse problem is solved in order to recover a 3D data-cube from a 2D measurement. Typically, this is accomplished using convex optimization and hand-picked priors. Alternatively, deep learning-based reconstruction methods offer the promise of better priors, but require many thousands of ground truth training pairs, which can be difficult or impossible to acquire. In this work, we propose an unsupervised approach based on untrained networks for compressive image recovery. Our approach does not require any labeled training data, but instead uses the measurement itself to update the network weights. We demonstrate our untrained approach on lensless compressive 2D imaging, single-shot high-speed video recovery using the camera's rolling shutter, and single-shot hyperspectral imaging. We provide simulation and experimental verification, showing that our method results in improved image quality over existing methods.

Fourier DiffuserScope: single-shot 3D Fourier light field microscopy with a diffuser.

Light field microscopy (LFM) uses a microlens array (MLA) near the sensor plane of a microscope to achieve single-shot 3D imaging of a sample without any moving parts. Unfortunately, the 3D capability of LFM comes with a significant loss of lateral resolution at the focal plane. Placing the MLA near the pupil plane of the microscope, instead of the image plane, can mitigate the artifacts and provide an efficient forward model, at the expense of field-of-view (FOV). Here, we demonstrate improved resolution across a large volume with Fourier DiffuserScope, which uses a diffuser in the pupil plane to encode 3D information, then computationally reconstructs the volume by solving a sparsity-constrained inverse problem. Our diffuser consists of randomly placed microlenses with varying focal lengths; the random positions provide a larger FOV compared to a conventional MLA, and the diverse focal lengths improve the axial depth range. To predict system performance based on diffuser parameters, we, for the first time, establish a theoretical framework and design guidelines, which are verified by numerical simulations, and then build an experimental system that achieves < 3 µm lateral and 4 µm axial resolution over a 1000 × 1000 × 280 µm3 volume. Our diffuser design outperforms the MLA used in LFM, providing more uniform resolution over a larger volume, both laterally and axially.

On-chip fluorescence microscopy with a random microlens diffuser.

We present an on-chip, widefield fluorescence microscope, which consists of a diffuser placed a few millimeters away from a traditional image sensor. The diffuser replaces the optics of a microscope, resulting in a compact and easy-to-assemble system with a practical working distance of over 1.5 mm. Furthermore, the diffuser encodes volumetric information, enabling refocusability in post-processing and three-dimensional (3D) imaging of sparse samples from a single acquisition. Reconstruction of images from the raw data requires a precise model of the system, so we introduce a practical calibration scheme and a physics-based forward model to efficiently account for the spatially-varying point spread function (PSF). To improve performance in low-light, we propose a random microlens diffuser, which consists of many small lenslets randomly placed on the mask surface and yields PSFs that are robust to noise. We build an experimental prototype and demonstrate our system on both planar and 3D samples.

Roadmap on 3D integral imaging: sensing, processing, and display.

This Roadmap article on three-dimensional integral imaging provides an overview of some of the research activities in the field of integral imaging. The article discusses various aspects of the field including sensing of 3D scenes, processing of captured information, and 3D display and visualization of information. The paper consists of a series of 15 sections from the experts presenting various aspects of the field on sensing, processing, displays, augmented reality, microscopy, object recognition, and other applications. Each section represents the vision of its author to describe the progress, potential, vision, and challenging issues in this field.

Learned reconstructions for practical mask-based lensless imaging.

Mask-based lensless imagers are smaller and lighter than traditional lensed cameras. In these imagers, the sensor does not directly record an image of the scene; rather, a computational algorithm reconstructs it. Typically, mask-based lensless imagers use a model-based reconstruction approach that suffers from long compute times and a heavy reliance on both system calibration and heuristically chosen denoisers. In this work, we address these limitations using a bounded-compute, trainable neural network to reconstruct the image. We leverage our knowledge of the physical system by unrolling a traditional model-based optimization algorithm, whose parameters we optimize using experimentally gathered ground-truth data. Optionally, images produced by the unrolled network are then fed into a jointly-trained denoiser. As compared to traditional methods, our architecture achieves better perceptual image quality and runs 20× faster, enabling interactive previewing of the scene. We explore a spectrum between model-based and deep learning methods, showing the benefits of using an intermediate approach. Finally, we test our network on images taken in the wild with a prototype mask-based camera, demonstrating that our network generalizes to natural images.

Quantitative differential phase contrast (DPC) microscopy with computational aberration correction.

Differential Phase Contrast (DPC) microscopy is a practical method for recovering quantitative phase from intensity images captured with different source patterns in an LED array microscope. Being a partially coherent imaging method, DPC does not suffer from speckle artifacts and achieves 2× better resolution than coherent methods. Like all imaging systems, however, DPC is susceptible to aberrations. Here, we propose a method of algorithmic self-calibration for DPC where we simultaneously recover the complex-field of the sample and the spatially-variant aberrations of the system, using 4 images with different illumination source patterns. The resulting phase reconstructions are digitally aberration-corrected.

Aberration recovery by imaging a weak diffuser.

We present a computational method for field-varying aberration recovery in optical systems by imaging a weak (index-matched) diffuser. Using multiple images acquired under plane wave illumination at distinct angles, the aberrations of the imaging system can be uniquely determined up to a sign. Our method is based on a statistical model for image formation that relates the spectrum of the speckled intensity image to the local aberrations at different locations in the field-of-view. The diffuser is treated as a wide-sense stationary scattering object, eliminating the need for precise knowledge of its surface shape. We validate our method both numerically and experimentally, showing that this relatively simple algorithmic calibration method can be reliably used to recover system aberrations quantitatively.

Efficient illumination angle self-calibration in Fourier ptychography.

Fourier ptychography captures intensity images with varying source patterns (illumination angles) in order to computationally reconstruct large space-bandwidth-product images. Accurate knowledge of the illumination angles is necessary for good image quality; hence, calibration methods are crucial, despite often being impractical or slow. Here, we propose a fast, robust, and accurate self-calibration algorithm that uses only experimentally collected data and general knowledge of the illumination setup. First, our algorithm makes a fast direct estimate of the brightfield illumination angles based on image processing. Then, a more computationally intensive spectral correlation method is used inside the iterative solver to further refine the angle estimates of both brightfield and darkfield images. We demonstrate our method for correcting large and small misalignment artifacts in 2D and 3D Fourier ptychography with different source types: an LED array, a galvo-steered laser, and a high-NA quasi-dome LED illuminator.

Multiplexed phase-space imaging for 3D fluorescence microscopy.

Optical phase-space functions describe spatial and angular information simultaneously; examples of optical phase-space functions include light fields in ray optics and Wigner functions in wave optics. Measurement of phase-space enables digital refocusing, aberration removal and 3D reconstruction. High-resolution capture of 4D phase-space datasets is, however, challenging. Previous scanning approaches are slow, light inefficient and do not achieve diffraction-limited resolution. Here, we propose a multiplexed method that solves these problems. We use a spatial light modulator (SLM) in the pupil plane of a microscope in order to sequentially pattern multiplexed coded apertures while capturing images in real space. Then, we reconstruct the 3D fluorescence distribution of our sample by solving an inverse problem via regularized least squares with a proximal accelerated gradient descent solver. We experimentally reconstruct a 101 Megavoxel 3D volume (1010×510×500µm with NA 0.4), demonstrating improved acquisition time, light throughput and resolution compared to scanning aperture methods. Our flexible patterning scheme further allows sparsity in the sample to be exploited for reduced data capture.

Video-rate processing in tomographic phase microscopy of biological cells using CUDA.

We suggest a new implementation for rapid reconstruction of three-dimensional (3-D) refractive index (RI) maps of biological cells acquired by tomographic phase microscopy (TPM). The TPM computational reconstruction process is extremely time consuming, making the analysis of large data sets unreasonably slow and the real-time 3-D visualization of the results impossible. Our implementation uses new phase extraction, phase unwrapping and Fourier slice algorithms, suitable for efficient CPU or GPU implementations. The experimental setup includes an external off-axis interferometric module connected to an inverted microscope illuminated coherently. We used single cell rotation by micro-manipulation to obtain interferometric projections from 73 viewing angles over a 180° angular range. Our parallel algorithms were implemented using Nvidia's CUDA C platform, running on Nvidia's Tesla K20c GPU. This implementation yields, for the first time to our knowledge, a 3-D reconstruction rate higher than video rate of 25 frames per second for 256 × 256-pixel interferograms with 73 different projection angles (64 × 64 × 64 output). This allows us to calculate additional cellular parameters, while still processing faster than video rate. This technique is expected to find uses for real-time 3-D cell visualization and processing, while yielding fast feedback for medical diagnosis and cell sorting.