Compressive hyperspectral microscopy for cancer detection.

Significance: Hyperspectral microscopy grants the ability to characterize unique properties of tissues based on their spectral fingerprint. The ability to label and measure multiple molecular probes simultaneously provides pathologists and oncologists with a powerful tool to enhance accurate diagnostic and prognostic decisions. As the pathological workload grows, having an objective tool that provides companion diagnostics is of immense importance. Therefore, fast whole-slide spectral imaging systems are of immense importance for automated cancer prognostics that meet current and future needs. Aim: We aim to develop a fast and accurate hyperspectral microscopy system that can be easily integrated with existing microscopes and provide flexibility for optimizing measurement time versus spectral resolution. Approach: The method employs compressive sensing (CS) and a spectrally encoded illumination device integrated into the illumination path of a standard microscope. The spectral encoding is obtained using a compact liquid crystal cell that is operated in a fast mode. It provides time-efficient measurements of the spectral information, is modular and versatile, and can also be used for other applications that require rapid acquisition of hyperspectral images. Results: We demonstrated the acquisition of breast cancer biopsies hyperspectral data of the whole camera area within ∼1 s. This means that a typical 1×1 cm2 biopsy can be measured in ∼10 min. The hyperspectral images with 250 spectral bands are reconstructed from 47 spectrally encoded images in the spectral range of 450 to 700 nm. Conclusions: CS hyperspectral microscopy was successfully demonstrated on a common lab microscope for measuring biopsies stained with the most common stains, such as hematoxylin and eosin. The high spectral resolution demonstrated here in a rather short time indicates the ability to use it further for coping with the highly demanding needs of pathological diagnostics, both for cancer diagnostics and prognostics.

Focus Issue Introduction: 3D Image Acquisition and Display: Technology, Perception and Applications.

This Feature Issue of Optics Express is organized in conjunction with the 2022 Optica conference on 3D Image Acquisition and Display: Technology, Perception and Applications which was held in hybrid format from 11 to 15, July 2022 as part of the Imaging and Applied Optics Congress and Optical Sensors and Sensing Congress 2022 in Vancouver, Canada. This Feature Issue presents 31 articles which cover the topics and scope of the 2022 3D Image Acquisition and Display conference. This Introduction provides a summary of these published articles that appear in this Feature Issue.

Progressive compressive sensing of large images with multiscale deep learning reconstruction.

Compressive sensing (CS) is a sub-Nyquist sampling framework that has been employed to improve the performance of numerous imaging applications during the last 15 years. Yet, its application for large and high-resolution imaging remains challenging in terms of the computation and acquisition effort involved. Often, low-resolution imaging is sufficient for most of the considered tasks and only a fraction of cases demand high resolution, but the problem is that the user does not know in advance when high-resolution acquisition is required. To address this, we propose a multiscale progressive CS method for the high-resolution imaging. The progressive sampling refines the resolution of the image, while incorporating the already sampled low-resolution information, making the process highly efficient. Moreover, the multiscale property of the progressively sensed samples is capitalized for a fast, deep learning (DL) reconstruction, otherwise infeasible due to practical limitations of training on high-resolution images. The progressive CS and the multiscale reconstruction method are analyzed numerically and demonstrated experimentally with a single pixel camera imaging system. We demonstrate 4-megapixel size progressive compressive imaging with about half the overall number of samples, more than an order of magnitude faster reconstruction, and improved reconstruction quality compared to alternative conventional CS approaches.

Object localization and tracking in three dimensions by space-to-time encoding.

In this paper, we present a novel method for measuring the location and estimating the dynamics of fast-moving small objects in free space. The proposed 3D localization method is realized by a space-to-time optical transform and measurement of time-of-flight. We present the underlying physical and mathematical model of the method and provide an example based on a simple configuration. In the simplest mode, the method is implemented by two plane mirrors, a spherical light pulse illuminator, and a single fast response photodetector. The 3D spatial information is retrieved from the temporal measurements by solving an inverse problem that uses a sparse approximation of the scene. System simulation shows the ability to track fast small objects that are moving in space using only a single time-resolved detector.

Focus issue introduction: 3D image acquisition and display: technology, perception and applications.

This Feature Issue of Optics Express is organized in conjunction with the 2021 Optica (OSA) conference on 3D Image Acquisition and Display: Technology, Perception and Applications which was held virtually from 19 to 23, July 2021 as part of the Imaging and Sensing Congress 2021. This Feature Issue presents 29 articles which cover the topics and scope of the 2021 3D conference. This Introduction provides a summary of these articles.

Roadmap on digital holography [Invited].

This Roadmap article on digital holography provides an overview of a vast array of research activities in the field of digital holography. The paper consists of a series of 25 sections from the prominent experts in digital holography presenting various aspects of the field on sensing, 3D imaging and displays, virtual and augmented reality, microscopy, cell identification, tomography, label-free live cell imaging, and other applications. Each section represents the vision of its author to describe the significant progress, potential impact, important developments, and challenging issues in the field of digital holography.

A Simplified Model for Optical Systems with Random Phase Screens.

A first-order optical system with arbitrary multiple masks placed at arbitrary positions is the basic scheme of various optical systems. Generally, masks in optical systems have a non-shift invariant (SI) effect; thus, the individual effect of each mask on the output cannot be entirely separated. The goal of this paper is to develop a technique where complete separation might be achieved in the common case of random phase screens (RPSs) as masks. RPSs are commonly used to model light propagation through the atmosphere or through biological tissues. We demonstrate the utility of the technique on an optical system with multiple RPSs that model random scattering media.

Compressive imaging for defending deep neural networks from adversarial attacks.

Despite their outstanding performance, convolutional deep neural networks (DNNs) are vulnerable to small adversarial perturbations. In this Letter, we introduce a novel approach to thwart adversarial attacks. We propose to employ compressive sensing (CS) to defend DNNs from adversarial attacks, and at the same time to encode the image, thus preventing counterattacks. We present computer simulations and optical experimental results of object classification in adversarial images captured with a CS single pixel camera.

Roadmap on 3D integral imaging: sensing, processing, and display.

This Roadmap article on three-dimensional integral imaging provides an overview of some of the research activities in the field of integral imaging. The article discusses various aspects of the field including sensing of 3D scenes, processing of captured information, and 3D display and visualization of information. The paper consists of a series of 15 sections from the experts presenting various aspects of the field on sensing, processing, displays, augmented reality, microscopy, object recognition, and other applications. Each section represents the vision of its author to describe the progress, potential, vision, and challenging issues in this field.

Dual-camera design for hyperspectral and panchromatic imaging, using a wedge shaped liquid crystal as a spectral multiplexer.

In this paper, we present a new hyperspectral compact camera which is designed to have high spatial and spectral resolutions, to be vibrations tolerant, and to achieve state-of-the-art high optical throughput values compared to existing nanosatellite hyperspectral imaging payloads with space heritage. These properties make it perfect for airborne and spaceborne remote sensing tasks. The camera has both hyperspectral and panchromatic imaging capabilities, achieved by employing a wedge-shaped liquid crystal cell together with computational image processing. The hyperspectral images are acquired through passive along-track spatial scanning when no voltage is applied to the cell, and the panchromatic images are quickly acquired in a single snapshot at a high signal-to-noise ratio when the cell is voltage driven.

DeepCubeNet: reconstruction of spectrally compressive sensed hyperspectral images with deep neural networks.

Several hyperspectral (HS) systems based on compressive sensing (CS) theory have been presented to capture HS images with high accuracy and with a lower number of measurements than needed by conventional systems. However, the reconstruction of HS compressed measurements is time-consuming and commonly involves hyperparameter tuning per each scenario. In this paper, we introduce a Convolutional Neural Network (CNN) designed for the reconstruction of HS cubes captured with CS imagers based on spectral modulation. Our Deep Neural Network (DNN), dubbed DeepCubeNet, provides significant reduction in the reconstruction time compared to classical iterative methods. The performance of DeepCubeNet is investigated on simulated data, and we demonstrate for the first time, to the best of our knowledge, real reconstruction of CS HS measurements using DNN. We demonstrate significantly enhanced reconstruction accuracy compared to iterative CS reconstruction, as well as improvement in reconstruction time by many orders of magnitude.

Compressive ultraspectral imaging using multiscale structured illumination.

We present a novel compressive spectral imaging technique that attains spatially resolved ultraspectral resolution. The technique employs a multiscale sampling technique based on the Hadamard basis for the single pixel hyperspectral imager. The proposed multiscale sampling method offers high-quality images at a low compression ratio while also facilitating a preview image at a lower resolution by using the fast Hadamard transform.

Fast Method for Liquid Crystal Cell Spatial Variations Estimation Based on Modeling the Spectral Transmission.

Liquid crystal phase retarders are utilized by photonic devices and imaging systems for various applications, such as tunable filtering, light modulation, polarimetric imaging, remote sensing and quality inspection. Due to technical difficulties in the manufacturing process, these phase retarders may suffer from spatial non-uniformities, which degrade the performance of the systems. These non-uniformities can be characterized by measuring the spectral transmission at each voltage and each point on the liquid crystal cell, which is time consuming. In this work, we present a new fast and simple method for measuring and computationally estimating the spatial variations of a liquid crystal phase retarder with planar alignment. The method is based on measuring the spectral transmission of the phase retarder at several spatial locations and estimating it at others. The experimental results show that the method provides an accurate spatial description of the phase retarder and can be employed for calibrating relevant systems.

Modeling optical memory effects with phase screens.

During the last decade, optical memory effects have been explored extensively for various applications. In this letter we propose phase screen models to facilitate the analysis and the simulation of wave propagation through optical media that exhibits memory effects. We show that the classical optical memory effect, which implies tilt wave correlations of the input and the scattered fields, can be readily modeled by a single random phase screen. For the recently discovered generalized optical memory effect, which implies the existence of shift wave correlations in addition to the tilt correlation, we propose an appropriate generalized random phase screen model.

Multi-aperture snapshot compressive hyperspectral camera.

In this Letter, we present a new snapshot hyperspectral (HS) camera based on a multi-aperture design. The technique uses an array of modified Fabry-Perot resonators together with a lens array in order to acquire an array of spectrally multiplexed modulated sub-images. Then the original HS image is reconstructed using a compressive sensing reconstruction algorithm. The HS camera has high optical throughput and enables acquisition of almost gigapixel HS datacubes with hundreds of spectral bands. Using our camera, we demonstrate optical compression of approximately 37:1.

Compressive Sensing Hyperspectral Imaging by Spectral Multiplexing with Liquid Crystal.

Hyperspectral (HS) imaging involves the sensing of a scene's spectral properties, which are often redundant in nature. The redundancy of the information motivates our quest to implement Compressive Sensing (CS) theory for HS imaging. This article provides a review of the Compressive Sensing Miniature Ultra-Spectral Imaging (CS-MUSI) camera, its evolution, and its different applications. The CS-MUSI camera was designed within the CS framework and uses a liquid crystal (LC) phase retarder in order to modulate the spectral domain. The outstanding advantage of the CS-MUSI camera is that the entire HS image is captured from an order of magnitude fewer measurements of the sensor array, compared to conventional HS imaging methods.

Evaluation of the influence of arbitrary masks on the output field of optical systems using ABCD matrices.

We propose a method to express the complex field amplitude (CFA) at the output of any optical system with arbitrary masks. The method provides a general equation for any first-order optical system with given parameters and arbitrary masks. The method is based on the ABCD matrices of the system components, ignoring the masks. The obtained output CFA is formulated in a way that helps recognize the influence of each mask on the output CFA by projecting its influence on the output with the aid of ABCD matrices. The method offers a more transparent approach for efficient organization and minimization of the calculation steps.

Compressive sensing resonator spectroscopy.

We present a new fast compressive spectroscopic technique based on the resonance spectrometric mechanism. This technique uses an appropriately designed Fabry-Perot resonator and a photo-sensor in order to acquire different multiplexed spectral modulations, from which the original signal is reconstructed using a compressive sensing reconstruction algorithm. We present experimental results that demonstrate the acquisition of hundreds of spectral bands with a compression ratio of about 1:13.

Compressive 4D spectro-volumetric imaging.

In this Letter, we present a method for hyperspectral imaging of three-dimensional objects. A compressive sensing approach is utilized to remedy the acquisition effort required to capture the large amount of data. The spectral dimension is compressively sensed by means of a liquid crystal-based encoder, and the volumetric data are captured using a synthetic aperture integral imaging setup. We demonstrate reconstruction of spectro-volumetric tesseracts with hundreds of spectral bands at different depths without compromise of spatial resolution.

Along-track scanning using a liquid crystal compressive hyperspectral imager.

In various applications, such as remote sensing and quality inspection, hyperspectral (HS) imaging is performed by spatially scanning an object. In this work, we present a new compressive hyperspectral imaging method that performs along-track scanning. The method relies on the compressive sensing miniature ultra-spectral imaging (CS-MUSI) system, which uses a single liquid crystal (LC) cell for spectral encoding and provides a more efficient way of HS data acquisition, compared to classical spatial scanning based systems. The experimental results show that a compression ratio of about 1:10 can be reached. Owing to the inherent compression, the captured data is preprepared for efficient storage and transmission.