Management of the axial modulation of the illumination pattern in structured illumination microscopy using an extended illumination source.

We have designed and implemented an approach for three-dimensional (3D) structured illumination (SI) microscopy (SIM) based on a quasi-monochromatic extended source illuminating a Wollaston prism to improve robustness, light efficiency and flexibility over our previous design. We show through analytical and experimental verification of the presented theoretical framework for our proposed tunable structured illumination microscopy (TSIM) system, that a simple and accurate determination of the axial modulation of the SI pattern is achieved, enabling a realistic characterization of the system's effective optical transfer function (OTF). System performance as a function of the extended source size is investigated with simulations. Results from a comparative performance analysis of the proposed TSIM system and traditional SIM systems show some advantages over the traditional two-wave and three-wave interference SIM systems. We show that by controlling the source size and thereby the axial modulation of the 3D SI pattern, the TSIM scheme offers increased OTF compact support and improved optical sectioning capability, quantified by the integrated intensity, under certain conditions, which may be desirable when imaging optically thick samples. The additional tunability of the 3D SI pattern, provides a unique opportunity for OTF engineering in our TSIM system.

Resolution limit in opto-digital systems revisited.

The resolution limit achievable with an optical system is a fundamental piece of information when characterizing its performance, mainly in case of microscopy imaging. Usually this information is given in the form of a distance, often expressed in microns, or in the form of a cutoff spatial frequency, often expressed in line pairs per mm. In modern imaging systems, where the final image is collected by pixelated digital cameras, the resolution limit is determined by the performance of both, the optical systems and the digital sensor. Usually, one of these factors is considered to be prevalent over the other for estimating the spatial resolution, leading to the global performance of the imaging system ruled by either the classical Abbe resolution limit, based on physical diffraction, or by the Nyquist resolution limit, based on the digital sensor features. This estimation fails significantly to predict the global performance of opto-digital imaging systems, like 3D microscopes, where none of the factors is negligible. In that case, which indeed is the most common, neither the Abbe formula nor the Nyquist formula provide by themselves a reliable prediction for the resolution limit. This is a serious drawback since systems designers often use those formulae as design input parameters. Aiming to overcome this lack, a simple mathematical expression obtained by finely articulating the Abbe and Nyquist formulas, to easily predict the spatial resolution limit of opto-digital imaging systems, is proposed here. The derived expression is tested experimentally, and shows to be valid in a broad range of opto-digital combinations.

Fourier lightfield microscopy: a practical design guide.

In this work, a practical guide for the design of a Fourier lightfield microscope is reported. The fundamentals of the Fourier lightfield are presented and condensed on a set of contour plots from which the user can select the design values of the spatial resolution, the field of view, and the depth of field, as function of the specifications of the hardware of the host microscope. This work guides the reader to select the parameters of the infinity-corrected microscope objective, the optical relay lenses, the aperture stop, the microlens array, and the digital camera. A user-friendly graphic calculator is included to ease the design, even to those who are not familiar with the lightfield technology. The guide is aimed to simplify the design process of a Fourier lightfield microscope, which sometimes could be a daunting task, and in this way, to invite the widespread use of this technology. An example of a design and experimental results on imaging different types of samples is also presented.

Handheld and Cost-Effective Fourier Lightfield Microscope.

In this work, the design, building, and testing of the most portable, easy-to-build, robust, handheld, and cost-effective Fourier Lightfield Microscope (FLMic) to date is reported. The FLMic is built by means of a surveillance camera lens and additional off-the-shelf optical elements, resulting in a cost-effective FLMic exhibiting all the regular sought features in lightfield microscopy, such as refocusing and gathering 3D information of samples by means of a single-shot approach. The proposed FLMic features reduced dimensions and light weight, which, combined with its low cost, turn the presented FLMic into a strong candidate for in-field application where 3D imaging capabilities are pursued. The use of cost-effective optical elements has a relatively low impact on the optical performance, regarding the figures dictated by the theory, while its price can be at least 100 times lower than that of a regular FLMic. The system operability is tested in both bright-field and fluorescent modes by imaging a resolution target, a honeybee wing, and a knot of dyed cotton fibers.

Roadmap on digital holography [Invited].

This Roadmap article on digital holography provides an overview of a vast array of research activities in the field of digital holography. The paper consists of a series of 25 sections from the prominent experts in digital holography presenting various aspects of the field on sensing, 3D imaging and displays, virtual and augmented reality, microscopy, cell identification, tomography, label-free live cell imaging, and other applications. Each section represents the vision of its author to describe the significant progress, potential impact, important developments, and challenging issues in the field of digital holography.

The Lightfield Microscope Eyepiece.

Lightfield microscopy has raised growing interest in the last few years. Its ability to get three-dimensional information about the sample in a single shot makes it suitable for many applications in which time resolution is fundamental. In this paper we present a novel device, which is capable of converting any conventional microscope into a lightfield microscope. Based on the Fourier integral microscope concept, we designed the lightfield microscope eyepiece. This is coupled to the eyepiece port, to let the user exploit all the host microscope's components (objective turret, illumination systems, translation stage, etc.) and get a 3D reconstruction of the sample. After the optical design, a proof-of-concept device was built with off-the-shelf optomechanical components. Here, its optical performances are demonstrated, which show good matching with the theoretical ones. Then, the pictures of different samples taken with the lightfield eyepiece are shown, along with the corresponding reconstructions. We demonstrated the functioning of the lightfield eyepiece and lay the foundation for the development of a commercial device that works with any microscope.

Three-dimensional real-time darkfield imaging through Fourier lightfield microscopy.

We report a protocol that takes advantage of the Fourier lightfield microscopy concept for providing 3D darkfield images of volumetric samples in a single-shot. This microscope takes advantage of the Fourier lightfield configuration, in which a lens array is placed at the Fourier plane of the microscope objective, providing a direct multiplexing of the spatio-angular information of the sample. Using the proper illumination beam, the system collects the light scattered by the sample while the background light is blocked out. This produces a set of orthographic perspective images with shifted spatial-frequency components that can be recombined to produce a 3D darkfield image. Applying the adequate reconstruction algorithm high-contrast darkfield optical sections are calculated in real time. The presented method is applied for fast volumetric reconstructions of unstained 3D samples.

What about computational super-resolution in fluorescence Fourier light field microscopy?

Recently, Fourier light field microscopy was proposed to overcome the limitations in conventional light field microscopy by placing a micro-lens array at the aperture stop of the microscope objective instead of the image plane. In this way, a collection of orthographic views from different perspectives are directly captured. When inspecting fluorescent samples, the sensitivity and noise of the sensors are a major concern and large sensor pixels are required to cope with low-light conditions, which implies under-sampling issues. In this context, we analyze the sampling patterns in Fourier light field microscopy to understand to what extent computational super-resolution can be triggered during deconvolution in order to improve the resolution of the 3D reconstruction of the imaged data.

Fast and robust phase-shift estimation in two-dimensional structured illumination microscopy.

A method of determining unknown phase-shifts between elementary images in two-dimensional Structured Illumination Microscopy (2D-SIM) is presented. The proposed method is based on the comparison of the peak intensity of spectral components. These components correspond to the inherent structured illumination spectral content and the residual component that appears from wrongly estimated phase-shifts. The estimation of the phase-shifts is carried out by finding the absolute maximum of a function defined as the normalized peak intensity difference in the Fourier domain. This task is performed by an optimization method providing a fast estimation of the phase-shift. The algorithm stability and robustness are tested for various levels of noise and contrasts of the structured illumination pattern. Furthermore, the proposed approach reduces the number of computations compared to other existing techniques. The method is supported by the theoretical calculations and validated by means of simulated and experimental results.

Optical transfer function engineering for a tunable 3D structured illumination microscope.

Two important features of three-dimensional structured illumination microscopy (3D-SIM) are its optical sectioning (OS) and super-resolution (SR) capabilities. Previous works on 3D-SIM systems show that these features are coupled. We demonstrate that a 3D-SIM system using a Fresnel biprism illuminated by multiple linear incoherent sources provides a structured illumination pattern whose lateral and axial modulation frequencies can be tuned separately. Therefore, the compact support of the synthetic optical transfer function (OTF) can be engineered to achieve the highest OS and SR capabilities for a particular imaging application. Theoretical performance of our engineered system based on the OTF support is compared to that achieved by other well-known SIM systems.

Robust Depth Estimation for Light Field Microscopy.

Light field technologies have seen a rise in recent years and microscopy is a field where such technology has had a deep impact. The possibility to provide spatial and angular information at the same time and in a single shot brings several advantages and allows for new applications. A common goal in these applications is the calculation of a depth map to reconstruct the three-dimensional geometry of the scene. Many approaches are applicable, but most of them cannot achieve high accuracy because of the nature of such images: biological samples are usually poor in features and do not exhibit sharp colors like natural scene. Due to such conditions, standard approaches result in noisy depth maps. In this work, a robust approach is proposed where accurate depth maps can be produced exploiting the information recorded in the light field, in particular, images produced with Fourier integral Microscope. The proposed approach can be divided into three main parts. Initially, it creates two cost volumes using different focal cues, namely correspondences and defocus. Secondly, it applies filtering methods that exploit multi-scale and super-pixels cost aggregation to reduce noise and enhance the accuracy. Finally, it merges the two cost volumes and extracts a depth map through multi-label optimization.

Tunable-frequency three-dimensional structured illumination microscopy with reduced data-acquisition.

The performance of a tunable three-dimensional (3D) structured illumination microscope (SIM) system and its ability to provide simultaneously super-resolution (SR) and optical-sectioning (OS) capabilities are investigated. Numerical results show that the performance of our 3D-SIM system is comparable with the one provided by a three-wave interference SIM, while requiring 40% fewer images for the reconstruction and providing frequency tunability in a cost-effective implementation. The performance of the system has been validated experimentally with images from test samples, which were also imaged with a commercial SIM based on incoherent-grid projection for comparison. Restored images from data acquired from an axially-thin fluorescent layer show a 1.6× improvement in OS capability compared to the commercial instrument while results from a fluorescent tilted USAF target show the OS and SR capabilities achieved by our system.

Three-dimensional imaging system with both improved lateral resolution and depth of field considering non-uniform system parameters.

In this paper, we propose a new 3D passive image sensing and visualization technique to improve lateral resolution and depth of field (DoF) of integral imaging simultaneously. There is a resolution trade-off between lateral resolution and DoF in integral imaging. To overcome this issue, a large aperture and a small aperture can be used to record the elemental images to reduce the diffraction effect and extend the DoF, respectively. Therefore, in this paper, we utilize these two pickup concepts with a non-uniform camera array. To show the feasibility of our proposed method, we implement an optical experiment. For comparison in details, we calculate the peak signal-to-noise ratio (PSNR) as the performance metric.

Large Depth-of-Field Integral Microscopy by Use of a Liquid Lens.

Integral microscopy is a 3D imaging technique that permits the recording of spatial and angular information of microscopic samples. From this information it is possible to calculate a collection of orthographic views with full parallax and to refocus computationally, at will, through the 3D specimen. An important drawback of integral microscopy, especially when dealing with thick samples, is the limited depth of field (DOF) of the perspective views. This imposes a significant limitation on the depth range of computationally refocused images. To overcome this problem, we propose here a new method that is based on the insertion, at the pupil plane of the microscope objective, of an electrically controlled liquid lens (LL) whose optical power can be changed by simply tuning the voltage. This new apparatus has the advantage of controlling the axial position of the objective focal plane while keeping constant the essential parameters of the integral microscope, that is, the magnification, the numerical aperture and the amount of parallax. Thus, given a 3D sample, the new microscope can provide a stack of integral images with complementary depth ranges. The fusion of the set of refocused images permits to enlarge the reconstruction range, obtaining images in focus over the whole region.

New Method of Microimages Generation for 3D Display.

In this paper, we propose a new method for the generation of microimages, which processes real 3D scenes captured with any method that permits the extraction of its depth information. The depth map of the scene, together with its color information, is used to create a point cloud. A set of elemental images of this point cloud is captured synthetically and from it the microimages are computed. The main feature of this method is that the reference plane of displayed images can be set at will, while the empty pixels are avoided. Another advantage of the method is that the center point of displayed images and also their scale and field of view can be set. To show the final results, a 3D InI display prototype is implemented through a tablet and a microlens array. We demonstrate that this new technique overcomes the drawbacks of previous similar ones and provides more flexibility setting the characteristics of the final image.

Improvement of two-dimensional structured illumination microscopy with an incoherent illumination pattern of tunable frequency.

In two-dimensional structured illumination microscopy (2D-SIM), high-resolution images with optimal optical sectioning (OS) cannot be obtained simultaneously. This tradeoff can be overcome by using a tunable-frequency 2D-SIM system and a proper reconstruction method. The goal of this work is twofold. First, we present a computational approach to reconstruct optical-sectioned images with super-resolution enhancement (OS-SR) by using a tunable SIM system. Second, we propose an incoherent tunable-frequency 2D-SIM system based on a Fresnel biprism implementation. Integration of the proposed computational method with this tunable structured illumination (SI) system results in a new 2D-SIM system that is advantageous compared to other 2D-SIM systems with comparable complexity, because it provides high-resolution OS images independent of the objective lens used, without the presence of coherent noise and without reducing the contrast of the structured pattern, as in other incoherent implementations. Evaluation of our proposed system is demonstrated with comparative studies of simulated and experimental reconstructed images to validate our theoretical findings. Our experimental results show a simultaneous improvement of the lateral resolution by a factor of 1.8× with the desired OS capability achieved in the resulting OS-SR combination image. Our experimental results also verify that our system can provide better OS capability than the commercial Zeiss ApoTome-SIM system in the investigated study.

Experimental validation of a customized phase mask designed to enable efficient computational optical sectioning microscopy through wavefront encoding.

In this paper, wavefront-encoded (WFE) computational optical sectioning microscopy (COSM) using a fabricated square cubic (SQUBIC) phase mask, designed to render the system less sensitive to depth-induced aberration, is investigated. The WFE-COSM system is characterized by a point spread function (PSF) that does not vary as rapidly with imaging depth compared to the conventional system. Thus, in WFE-COSM, image restoration from large volumes can be achieved using computationally efficient space-invariant (SI) algorithms, thereby avoiding the use of depth-variant algorithms. The fabricated SQUBIC phase mask was first evaluated and found to have a 75% fidelity compared to the theoretical design; it was then integrated in a commercial wide-field microscope to implement a WFE-COSM system. Evaluation of the WFE-COSM system is demonstrated with comparative studies of theoretical and experimental PSFs and simulated and measured images of spherical shells located at different depths in a test sample. These comparisons show that PSF and imaging models capture major trends in experimental data with a 99% correlation between forward image intensity distribution in experimental and simulated images of spherical shells. Our experimental SI restoration results demonstrate that the WFE-COSM system achieves more than a twofold performance improvement over the conventional system of up to a 65 µm depth below the coverslip investigated in this study.

Reducing effects of aberration in 3D fluorescence imaging using wavefront coding with a radially symmetric phase mask.

In this work, a wavefront encoded (WFE) imaging system built using a squared cubic phase mask, designed to reduce the sensitivity of the imaging system to spherical aberration, is investigated. The proposed system allows the use of a space-invariant image restoration algorithm, which uses a single PSF, to restore intensity distribution in images suffering aberration, such as sample-induced aberration in thick tissue. This provides a computational advantage over depth-variant image restoration algorithms developed previously to address this aberration. Simulated PSFs of the proposed system are shown to change up to 25% compared to the 0 µm depth PSF (quantified by the structural similarity index) over a 100 µm depth range, while the conventional system PSFs change up to 84%. Results from experimental test-sample images show that restoration error is reduced by 29% when the proposed WFE system is used instead of the conventional system over a 30 µm depth range.

Phase-shifting by means of an electronically tunable lens: quantitative phase imaging of biological specimens with digital holographic microscopy.

The use of an electronically tunable lens (ETL) to produce controlled phase shifts in interferometric arrangements is shown. The performance of the ETL as a phase-shifting device is experimentally validated in phase-shifting digital holographic microscopy. Quantitative phase maps of a section of the thorax of a Drosophila melanogaster fly and of human red blood cells have been obtained using our proposal. The experimental results validate the possibility of using the ETL as a reliable phase-shifter device.

Mapping electron-beam-injected trapped charge with scattering scanning near-field optical microscopy.

Scattering scanning near-field optical microscopy (s-SNOM) has been demonstrated as a valuable tool for mapping the optical and optoelectronic properties of materials with nanoscale resolution. Here we report experimental evidence that trapped electric charges injected by an electron beam at the surface of dielectric samples affect the sample-dipole interaction, which has direct impact on the s-SNOM image content. Nanoscale mapping of the surface trapped charge holds significant potential for the precise tailoring of the electrostatic properties of dielectric and semiconductive samples, such as hydroxyapatite, which has particular importance with respect to biomedical applications. The methodology developed here is highly relevant to semiconductor device fabrication as well.