Greedy Ensemble Hyperspectral Anomaly Detection.

Hyperspectral images include information from a wide range of spectral bands deemed valuable for computer vision applications in various domains such as agriculture, surveillance, and reconnaissance. Anomaly detection in hyperspectral images has proven to be a crucial component of change and abnormality identification, enabling improved decision-making across various applications. These abnormalities/anomalies can be detected using background estimation techniques that do not require the prior knowledge of outliers. However, each hyperspectral anomaly detection (HS-AD) algorithm models the background differently. These different assumptions may fail to consider all the background constraints in various scenarios. We have developed a new approach called Greedy Ensemble Anomaly Detection (GE-AD) to address this shortcoming. It includes a greedy search algorithm to systematically determine the suitable base models from HS-AD algorithms and hyperspectral unmixing for the first stage of a stacking ensemble and employs a supervised classifier in the second stage of a stacking ensemble. It helps researchers with limited knowledge of the suitability of the HS-AD algorithms for the application scenarios to select the best methods automatically. Our evaluation shows that the proposed method achieves a higher average F1-macro score with statistical significance compared to the other individual methods used in the ensemble. This is validated on multiple datasets, including the Airport-Beach-Urban (ABU) dataset, the San Diego dataset, the Salinas dataset, the Hydice Urban dataset, and the Arizona dataset. The evaluation using the airport scenes from the ABU dataset shows that GE-AD achieves a 14.97% higher average F1-macro score than our previous method (HUE-AD), at least 17.19% higher than the individual methods used in the ensemble, and at least 28.53% higher than the other state-of-the-art ensemble anomaly detection algorithms. As using the combination of greedy algorithm and stacking ensemble to automatically select suitable base models and associated weights have not been widely explored in hyperspectral anomaly detection, we believe that our work will expand the knowledge in this research area and contribute to the wider application of this approach.

Management of the axial modulation of the illumination pattern in structured illumination microscopy using an extended illumination source.

We have designed and implemented an approach for three-dimensional (3D) structured illumination (SI) microscopy (SIM) based on a quasi-monochromatic extended source illuminating a Wollaston prism to improve robustness, light efficiency and flexibility over our previous design. We show through analytical and experimental verification of the presented theoretical framework for our proposed tunable structured illumination microscopy (TSIM) system, that a simple and accurate determination of the axial modulation of the SI pattern is achieved, enabling a realistic characterization of the system's effective optical transfer function (OTF). System performance as a function of the extended source size is investigated with simulations. Results from a comparative performance analysis of the proposed TSIM system and traditional SIM systems show some advantages over the traditional two-wave and three-wave interference SIM systems. We show that by controlling the source size and thereby the axial modulation of the 3D SI pattern, the TSIM scheme offers increased OTF compact support and improved optical sectioning capability, quantified by the integrated intensity, under certain conditions, which may be desirable when imaging optically thick samples. The additional tunability of the 3D SI pattern, provides a unique opportunity for OTF engineering in our TSIM system.

Artificial Intelligence and Machine Learning in Optical Information Processing: introduction to the feature issue.

This special feature issue covers the intersection of topical areas in artificial intelligence (AI)/machine learning (ML) and optics. The papers broadly span the current state-of-the-art advances in areas including image recognition, signal and image processing, machine inspection/vision and automotive as well as areas of traditional optical sensing, interferometry and imaging.

Improved resolution in 3D structured illumination microscopy using 3D model-based restoration with positivity-constraint.

The performance of structured illumination microscopy (SIM) systems depends on the computational method used to process the raw data. In this paper, we present a regularized three-dimensional (3D) model-based (MB) restoration method with positivity constraint (PC) for 3D processing of data from 3D-SIM (or 3-beam interference SIM), in which the structured illumination pattern varies laterally and axially. The proposed 3D-MBPC method introduces positivity in the solution through the reconstruction of an auxiliary function using a conjugate-gradient method that minimizes the mean squared error between the data and the 3D imaging model. The 3D-MBPC method provides axial super resolution, which is not the same as improved optical sectioning demonstrated with model-based approaches based on the 2D-SIM (or 2-beam interference SIM) imaging model, for either 2D or 3D processing of a single plane from a 3D-SIM dataset. Results obtained with our 3D-MBPC method show improved 3D resolution over what is achieved by the standard generalized Wiener filter method, the first known method that performs 3D processing of 3D-SIM data. Noisy simulation results quantify the achieved 3D resolution, which is shown to match theoretical predictions. Experimental verification of the 3D-MBPC method with biological data demonstrates successful application to data volumes of different sizes.

Optical transfer function engineering for a tunable 3D structured illumination microscope.

Two important features of three-dimensional structured illumination microscopy (3D-SIM) are its optical sectioning (OS) and super-resolution (SR) capabilities. Previous works on 3D-SIM systems show that these features are coupled. We demonstrate that a 3D-SIM system using a Fresnel biprism illuminated by multiple linear incoherent sources provides a structured illumination pattern whose lateral and axial modulation frequencies can be tuned separately. Therefore, the compact support of the synthetic optical transfer function (OTF) can be engineered to achieve the highest OS and SR capabilities for a particular imaging application. Theoretical performance of our engineered system based on the OTF support is compared to that achieved by other well-known SIM systems.

Wollaston prism-based structured illumination microscope with tunable frequency.

This work proposes an alternative structured illumination (SI) system based on a Wollaston prism (WP) illuminated by the diffracted field of an incoherent linear source. The proposed WP-based SI system presents several advantages. First, the generated fringes can be approximated by a pure sinusoidal pattern; thereby, computational methods developed for sinusoidal SI can be used without the need for additional processing. Second, the SI pattern's period can be continuously varied up to the cutoff frequency of the native widefield system. Most significantly, the phase shifting of the SI pattern required for demodulation in SI microscopy can be easily accomplished using a de Sénarmont compensator, which provides accurate lateral displacement of the fringes independently of the lateral modulation frequency. Experimental verifications confirm the presented theoretical predictions.

Tunable-frequency three-dimensional structured illumination microscopy with reduced data-acquisition.

The performance of a tunable three-dimensional (3D) structured illumination microscope (SIM) system and its ability to provide simultaneously super-resolution (SR) and optical-sectioning (OS) capabilities are investigated. Numerical results show that the performance of our 3D-SIM system is comparable with the one provided by a three-wave interference SIM, while requiring 40% fewer images for the reconstruction and providing frequency tunability in a cost-effective implementation. The performance of the system has been validated experimentally with images from test samples, which were also imaged with a commercial SIM based on incoherent-grid projection for comparison. Restored images from data acquired from an axially-thin fluorescent layer show a 1.6× improvement in OS capability compared to the commercial instrument while results from a fluorescent tilted USAF target show the OS and SR capabilities achieved by our system.

Image restoration approach to address reduced modulation contrast in structured illumination microscopy.

The performance of structured illumination microscopy (SIM) is hampered in many biological applications due to the inability to modulate the light when imaging deep into the sample. This is in part because sample-induced aberration reduces the modulation contrast of the structured pattern. In this paper, we present an image restoration approach suitable for processing raw incoherent-grid-projection SIM data with a low fringe contrast. Restoration results from simulated and experimental ApoTome SIM data show results with improved signal-to-noise ratio (SNR) and optical sectioning compared to the results obtained from existing methods, such as 2D demodulation and 3D SIM deconvolution. Our proposed method provides satisfactory results (quantified by the achieved SNR and normalized mean square error) even when the modulation contrast of the illumination pattern is as low as 7%.

Improvement of two-dimensional structured illumination microscopy with an incoherent illumination pattern of tunable frequency.

In two-dimensional structured illumination microscopy (2D-SIM), high-resolution images with optimal optical sectioning (OS) cannot be obtained simultaneously. This tradeoff can be overcome by using a tunable-frequency 2D-SIM system and a proper reconstruction method. The goal of this work is twofold. First, we present a computational approach to reconstruct optical-sectioned images with super-resolution enhancement (OS-SR) by using a tunable SIM system. Second, we propose an incoherent tunable-frequency 2D-SIM system based on a Fresnel biprism implementation. Integration of the proposed computational method with this tunable structured illumination (SI) system results in a new 2D-SIM system that is advantageous compared to other 2D-SIM systems with comparable complexity, because it provides high-resolution OS images independent of the objective lens used, without the presence of coherent noise and without reducing the contrast of the structured pattern, as in other incoherent implementations. Evaluation of our proposed system is demonstrated with comparative studies of simulated and experimental reconstructed images to validate our theoretical findings. Our experimental results show a simultaneous improvement of the lateral resolution by a factor of 1.8× with the desired OS capability achieved in the resulting OS-SR combination image. Our experimental results also verify that our system can provide better OS capability than the commercial Zeiss ApoTome-SIM system in the investigated study.

Performance evaluation of extended depth of field microscopy in the presence of spherical aberration and noise.

Effectiveness of extended depth of field microscopy (EDFM) implementation with wavefront encoding methods is reduced by depth-induced spherical aberration (SA) due to reliance of this approach on a defined point spread function (PSF). Evaluation of the engineered PSF's robustness to SA, when a specific phase mask design is used, is presented in terms of the final restored image quality. Synthetic intermediate images were generated using selected generalized cubic and cubic phase mask designs. Experimental intermediate images were acquired using the same phase mask designs projected from a liquid crystal spatial light modulator. Intermediate images were restored using the penalized space-invariant expectation maximization and the regularized linear least squares algorithms. In the presence of depth-induced SA, systems characterized by radially symmetric PSFs, coupled with model-based computational methods, achieve microscope imaging performance with fewer deviations in structural fidelity (e.g., artifacts) in simulation and experiment and 50% more accurate positioning of 1-µm beads at 10-µm depth in simulation than those with radially asymmetric PSFs. Despite a drop in the signal-to-noise ratio after processing, EDFM is shown to achieve the conventional resolution limit when a model-based reconstruction algorithm with appropriate regularization is used. These trends are also found in images of fixed fluorescently labeled brine shrimp, not adjacent to the coverslip, and fluorescently labeled mitochondria in live cells.

Modern imaging: introduction to the feature issue.

This special issue of Applied Optics contains selected papers reflecting the various disciplines that are needed for the design, implementation and advancement of imaging technology and systems, and it highlights the state-of-the-art research developments in the areas of modern imaging use.

Experimental validation of a customized phase mask designed to enable efficient computational optical sectioning microscopy through wavefront encoding.

In this paper, wavefront-encoded (WFE) computational optical sectioning microscopy (COSM) using a fabricated square cubic (SQUBIC) phase mask, designed to render the system less sensitive to depth-induced aberration, is investigated. The WFE-COSM system is characterized by a point spread function (PSF) that does not vary as rapidly with imaging depth compared to the conventional system. Thus, in WFE-COSM, image restoration from large volumes can be achieved using computationally efficient space-invariant (SI) algorithms, thereby avoiding the use of depth-variant algorithms. The fabricated SQUBIC phase mask was first evaluated and found to have a 75% fidelity compared to the theoretical design; it was then integrated in a commercial wide-field microscope to implement a WFE-COSM system. Evaluation of the WFE-COSM system is demonstrated with comparative studies of theoretical and experimental PSFs and simulated and measured images of spherical shells located at different depths in a test sample. These comparisons show that PSF and imaging models capture major trends in experimental data with a 99% correlation between forward image intensity distribution in experimental and simulated images of spherical shells. Our experimental SI restoration results demonstrate that the WFE-COSM system achieves more than a twofold performance improvement over the conventional system of up to a 65 µm depth below the coverslip investigated in this study.

Reducing effects of aberration in 3D fluorescence imaging using wavefront coding with a radially symmetric phase mask.

In this work, a wavefront encoded (WFE) imaging system built using a squared cubic phase mask, designed to reduce the sensitivity of the imaging system to spherical aberration, is investigated. The proposed system allows the use of a space-invariant image restoration algorithm, which uses a single PSF, to restore intensity distribution in images suffering aberration, such as sample-induced aberration in thick tissue. This provides a computational advantage over depth-variant image restoration algorithms developed previously to address this aberration. Simulated PSFs of the proposed system are shown to change up to 25% compared to the 0 µm depth PSF (quantified by the structural similarity index) over a 100 µm depth range, while the conventional system PSFs change up to 84%. Results from experimental test-sample images show that restoration error is reduced by 29% when the proposed WFE system is used instead of the conventional system over a 30 µm depth range.

Three-dimensional block-based restoration integrated with wide-field fluorescence microscopy for the investigation of thick specimens with spatially variant refractive index.

Development of a block-based restoration (BBR) method that addresses spatially variant (SV) imaging in wide-field fluorescence microscopy of thick samples is presented. The BBR method is based on a block-based imaging model, which approximates SV imaging using an efficient orthonormal basis decomposition of multiple SV point-spread functions computed at block vertices. The effect of reducing the number of blocks needed to account for SV imaging on the restoration accuracy was investigated with simulations using a numerical lung tissue phantom relevant to biological studies. Results show that reducing the number of blocks by 82% and 98% resulted in a 19% and 27% reduction in restoration accuracy, respectively, thereby establishing a reasonable tradeoff between computational resources and accuracy. Comparison of the BBR method to existing methods (deconvolution) that do not account for SV imaging demonstrates a 90% improvement in restoration accuracy. BBR results from synthetic and experimental images of a controlled test sample with SV refractive index (RI) show consistency, providing a validation of the BBR approach. In this study, information from DIC and fluorescence images was combined to identify regions with changing RI within the imaging volume. The BBR method provides a first step toward computationally tractable reconstruction of images from thick samples.

Image restoration for three-dimensional fluorescence microscopy using an orthonormal basis for efficient representation of depth-variant point-spread functions.

A depth-variant (DV) image restoration algorithm for wide field fluorescence microscopy, using an orthonormal basis decomposition of DV point-spread functions (PSFs), is investigated in this study. The efficient PSF representation is based on a previously developed principal component analysis (PCA), which is computationally intensive. We present an approach developed to reduce the number of DV PSFs required for the PCA computation, thereby making the PCA-based approach computationally tractable for thick samples. Restoration results from both synthetic and experimental images show consistency and that the proposed algorithm addresses efficiently depth-induced aberration using a small number of principal components. Comparison of the PCA-based algorithm with a previously-developed strata-based DV restoration algorithm demonstrates that the proposed method improves performance by 50% in terms of accuracy and simultaneously reduces the processing time by 64% using comparable computational resources.

Spatial light modulator phase mask implementation of wavefront encoded 3D computational-optical microscopy.

Spatial light modulator (SLM) implementation of wavefront encoding enables various types of engineered point-spread functions (PSFs), including the generalized-cubic and squared-cubic phase mask wavefront encoded (WFE) PSFs, shown to reduce the impact of sample-induced spherical aberration in fluorescence microscopy. This investigation validates dynamic experimental parameter variation of these WFE-PSFs. We find that particular design parameter bounds exist, within which the divergence of computed and experimental WFE-PSFs is of the same order of magnitude as that of computed and experimental conventional PSFs, such that model-based approaches for solving the inverse imaging problem can be applied to a wide range of SLM-WFE systems. Interferometric measurements were obtained to evaluate the SLM implementation of the numeric mask. Agreement between experiment and theory in terms of a wrapped phase, 0-2π, validates the phase mask implementation and allows characterization of the SLM response. These measurements substantiate experimental practice of computational-optical microscope imaging with an SLM-engineered PSF.

Fluorescence microscopy point spread function model accounting for aberrations due to refractive index variability within a specimen.

A three-dimensional (3-D) point spread function (PSF) model for wide-field fluorescence microscopy, suitable for imaging samples with variable refractive index (RI) in multilayered media, is presented. This PSF model is a key component for accurate 3-D image restoration of thick biological samples, such as lung tissue. Microscope- and specimen-derived parameters are combined with a rigorous vectorial formulation to obtain a new PSF model that accounts for additional aberrations due to specimen RI variability. Experimental evaluation and verification of the PSF model was accomplished using images from 175-nm fluorescent beads in a controlled test sample. Fundamental experimental validation of the advantage of using improved PSFs in depth-variant restoration was accomplished by restoring experimental data from beads (6 µm in diameter) mounted in a sample with RI variation. In the investigated study, improvement in restoration accuracy in the range of 18 to 35% was observed when PSFs from the proposed model were used over restoration using PSFs from an existing model. The new PSF model was further validated by showing that its prediction compares to an experimental PSF (determined from 175-nm beads located below a thick rat lung slice) with a 42% improved accuracy over the current PSF model prediction.

Characterization of a three-dimensional double-helix point-spread function for fluorescence microscopy in the presence of spherical aberration.

We characterize the three-dimensional (3-D) double-helix (DH) point-spread function (PSF) for depth-variant fluorescence microscopy imaging motivated by our interest to integrate the DH-PSF in computational optical sectioning microscopy (COSM) imaging. Physical parameters, such as refractive index and thickness variability of imaging layers encountered in 3-D microscopy give rise to depth-induced spherical aberration (SA) that change the shape of the PSF at different focusing depths and render computational approaches less practical. Theoretical and experimental studies performed to characterize the DH-PSF under varying imaging conditions are presented. Results show reasonable agreement between theoretical and experimental DH-PSFs suggesting that our model can predict the main features of the data. The depth-variability of the DH-PSF due to SA, quantified using a normalized mean square error, shows that the DH-PSF is more robust to SA than the conventional PSF. This result is also supported by the frequency analysis of the DH-PSF shown. Our studies suggest that further investigation of the DH-PSF's use in COSM is warranted, and that particle localization accuracy using the DH-PSF calibration curve in the presence of SA can be improved by accounting for the axial shift due to SA.

Point-spread function engineering to reduce the impact of spherical aberration on 3D computational fluorescence microscopy imaging.

Wavefront encoding (WFE) with different cubic phase mask designs was investigated in engineering 3D point-spread functions (PSF) to reduce their sensitivity to depth-induced spherical aberration (SA) which affects computational complexity in 3D microscopy imaging. The sensitivity of WFE-PSFs to defocus and to SA was evaluated as a function of phase mask parameters using mean-square-error metrics to facilitate the selection of mask designs for extended-depth-of-field (EDOF) microscopy and for computational optical sectioning microscopy (COSM). Further studies on pupil phase contribution and simulated WFE-microscope images evaluated the engineered PSFs and demonstrated SA insensitivity over sample depths of 30 µm. Despite its low sensitivity to SA, the successful WFE design for COSM maintains a high sensitivity to defocus as it is desired for optical sectioning.

Quantitative phase microscopy through differential interference imaging.

An extension of Nomarski differential interference contrast microscopy enables isotropic linear phase imaging through the combination of phase shifting, two directions of shear, and Fourier space integration using a modified spiral phase transform. We apply this method to simulated and experimentally acquired images of partially absorptive test objects. A direct comparison of the computationally determined phase to the true object phase demonstrates the capabilities of the method. Simulation results predict and confirm results obtained from experimentally acquired images.