Fast and robust feature-based stitching algorithm for microscopic images.

The limited field of view of high-resolution microscopic images hinders the study of biological samples in a single shot. Stitching of microscope images (tiles) captured by the whole-slide imaging (WSI) technique solves this problem. However, stitching is challenging due to the repetitive textures of tissues, the non-informative background part of the slide, and the large number of tiles that impact performance and computational time. To address these challenges, we proposed the Fast and Robust Microscopic Image Stitching (FRMIS) algorithm, which relies on pairwise and global alignment. The speeded up robust features (SURF) were extracted and matched within a small part of the overlapping region to compute the transformation and align two neighboring tiles. In cases where the transformation could not be computed due to an insufficient number of matched features, features were extracted from the entire overlapping region. This enhances the efficiency of the algorithm since most of the computational load is related to pairwise registration and reduces misalignment that may occur by matching duplicated features in tiles with repetitive textures. Then, global alignment was achieved by constructing a weighted graph where the weight of each edge is determined by the normalized inverse of the number of matched features between two tiles. FRMIS has been evaluated on experimental and synthetic datasets from different modalities with different numbers of tiles and overlaps, demonstrating faster stitching time compared to existing algorithms such as the Microscopy Image Stitching Tool (MIST) toolbox. FRMIS outperforms MIST by 481% for bright-field, 259% for phase-contrast, and 282% for fluorescence modalities, while also being robust to uneven illumination.

A comparative analysis of pairwise image stitching techniques for microscopy images.

Stitching of microscopic images is a technique used to combine multiple overlapping images (tiles) from biological samples with a limited field of view and high resolution to create a whole slide image. Image stitching involves two main steps: pairwise registration and global alignment. Most of the computational load and the accuracy of the stitching algorithm depend on the pairwise registration method. Therefore, choosing an efficient, accurate, robust, and fast pairwise registration method is crucial in the whole slide imaging technique. This paper presents a detailed comparative analysis of different pairwise registration techniques in terms of execution time and quality. These techniques included feature-based methods such as Harris, Shi-Thomasi, FAST, ORB, BRISK, SURF, SIFT, KAZE, MSER, and deep learning-based SuperPoint features. Additionally, region-based methods were analyzed, which were based on the normalized cross-correlation (NCC) and the combination of phase correlation and NCC. Investigations have been conducted on microscopy images from different modalities such as bright-field, phase-contrast, and fluorescence. The feature-based methods were highly robust to uneven illumination in tiles. Moreover, some features were found to be more accurate and faster than region-based methods, with the SURF features identified as the most effective technique. This study provides valuable insights into the selection of the most efficient and accurate pairwise registration method for creating whole slide images, which is essential for the advancement of computational pathology and biology.

Reduced Deep Convolutional Activation Features (R-DeCAF) in Histopathology Images to Improve the Classification Performance for Breast Cancer Diagnosis.

Breast cancer is the second most common cancer among women worldwide, and the diagnosis by pathologists is a time-consuming procedure and subjective. Computer-aided diagnosis frameworks are utilized to relieve pathologist workload by classifying the data automatically, in which deep convolutional neural networks (CNNs) are effective solutions. The features extracted from the activation layer of pre-trained CNNs are called deep convolutional activation features (DeCAF). In this paper, we have analyzed that all DeCAF features are not necessarily led to higher accuracy in the classification task and dimension reduction plays an important role. We have proposed reduced DeCAF (R-DeCAF) for this purpose, and different dimension reduction methods are applied to achieve an effective combination of features by capturing the essence of DeCAF features. This framework uses pre-trained CNNs such as AlexNet, VGG-16, and VGG-19 as feature extractors in transfer learning mode. The DeCAF features are extracted from the first fully connected layer of the mentioned CNNs, and a support vector machine is used for classification. Among linear and nonlinear dimensionality reduction algorithms, linear approaches such as principal component analysis (PCA) represent a better combination among deep features and lead to higher accuracy in the classification task using a small number of features considering a specific amount of cumulative explained variance (CEV) of features. The proposed method is validated using experimental BreakHis and ICIAR datasets. Comprehensive results show improvement in the classification accuracy up to 4.3% with a feature vector size (FVS) of 23 and CEV equal to 0.15.

Optical transfer function engineering for a tunable 3D structured illumination microscope.

Two important features of three-dimensional structured illumination microscopy (3D-SIM) are its optical sectioning (OS) and super-resolution (SR) capabilities. Previous works on 3D-SIM systems show that these features are coupled. We demonstrate that a 3D-SIM system using a Fresnel biprism illuminated by multiple linear incoherent sources provides a structured illumination pattern whose lateral and axial modulation frequencies can be tuned separately. Therefore, the compact support of the synthetic optical transfer function (OTF) can be engineered to achieve the highest OS and SR capabilities for a particular imaging application. Theoretical performance of our engineered system based on the OTF support is compared to that achieved by other well-known SIM systems.

Tunable-frequency three-dimensional structured illumination microscopy with reduced data-acquisition.

The performance of a tunable three-dimensional (3D) structured illumination microscope (SIM) system and its ability to provide simultaneously super-resolution (SR) and optical-sectioning (OS) capabilities are investigated. Numerical results show that the performance of our 3D-SIM system is comparable with the one provided by a three-wave interference SIM, while requiring 40% fewer images for the reconstruction and providing frequency tunability in a cost-effective implementation. The performance of the system has been validated experimentally with images from test samples, which were also imaged with a commercial SIM based on incoherent-grid projection for comparison. Restored images from data acquired from an axially-thin fluorescent layer show a 1.6× improvement in OS capability compared to the commercial instrument while results from a fluorescent tilted USAF target show the OS and SR capabilities achieved by our system.

Improvement of two-dimensional structured illumination microscopy with an incoherent illumination pattern of tunable frequency.

In two-dimensional structured illumination microscopy (2D-SIM), high-resolution images with optimal optical sectioning (OS) cannot be obtained simultaneously. This tradeoff can be overcome by using a tunable-frequency 2D-SIM system and a proper reconstruction method. The goal of this work is twofold. First, we present a computational approach to reconstruct optical-sectioned images with super-resolution enhancement (OS-SR) by using a tunable SIM system. Second, we propose an incoherent tunable-frequency 2D-SIM system based on a Fresnel biprism implementation. Integration of the proposed computational method with this tunable structured illumination (SI) system results in a new 2D-SIM system that is advantageous compared to other 2D-SIM systems with comparable complexity, because it provides high-resolution OS images independent of the objective lens used, without the presence of coherent noise and without reducing the contrast of the structured pattern, as in other incoherent implementations. Evaluation of our proposed system is demonstrated with comparative studies of simulated and experimental reconstructed images to validate our theoretical findings. Our experimental results show a simultaneous improvement of the lateral resolution by a factor of 1.8× with the desired OS capability achieved in the resulting OS-SR combination image. Our experimental results also verify that our system can provide better OS capability than the commercial Zeiss ApoTome-SIM system in the investigated study.

Experimental validation of a customized phase mask designed to enable efficient computational optical sectioning microscopy through wavefront encoding.

In this paper, wavefront-encoded (WFE) computational optical sectioning microscopy (COSM) using a fabricated square cubic (SQUBIC) phase mask, designed to render the system less sensitive to depth-induced aberration, is investigated. The WFE-COSM system is characterized by a point spread function (PSF) that does not vary as rapidly with imaging depth compared to the conventional system. Thus, in WFE-COSM, image restoration from large volumes can be achieved using computationally efficient space-invariant (SI) algorithms, thereby avoiding the use of depth-variant algorithms. The fabricated SQUBIC phase mask was first evaluated and found to have a 75% fidelity compared to the theoretical design; it was then integrated in a commercial wide-field microscope to implement a WFE-COSM system. Evaluation of the WFE-COSM system is demonstrated with comparative studies of theoretical and experimental PSFs and simulated and measured images of spherical shells located at different depths in a test sample. These comparisons show that PSF and imaging models capture major trends in experimental data with a 99% correlation between forward image intensity distribution in experimental and simulated images of spherical shells. Our experimental SI restoration results demonstrate that the WFE-COSM system achieves more than a twofold performance improvement over the conventional system of up to a 65 µm depth below the coverslip investigated in this study.