SSTU: Swin-Spectral Transformer U-Net for hyperspectral whole slide image reconstruction.

Whole Slide Imaging and Hyperspectral Microscopic Imaging provide great quality data with high spatial and spectral resolution for histopathology. Existing Hyperspectral Whole Slide Imaging systems combine the advantages of the techniques above, thus providing rich information for pathological diagnosis. However, it cannot avoid the problems of slow acquisition speed and mass data storage demand. Inspired by the spectral reconstruction task in computer vision and remote sensing, the Swin-Spectral Transformer U-Net (SSTU) has been developed to reconstruct Hyperspectral Whole Slide images (HWSis) from multiple Hyperspectral Microscopic images (HMis) of small Field of View and Whole Slide images (WSis). The Swin-Spectral Transformer (SST) module in SSTU takes full advantage of Transformer in extracting global attention. Firstly, Swin Transformer is exploited in space domain, which overcomes the high computation cost in Vision Transformer structures, while it maintains the spatial features extracted from WSis. Furthermore, Spectral Transformer is exploited to collect the long-range spectral features in HMis. Combined with the multi-scale encoder-bottleneck-decoder structure of U-Net, SSTU network is formed by sequential and symmetric residual connections of SSTs, which reconstructs a selected area of HWSi from coarse to fine. Qualitative and quantitative experiments prove the performance of SSTU in HWSi reconstruction task superior to other state-of-the-art spectral reconstruction methods.

Rapid Identification of Infectious Pathogens at the Single-Cell Level via Combining Hyperspectral Microscopic Images and Deep Learning.

Identifying infectious pathogens quickly and accurately is significant for patients and doctors. Identifying single bacterial strains is significant in eliminating culture and speeding up diagnosis. We present an advanced optical method for the rapid detection of infectious (including common and uncommon) pathogens by combining hyperspectral microscopic imaging and deep learning. To acquire more information regarding the pathogens, we developed a hyperspectral microscopic imaging system with a wide wavelength range and fine spectral resolution. Furthermore, an end-to-end deep learning network based on feature fusion, called BI-Net, was designed to extract the species-dependent features encoded in cell-level hyperspectral images as the fingerprints for species differentiation. After being trained based on a large-scale dataset that we built to identify common pathogens, BI-Net was used to classify uncommon pathogens via transfer learning. An extensive analysis demonstrated that BI-Net was able to learn species-dependent characteristics, with the classification accuracy and Kappa coefficients being 92% and 0.92, respectively, for both common and uncommon species. Our method outperformed state-of-the-art methods by a large margin and its excellent performance demonstrates its excellent potential in clinical practice.

Staging of Skin Cancer Based on Hyperspectral Microscopic Imaging and Machine Learning.

Skin cancer, a common type of cancer, is generally divided into basal cell carcinoma (BCC), squamous cell carcinoma (SCC) and malignant melanoma (MM). The incidence of skin cancer has continued to increase worldwide in recent years. Early detection can greatly reduce its morbidity and mortality. Hyperspectral microscopic imaging (HMI) technology can be used as a powerful tool for skin cancer diagnosis by reflecting the changes in the physical structure and microenvironment of the sample through the differences in the HMI data cube. Based on spectral data, this work studied the staging identification of SCC and the influence of the selected region of interest (ROI) on the staging results. In the SCC staging identification process, the optimal result corresponded to the standard normal variate transformation (SNV) for spectra preprocessing, the partial least squares (PLS) for dimensionality reduction, the hold-out method for dataset partition and the random forest (RF) model for staging identification, with the highest staging accuracy of 0.952 ± 0.014, and a kappa value of 0.928 ± 0.022. By comparing the staging results based on spectral characteristics from the nuclear compartments and peripheral regions, the spectral data of the nuclear compartments were found to contribute more to the accurate staging of SCC.

Hyperspectral Microscopic Imaging for the Detection of Head and Neck Squamous Cell Carcinoma on Histologic Slides.

The purpose of this study is to investigate hyperspectral microscopic imaging and deep learning methods for automatic detection of head and neck squamous cell carcinoma (SCC) on histologic slides. Hyperspectral imaging (HSI) cubes were acquired from pathologic slides of 18 patients with SCC of the larynx, hypopharynx, and buccal mucosa. An Inception-based two-dimensional convolutional neural network (CNN) was trained and validated for the HSI data. The automatic deep learning method was tested with independent data of human patients. This study demonstrated the feasibility of using hyperspectral microscopic imaging and deep learning classification to aid pathologists in detecting SCC on histologic slides.

Hyperspectral imaging as a diagnostic tool to differentiate between amalgam tattoos and other dark pigmented intraoral lesions.

The goal of this project is to identify any in-depth benefits and drawbacks in the diagnosis of amalgam tattoos and other pigmented intraoral lesions using hyperspectral imagery collected from amalgam tattoos, benign, and malignant melanocytic neoplasms. Software solutions capable of classifying pigmented lesions of the skin already exist, but conventional red, green and blue images may be reaching an upper limit in their performance. Emerging technologies, such as hyperspectral imaging (HSI) utilize more than a hundred, continuous data channels, while also collecting data in the infrared. A total of 18 paraffin-embedded human tissue specimens of dark pigmented intraoral lesions (including the lip) were analyzed using visible and near-infrared (VIS-NIR) hyperspectral imagery obtained from HE-stained histopathological slides. Transmittance data were collected between 450 and 900 nm using a snapshot camera mounted to a microscope with a halogen light source. VIS-NIR spectra collected from different specimens, such as melanocytic cells and other tissues (eg, epithelium), produced distinct and diagnostic spectra that were used to identify these materials in several regions of interest, making it possible to distinguish between intraoral amalgam tattoos (intramucosal metallic foreign bodies) and melanocytic lesions of the intraoral mucosa and the lip (each with P < .01 using the independent t test). HSI is presented as a diagnostic tool for the rapidly growing field of digital pathology. In this preliminary study, amalgam tattoos were reliably differentiated from melanocytic lesions of the oral cavity and the lip.

Local anomaly detection and quantitative analysis of contaminants in soybean meal using near infrared imaging: The example of non-protein nitrogen.

The melamine scandal indicates that traditional targeted detection methods only detect the specifically listed forms of contamination, which leads to the failure to identify new adulterants in time. In order to deal with continually changing forms of adulterations in food and feed and make up for the inadequacy of targeted detection methods, an untargeted detection method based on local anomaly detection (LAD) using near infrared (NIR) imaging was examined in this study. In the LAD method, with a particular size of window filter and at a 99% level of confidence, a specific value of Global H (GH, modified Mahalanobis distance) can be used as a threshold for anomalous spectra detection and quantitative analysis. The results showed an acceptable performance for the detection of contaminations with the advantage of no need of building a 'clean' library. And, a high coefficient of determination (R2LAD = 0.9984 and R2PLS-DA = 0.9978) for the quantitative analysis of melamine with a limit of detection lower than 0.01% was obtained. This indicates that the new strategy of untargeted detection has the potential to move from passive to active for food and feed safety control.

A Novel Hyperspectral Microscopic Imaging System for Evaluating Fresh Degree of Pork.

This study proposed a rapid microscopic examination method for pork freshness evaluation by using the self-assembled hyperspectral microscopic imaging (HMI) system with the help of feature extraction algorithm and pattern recognition methods. Pork samples were stored for different days ranging from 0 to 5 days and the freshness of samples was divided into three levels which were determined by total volatile basic nitrogen (TVB-N) content. Meanwhile, hyperspectral microscopic images of samples were acquired by HMI system and processed by the following steps for the further analysis. Firstly, characteristic hyperspectral microscopic images were extracted by using principal component analysis (PCA) and then texture features were selected based on the gray level co-occurrence matrix (GLCM). Next, features data were reduced dimensionality by fisher discriminant analysis (FDA) for further building classification model. Finally, compared with linear discriminant analysis (LDA) model and support vector machine (SVM) model, good back propagation artificial neural network (BP-ANN) model obtained the best freshness classification with a 100 % accuracy rating based on the extracted data. The results confirm that the fabricated HMI system combined with multivariate algorithms has ability to evaluate the fresh degree of pork accurately in the microscopic level, which plays an important role in animal food quality control.