Toward digital histopathological assessment in surgery for central nervous system tumors using stimulated Raman histology.

OBJECTIVE: Intraoperative neuropathological assessment with conventional frozen sections supports the neurosurgeon in optimizing the surgical strategy. However, preparation and review of frozen sections can take as long as 45 minutes. Stimulated Raman histology (SRH) was introduced as a novel technique to provide rapid high-resolution digital images of unprocessed tissue samples directly in the operating room that are comparable to conventional histopathological images. Additionally, SRH images are simultaneously and easily accessible for neuropathological judgment. Recently, the first study showed promising results regarding the accuracy and feasibility of SRH compared with conventional histopathology. Thus, the aim of this study was to compare SRH with conventional H&E images and frozen sections in a large cohort of patients with different suspected central nervous system (CNS) tumors. METHODS: The authors included patients who underwent resection or stereotactic biopsy of suspected CNS neoplasm, including brain and spinal tumors. Intraoperatively, tissue samples were safely collected and SRH analysis was performed directly in the operating room. To enable optimal comparison of SRH with H&E images and frozen sections, the authors created a digital databank that included images obtained with all 3 imaging modalities. Subsequently, 2 neuropathologists investigated the diagnostic accuracy, tumor cellularity, and presence of diagnostic histopathological characteristics (score 0 [not present] through 3 [excellent]) determined with SRH images and compared these data to those of H&E images and frozen sections, if available. RESULTS: In total, 94 patients with various suspected CNS tumors were included, and the application of SRH directly in the operating room was feasible in all cases. The diagnostic accuracy based on SRH images was 99% when compared with the final histopathological diagnosis based on H&E images. Additionally, the same histopathological diagnosis was established in all SRH images (100%) when compared with that of the corresponding frozen sections. Moreover, the authors found a statistically significant correlation in tumor cellularity between SRH images and corresponding H&E images (p < 0.0005 and R = 0.867, Pearson correlation coefficient). Finally, excellent (score 3) or good (2) accordance between diagnostic histopathological characteristics and H&E images was present in 95% of cases. CONCLUSIONS: The results of this retrospective analysis demonstrate the near-perfect diagnostic accuracy and capability of visualizing relevant histopathological characteristics with SRH compared with conventional H&E staining and frozen sections. Therefore, digital SRH histopathology seems especially useful for rapid intraoperative investigation to confirm the presence of diagnostic tumor tissue and the precise tumor entity, as well as to rapidly analyze multiple tissue biopsies from the suspected tumor margin. A real-time analysis comparing SRH images and conventional histological images at the time of surgery should be performed as the next step in future studies.

Label-free DNA imaging in vivo with stimulated Raman scattering microscopy.

Label-free DNA imaging is highly desirable in biology and medicine to perform live imaging without affecting cell function and to obtain instant histological tissue examination during surgical procedures. Here we show a label-free DNA imaging method with stimulated Raman scattering (SRS) microscopy for visualization of the cell nuclei in live animals and intact fresh human tissues with subcellular resolution. Relying on the distinct Raman spectral features of the carbon-hydrogen bonds in DNA, the distribution of DNA is retrieved from the strong background of proteins and lipids by linear decomposition of SRS images at three optimally selected Raman shifts. Based on changes on DNA condensation in the nucleus, we were able to capture chromosome dynamics during cell division both in vitro and in vivo. We tracked mouse skin cell proliferation, induced by drug treatment, through in vivo counting of the mitotic rate. Furthermore, we demonstrated a label-free histology method for human skin cancer diagnosis that provides comparable results to other conventional tissue staining methods such as H&E. Our approach exhibits higher sensitivity than SRS imaging of DNA in the fingerprint spectral region. Compared with spontaneous Raman imaging of DNA, our approach is three orders of magnitude faster, allowing both chromatin dynamic studies and label-free optical histology in real time.

Improving the accuracy of brain tumor surgery via Raman-based technology.

Despite advances in the surgical management of brain tumors, achieving optimal surgical results and identification of tumor remains a challenge. Raman spectroscopy, a laser-based technique that can be used to nondestructively differentiate molecules based on the inelastic scattering of light, is being applied toward improving the accuracy of brain tumor surgery. Here, the authors systematically review the application of Raman spectroscopy for guidance during brain tumor surgery. Raman spectroscopy can differentiate normal brain from necrotic and vital glioma tissue in human specimens based on chemical differences, and has recently been shown to differentiate tumor-infiltrated tissues from noninfiltrated tissues during surgery. Raman spectroscopy also forms the basis for coherent Raman scattering (CRS) microscopy, a technique that amplifies spontaneous Raman signals by 10,000-fold, enabling real-time histological imaging without the need for tissue processing, sectioning, or staining. The authors review the relevant basic and translational studies on CRS microscopy as a means of providing real-time intraoperative guidance. Recent studies have demonstrated how CRS can be used to differentiate tumor-infiltrated tissues from noninfiltrated tissues and that it has excellent agreement with traditional histology. Under simulated operative conditions, CRS has been shown to identify tumor margins that would be undetectable using standard bright-field microscopy. In addition, CRS microscopy has been shown to detect tumor in human surgical specimens with near-perfect agreement to standard H & E microscopy. The authors suggest that as the intraoperative application and instrumentation for Raman spectroscopy and imaging matures, it will become an essential component in the neurosurgical armamentarium for identifying residual tumor and improving the surgical management of brain tumors.

RNAi screening for fat regulatory genes with SRS microscopy.

Identification of genes regulating fat accumulation is important for basic and medical research; genetic screening for those genes in Caenorhabditis elegans, a widely used model organism, requires in vivo quantification of lipids. We demonstrated RNA interference screening based on quantitative imaging of lipids with label-free stimulated Raman scattering (SRS) microscopy, which overcomes major limitations of coherent anti-Stokes Raman scattering microscopy. Our screening yielded eight new genetic regulators of fat storage.

Localization of protoporphyrin IX during glioma-resection surgery via paired stimulated Raman histology and fluorescence microscopy.

The most widely used fluorophore in glioma-resection surgery, 5-aminolevulinic acid (5-ALA), is thought to cause the selective accumulation of fluorescent protoporphyrin IX (PpIX) in tumour cells. Here we show that the clinical detection of PpIX can be improved via a microscope that performs paired stimulated Raman histology and two-photon excitation fluorescence microscopy (TPEF). We validated the technique in fresh tumour specimens from 115 patients with high-grade gliomas across four medical institutions. We found a weak negative correlation between tissue cellularity and the fluorescence intensity of PpIX across all imaged specimens. Semi-supervised clustering of the TPEF images revealed five distinct patterns of PpIX fluorescence, and spatial transcriptomic analyses of the imaged tissue showed that myeloid cells predominate in areas where PpIX accumulates in the intracellular space. Further analysis of external spatially resolved metabolomics, transcriptomics and RNA-sequencing datasets from glioblastoma specimens confirmed that myeloid cells preferentially accumulate and metabolize PpIX. Our findings question 5-ALA-induced fluorescence in glioma cells and show how 5-ALA and TPEF imaging can provide a window into the immune microenvironment of gliomas.

Multicolor stimulated Raman scattering microscopy with a rapidly tunable optical parametric oscillator.

Stimulated Raman scattering (SRS) microscopy allows label-free chemical imaging based on vibrational spectroscopy. Narrowband excitation with picosecond lasers creates the highest signal levels and enables imaging speeds up to video-rate, but it sacrifices chemical specificity in samples with overlapping bands compared to broadband (multiplex) excitation. We develop a rapidly tunable picosecond optical parametric oscillator with an electro-optical tunable Lyot filter, and demonstrate multicolor SRS microscopy with synchronized line-by-line wavelength tuning to avoid spectral artifacts due to sample movement. We show sensitive imaging of three different kinds of polymer beads and live HeLa cells with moving intracellular lipid droplets.

Hierarchical discriminative learning improves visual representations of biomedical microscopy.

Learning high-quality, self-supervised, visual representations is essential to advance the role of computer vision in biomedical microscopy and clinical medicine. Previous work has focused on self-supervised representation learning (SSL) methods developed for instance discrimination and applied them directly to image patches, or fields-of-view, sampled from gigapixel whole-slide images (WSIs) used for cancer diagnosis. However, this strategy is limited because it (1) assumes patches from the same patient are independent, (2) neglects the patient-slide-patch hierarchy of clinical biomedical microscopy, and (3) requires strong data augmentations that can degrade downstream performance. Importantly, sampled patches from WSIs of a patient's tumor are a diverse set of image examples that capture the same underlying cancer diagnosis. This motivated HiDisc, a data-driven method that leverages the inherent patient-slide-patch hierarchy of clinical biomedical microscopy to define a hierarchical discriminative learning task that implicitly learns features of the underlying diagnosis. HiDisc uses a self-supervised contrastive learning framework in which positive patch pairs are defined based on a common ancestry in the data hierarchy, and a unified patch, slide, and patient discriminative learning objective is used for visual SSL. We benchmark HiDisc visual representations on two vision tasks using two biomedical microscopy datasets, and demonstrate that (1) HiDisc pretraining outperforms current state-of-the-art self-supervised pretraining methods for cancer diagnosis and genetic mutation prediction, and (2) HiDisc learns high-quality visual representations using natural patch diversity without strong data augmentations.

Multicolored stain-free histopathology with coherent Raman imaging.

Conventional histopathology with hematoxylin & eosin (H&E) has been the gold standard for histopathological diagnosis of a wide range of diseases. However, it is not performed in vivo and requires thin tissue sections obtained after tissue biopsy, which carries risk, particularly in the central nervous system. Here we describe the development of an alternative, multicolored way to visualize tissue in real-time through the use of coherent Raman imaging (CRI), without the use of dyes. CRI relies on intrinsic chemical contrast based on vibrational properties of molecules and intrinsic optical sectioning by nonlinear excitation. We demonstrate that multicolor images originating from CH(2) and CH(3) vibrations of lipids and protein, as well as two-photon absorption of hemoglobin, can be obtained with subcellular resolution from fresh tissue. These stain-free histopathological images show resolutions similar to those obtained by conventional techniques, but do not require tissue fixation, sectioning or staining of the tissue analyzed.

Coherent nonlinear optical imaging: beyond fluorescence microscopy.

The quest for ultrahigh detection sensitivity with spectroscopic contrasts other than fluorescence has led to various novel approaches to optical microscopy of biological systems. Coherent nonlinear optical imaging, especially the recently developed nonlinear dissipation microscopy (including stimulated Raman scattering and two-photon absorption) and pump-probe microscopy (including excited-state absorption, stimulated emission, and ground-state depletion), provides new image contrasts for nonfluorescent species. Thanks to the high-frequency modulation transfer scheme, these imaging techniques exhibit superb detection sensitivity. By directly interrogating vibrational and/or electronic energy levels of molecules, they offer high molecular specificity. Here we review the underlying principles and excitation and detection schemes, as well as exemplary biomedical applications of this emerging class of molecular imaging techniques.

Label-free live-cell imaging of nucleic acids using stimulated Raman scattering microscopy.

Imaging of nucleic acids is important for studying cellular processes such as cell division and apoptosis. A noninvasive label-free technique is attractive. Raman spectroscopy provides rich chemical information based on specific vibrational peaks. However, the signal from spontaneous Raman scattering is weak and long integration times are required, which drastically limits the imaging speed when used for microscopy. Coherent Raman scattering techniques, comprising coherent anti-Stokes Raman scattering (CARS) and stimulated Raman scattering (SRS) microscopy, overcome this problem by enhancing the signal level by up to five orders of magnitude. CARS microscopy suffers from a nonresonant background signal, which distorts Raman spectra and limits sensitivity. This makes CARS imaging of weak transitions in spectrally congested regions challenging. This is especially the case in the fingerprint region, where nucleic acids show characteristic peaks. The recently developed SRS microscopy is free from these limitations; excitation spectra are identical to those of spontaneous Raman and sensitivity is close to shot-noise limited. Herein we demonstrate the use of SRS imaging in the fingerprint region to map the distribution of nucleic acids in addition to proteins and lipids in single salivary gland cells of Drosophila larvae, and in single mammalian cells. This allows the imaging of DNA condensation associated with cell division and opens up possibilities of imaging such processes in vivo.

OpenSRH: optimizing brain tumor surgery using intraoperative stimulated Raman histology.

Accurate intraoperative diagnosis is essential for providing safe and effective care during brain tumor surgery. Our standard-of-care diagnostic methods are time, resource, and labor intensive, which restricts access to optimal surgical treatments. To address these limitations, we propose an alternative workflow that combines stimulated Raman histology (SRH), a rapid optical imaging method, with deep learning-based automated interpretation of SRH images for intraoperative brain tumor diagnosis and real-time surgical decision support. Here, we present OpenSRH, the first public dataset of clinical SRH images from 300+ brain tumors patients and 1300+ unique whole slide optical images. OpenSRH contains data from the most common brain tumors diagnoses, full pathologic annotations, whole slide tumor segmentations, raw and processed optical imaging data for end-to-end model development and validation. We provide a framework for patch-based whole slide SRH classification and inference using weak (i.e. patient-level) diagnostic labels. Finally, we benchmark two computer vision tasks: multiclass histologic brain tumor classification and patch-based contrastive representation learning. We hope OpenSRH will facilitate the clinical translation of rapid optical imaging and real-time ML-based surgical decision support in order to improve the access, safety, and efficacy of cancer surgery in the era of precision medicine. Dataset access, code, and benchmarks are available at https://opensrh.mlins.org.

Novel rapid intraoperative qualitative tumor detection by a residual convolutional neural network using label-free stimulated Raman scattering microscopy.

Determining the presence of tumor in biopsies and the decision-making during resections is often dependent on intraoperative rapid frozen-section histopathology. Recently, stimulated Raman scattering microscopy has been introduced to rapidly generate digital hematoxylin-and-eosin-stained-like images (stimulated Raman histology) for intraoperative analysis. To enable intraoperative prediction of tumor presence, we aimed to develop a new deep residual convolutional neural network in an automated pipeline and tested its validity. In a monocentric prospective clinical study with 94 patients undergoing biopsy, brain or spinal tumor resection, Stimulated Raman histology images of intraoperative tissue samples were obtained using a fiber-laser-based stimulated Raman scattering microscope. A residual network was established and trained in ResNetV50 to predict three classes for each image: (1) tumor, (2) non-tumor, and (3) low-quality. The residual network was validated on images obtained in three small random areas within the tissue samples and were blindly independently reviewed by a neuropathologist as ground truth. 402 images derived from 132 tissue samples were analyzed representing the entire spectrum of neurooncological surgery. The automated workflow took in a mean of 240 s per case, and the residual network correctly classified tumor (305/326), non-tumorous tissue (49/67), and low-quality (6/9) images with an inter-rater agreement of 89.6% (κ = 0.671). An excellent internal consistency was found among the random areas with 90.2% (Cα = 0.942) accuracy. In conclusion, the novel stimulated Raman histology-based residual network can reliably detect the microscopic presence of tumor and differentiate from non-tumorous brain tissue in resection and biopsy samples within 4 min and may pave a promising way for an alternative rapid intraoperative histopathological decision-making tool.

Rapid Automated Analysis of Skull Base Tumor Specimens Using Intraoperative Optical Imaging and Artificial Intelligence.

BACKGROUND: Accurate specimen analysis of skull base tumors is essential for providing personalized surgical treatment strategies. Intraoperative specimen interpretation can be challenging because of the wide range of skull base pathologies and lack of intraoperative pathology resources. OBJECTIVE: To develop an independent and parallel intraoperative workflow that can provide rapid and accurate skull base tumor specimen analysis using label-free optical imaging and artificial intelligence. METHODS: We used a fiber laser-based, label-free, nonconsumptive, high-resolution microscopy method (<60 seconds per 1 × 1 mm2), called stimulated Raman histology (SRH), to image a consecutive, multicenter cohort of patients with skull base tumor. SRH images were then used to train a convolutional neural network model using 3 representation learning strategies: cross-entropy, self-supervised contrastive learning, and supervised contrastive learning. Our trained convolutional neural network models were tested on a held-out, multicenter SRH data set. RESULTS: SRH was able to image the diagnostic features of both benign and malignant skull base tumors. Of the 3 representation learning strategies, supervised contrastive learning most effectively learned the distinctive and diagnostic SRH image features for each of the skull base tumor types. In our multicenter testing set, cross-entropy achieved an overall diagnostic accuracy of 91.5%, self-supervised contrastive learning 83.9%, and supervised contrastive learning 96.6%. Our trained model was able to segment tumor-normal margins and detect regions of microscopic tumor infiltration in meningioma SRH images. CONCLUSION: SRH with trained artificial intelligence models can provide rapid and accurate intraoperative analysis of skull base tumor specimens to inform surgical decision-making.

Coherent Raman scanning fiber endoscopy.

Coherent Raman scattering methods allow for label-free imaging of tissue with chemical contrast and high spatial and temporal resolution. However, their imaging depth in scattering tissue is limited to less than 1 mm, requiring the development of endoscopes to obtain images deep inside the body. Here, we describe a coherent Raman endoscope that provides stimulated Raman scattering images at seven frames per second using a miniaturized fiber scanner, a custom-designed objective lens, and an optimized scheme for collection of scattered light from the tissue. We characterize the system and demonstrate chemical selectivity in mouse tissue images.

Rapid, label-free detection of diffuse glioma recurrence using intraoperative stimulated Raman histology and deep neural networks.

BACKGROUND: Detection of glioma recurrence remains a challenge in modern neuro-oncology. Noninvasive radiographic imaging is unable to definitively differentiate true recurrence versus pseudoprogression. Even in biopsied tissue, it can be challenging to differentiate recurrent tumor and treatment effect. We hypothesized that intraoperative stimulated Raman histology (SRH) and deep neural networks can be used to improve the intraoperative detection of glioma recurrence. METHODS: We used fiber laser-based SRH, a label-free, nonconsumptive, high-resolution microscopy method (<60 sec per 1 × 1 mm2) to image a cohort of patients (n = 35) with suspected recurrent gliomas who underwent biopsy or resection. The SRH images were then used to train a convolutional neural network (CNN) and develop an inference algorithm to detect viable recurrent glioma. Following network training, the performance of the CNN was tested for diagnostic accuracy in a retrospective cohort (n = 48). RESULTS: Using patch-level CNN predictions, the inference algorithm returns a single Bernoulli distribution for the probability of tumor recurrence for each surgical specimen or patient. The external SRH validation dataset consisted of 48 patients (recurrent, 30; pseudoprogression, 18), and we achieved a diagnostic accuracy of 95.8%. CONCLUSION: SRH with CNN-based diagnosis can be used to improve the intraoperative detection of glioma recurrence in near-real time. Our results provide insight into how optical imaging and computer vision can be combined to augment conventional diagnostic methods and improve the quality of specimen sampling at glioma recurrence.

Synchronized time-lens source for coherent Raman scattering microscopy.

We use the time-lens concept to demonstrate a new scheme for synchronization of two pulsed light sources for biological imaging. An all fiber, 1064 nm time-lens source is synchronized to a picosecond solid-state Ti: Sapphire mode-locked laser by using the mode-locked laser pulses as the clock. We demonstrate the application of this synchronized source for CARS and SRS imaging by imaging mouse tissues. Synchronized two wavelength pulsed source is an important technical difficulty for CARS and SRS imaging. The time-lens source demonstrated here may provide an all fiber, user friendly alternative for future SRS imaging.

Near real-time intraoperative brain tumor diagnosis using stimulated Raman histology and deep neural networks.

Intraoperative diagnosis is essential for providing safe and effective care during cancer surgery1. The existing workflow for intraoperative diagnosis based on hematoxylin and eosin staining of processed tissue is time, resource and labor intensive2,3. Moreover, interpretation of intraoperative histologic images is dependent on a contracting, unevenly distributed, pathology workforce4. In the present study, we report a parallel workflow that combines stimulated Raman histology (SRH)5-7, a label-free optical imaging method and deep convolutional neural networks (CNNs) to predict diagnosis at the bedside in near real-time in an automated fashion. Specifically, our CNNs, trained on over 2.5 million SRH images, predict brain tumor diagnosis in the operating room in under 150 s, an order of magnitude faster than conventional techniques (for example, 20-30 min)2. In a multicenter, prospective clinical trial (n = 278), we demonstrated that CNN-based diagnosis of SRH images was noninferior to pathologist-based interpretation of conventional histologic images (overall accuracy, 94.6% versus 93.9%). Our CNNs learned a hierarchy of recognizable histologic feature representations to classify the major histopathologic classes of brain tumors. In addition, we implemented a semantic segmentation method to identify tumor-infiltrated diagnostic regions within SRH images. These results demonstrate how intraoperative cancer diagnosis can be streamlined, creating a complementary pathway for tissue diagnosis that is independent of a traditional pathology laboratory.

Intracavity wavelength modulation of an optical parametric oscillator for coherent Raman microscopy.

We present a novel intracavity frequency modulation scheme in a tunable, picosecond optical parametric oscillator (OPO). The OPO signal wavelength can be modulated with a depth of more than 10 nm at a rate of 38 MHz (one half its repetition rate). We discuss the design and construction of the light source and its application to the recently-developed frequency modulation coherent anti-Stokes Raman scattering (FM-CARS) and stimulated Raman scattering (SRS) techniques. The new light source allows for real time subtraction of the interfering background signal in coherent Raman imaging, yielding images with purely chemical contrast.

Label-free imaging of amyloid plaques in Alzheimer's disease with stimulated Raman scattering microscopy.

One of the key pathological features of Alzheimer's disease (AD) is the existence of extracellular deposition of amyloid plaques formed with misfolded amyloid-ß (Aß). The conformational change of proteins leads to enriched contents of ß sheets, resulting in remarkable changes of vibrational spectra, especially the spectral shifts of the amide I mode. Here, we applied stimulated Raman scattering (SRS) microscopy to image amyloid plaques in the brain tissue of an AD mouse model. We have demonstrated the capability of SRS microscopy as a rapid, label-free imaging modality to differentiate misfolded from normal proteins based on the blue shift (~10 cm-1) of amide I SRS spectra. Furthermore, SRS imaging of Aß plaques was verified by antibody staining of frozen thin sections and fluorescence imaging of fresh tissues. Our method may provide a new approach for studies of AD pathology, as well as other neurodegenerative diseases associated with protein misfolding.

Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy.

Conventional methods for intraoperative histopathologic diagnosis are labour- and time-intensive, and may delay decision-making during brain-tumour surgery. Stimulated Raman scattering (SRS) microscopy, a label-free optical process, has been shown to rapidly detect brain-tumour infiltration in fresh, unprocessed human tissues. Here, we demonstrate the first application of SRS microscopy in the operating room by using a portable fibre-laser-based microscope and unprocessed specimens from 101 neurosurgical patients. We also introduce an image-processing method - stimulated Raman histology (SRH) - which leverages SRS images to create virtual haematoxylin-and-eosin-stained slides, revealing essential diagnostic features. In a simulation of intraoperative pathologic consultation in 30 patients, we found a remarkable concordance of SRH and conventional histology for predicting diagnosis (Cohen's kappa, κ > 0.89), with accuracy exceeding 92%. We also built and validated a multilayer perceptron based on quantified SRH image attributes that predicts brain-tumour subtype with 90% accuracy. Our findings provide insight into how SRH can now be used to improve the surgical care of brain tumour patients.