Analysis of 3D pathology samples using weakly supervised AI.

Human tissue, which is inherently three-dimensional (3D), is traditionally examined through standard-of-care histopathology as limited two-dimensional (2D) cross-sections that can insufficiently represent the tissue due to sampling bias. To holistically characterize histomorphology, 3D imaging modalities have been developed, but clinical translation is hampered by complex manual evaluation and lack of computational platforms to distill clinical insights from large, high-resolution datasets. We present TriPath, a deep-learning platform for processing tissue volumes and efficiently predicting clinical outcomes based on 3D morphological features. Recurrence risk-stratification models were trained on prostate cancer specimens imaged with open-top light-sheet microscopy or microcomputed tomography. By comprehensively capturing 3D morphologies, 3D volume-based prognostication achieves superior performance to traditional 2D slice-based approaches, including clinical/histopathological baselines from six certified genitourinary pathologists. Incorporating greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, further emphasizing the value of capturing larger extents of heterogeneous morphology.

A hybrid open-top light-sheet microscope for versatile multi-scale imaging of cleared tissues.

Light-sheet microscopy has emerged as the preferred means for high-throughput volumetric imaging of cleared tissues. However, there is a need for a flexible system that can address imaging applications with varied requirements in terms of resolution, sample size, tissue-clearing protocol, and transparent sample-holder material. Here, we present a 'hybrid' system that combines a unique non-orthogonal dual-objective and conventional (orthogonal) open-top light-sheet (OTLS) architecture for versatile multi-scale volumetric imaging. We demonstrate efficient screening and targeted sub-micrometer imaging of sparse axons within an intact, cleared mouse brain. The same system enables high-throughput automated imaging of multiple specimens, as spotlighted by a quantitative multi-scale analysis of brain metastases. Compared with existing academic and commercial light-sheet microscopy systems, our hybrid OTLS system provides a unique combination of versatility and performance necessary to satisfy the diverse requirements of a growing number of cleared-tissue imaging applications.

Axially swept open-top light-sheet microscopy for densely labeled clinical specimens.

Open-top light-sheet (OTLS) microscopy offers rapid 3D imaging of large optically cleared specimens. This enables nondestructive 3D pathology, which provides key advantages over conventional slide-based histology including comprehensive sampling without tissue sectioning/destruction and visualization of diagnostically important 3D structures. With 3D pathology, clinical specimens are often labeled with small-molecule stains that broadly target nucleic acids and proteins, mimicking conventional hematoxylin and eosin (H&E) dyes. Tight optical sectioning helps to minimize out-of-focus fluorescence for high-contrast imaging in these densely labeled tissues but has been challenging to achieve in OTLS systems due to trade-offs between optical sectioning and field of view. Here we present an OTLS microscope with voice-coil-based axial sweeping to circumvent this trade-off, achieving 2 µm axial resolution over a 750 × 375 µm field of view. We implement our design in a non-orthogonal dual-objective (NODO) architecture, which enables a 10-mm working distance with minimal sensitivity to refractive index mismatches, for high-contrast 3D imaging of clinical specimens.

Visual Assessment of 2-Dimensional Levels Within 3-Dimensional Pathology Data Sets of Prostate Needle Biopsies Reveals Substantial Spatial Heterogeneity.

Prostate cancer prognostication largely relies on visual assessment of a few thinly sectioned biopsy specimens under a microscope to assign a Gleason grade group (GG). Unfortunately, the assigned GG is not always associated with a patient's outcome in part because of the limited sampling of spatially heterogeneous tumors achieved by 2-dimensional histopathology. In this study, open-top light-sheet microscopy was used to obtain 3-dimensional pathology data sets that were assessed by 4 human readers. Intrabiopsy variability was assessed by asking readers to perform Gleason grading of 5 different levels per biopsy for a total of 20 core needle biopsies (ie, 100 total images). Intrabiopsy variability (Cohen κ) was calculated as the worst pairwise agreement in GG between individual levels within each biopsy and found to be 0.34, 0.34, 0.38, and 0.43 for the 4 pathologists. These preliminary results reveal that even within a 1-mm-diameter needle core, GG based on 2-dimensional images can vary dramatically depending on the location within a biopsy being analyzed. We believe that morphologic assessment of whole biopsies in 3 dimension has the potential to enable more reliable and consistent tumor grading.

Artificial Intelligence-Triaged 3-Dimensional Pathology to Improve Detection of Esophageal Neoplasia While Reducing Pathologist Workloads.

Early detection of esophageal neoplasia via evaluation of endoscopic surveillance biopsies is the key to maximizing survival for patients with Barrett's esophagus, but it is hampered by the sampling limitations of conventional slide-based histopathology. Comprehensive evaluation of whole biopsies with 3-dimensional (3D) pathology may improve early detection of malignancies, but large 3D pathology data sets are tedious for pathologists to analyze. Here, we present a deep learning-based method to automatically identify the most critical 2-dimensional (2D) image sections within 3D pathology data sets for pathologists to review. Our method first generates a 3D heatmap of neoplastic risk for each biopsy, then classifies all 2D image sections within the 3D data set in order of neoplastic risk. In a clinical validation study, we diagnose esophageal biopsies with artificial intelligence-triaged 3D pathology (3 images per biopsy) vs standard slide-based histopathology (16 images per biopsy) and show that our method improves detection sensitivity while reducing pathologist workloads.

Handheld line-scanned dual-axis confocal microscope with pistoned MEMS actuation for flat-field fluorescence imaging.

A handheld line-scanned dual-axis confocal (LS-DAC) microscope has been developed for high-speed (16 frames/s) fluorescence imaging of tissues with sub-nuclear resolution. This is the first miniature fluorescence LS-DAC system that has been fully packaged for handheld clinical use on patients. A novel micro-electro-mechanical system scanning mechanism, with synchronized tilting and pistoning, is used to achieve flat-field en face imaging. We show that this facilitates video mosaicking to generate images that sample an extended lateral field of view.

Solid immersion meniscus lens (SIMlens) for open-top light-sheet microscopy.

Open-top light-sheet (OTLS) microscopy has been developed for rapid volumetric imaging of large pathology specimens. A challenge with OTLS microscopy is the transmission of oblique illumination and detection beams through a horizontal sample plate without introducing aberrations. Previous solutions prevented vertical sample movement, constrained the refractive index of the sample, and/or hindered multi-resolution imaging. Here we describe a solid immersion meniscus lens, a wavefront-matching element that suppresses aberrations when illumination and detection beam transition between air and various high-index immersion media, thereby enabling multi-resolution OTLS microscopy of specimens cleared with diverse protocols.

Visualization technologies for 5-ALA-based fluorescence-guided surgeries.

INTRODUCTION: 5-ALA-based fluorescence-guided surgery has been shown to be a safe and effective method to improve intraoperative visualization and resection of malignant gliomas. However, it remains ineffective in guiding the resection of lower-grade, non-enhancing, and deep-seated tumors, mainly because these tumors do not produce detectable fluorescence with conventional visualization technologies, namely, wide-field (WF) surgical microscopy. METHODS: We describe some of the main factors that limit the sensitivity and accuracy of conventional WF surgical microscopy, and then provide a survey of commercial and research prototypes being developed to address these challenges, along with their principles, advantages and disadvantages, as well as the current status of clinical translation for each technology. We also provide a neurosurgical perspective on how these visualization technologies might best be implemented for guiding glioma surgeries in the future. RESULTS: Detection of PpIX expression in low-grade gliomas and at the infiltrative margins of all gliomas has been achieved with high-sensitivity probe-based visualization techniques. Deep-tissue PpIX imaging of up to 5 mm has also been achieved using red-light illumination techniques. Spectroscopic approaches have enabled more accurate quantification of PpIX expression. CONCLUSION: Advancements in visualization technologies have extended the sensitivity and accuracy of conventional WF surgical microscopy. These technologies will continue to be refined to further improve the extent of resection in glioma patients using 5-ALA-induced fluorescence.

Dual-axis confocal microscopy for point-of-care pathology.

Dual-axis confocal (DAC) microscopy is an optical imaging modality that utilizes simple low-numerical aperture (NA) lenses to achieve effective optical sectioning and superior image contrast in biological tissues. The unique architecture of DAC microscopy also provides some advantages for miniaturization, facilitating the development of endoscopic and handheld DAC systems for in vivo imaging. This article reviews the principles of DAC microscopy, including its differences from conventional confocal microscopy, and surveys several variations of DAC microscopy that have been developed and investigated as non-invasive real-time alternatives to conventional biopsy and histopathology.

Multiplexed Molecular Imaging of Fresh Tissue Surfaces Enabled by Convection-Enhanced Topical Staining with SERS-Coded Nanoparticles.

There is a need for intraoperative imaging technologies to guide breast-conserving surgeries and to reduce the high rates of re-excision for patients in which residual tumor is found at the surgical margins during postoperative pathology analyses. Feasibility studies have shown that utilizing topically applied surface-enhanced Raman scattering (SERS) nanoparticles (NPs), in conjunction with the ratiometric imaging of targeted versus untargeted NPs, enables the rapid visualization of multiple cell-surface biomarkers of cancer that are overexpressed at the surfaces of freshly excised breast tissues. In order to reliably and rapidly perform multiplexed Raman-encoded molecular imaging of large numbers of biomarkers (with five or more NP flavors), an enhanced staining method has been developed in which tissue surfaces are cyclically dipped into an NP-staining solution and subjected to high-frequency mechanical vibration. This dipping and mechanical vibration (DMV) method promotes the convection of the SERS NPs at fresh tissue surfaces, which accelerates their binding to their respective biomarker targets. By utilizing a custom-developed device for automated DMV staining, this study demonstrates the ability to simultaneously image four cell-surface biomarkers of cancer at the surfaces of fresh human breast tissues with a mixture of five flavors of SERS NPs (four targeted and one untargeted control) topically applied for 5 min and imaged at a spatial resolution of 0.5 mm and a raster-scanned imaging rate of >5 cm2 min-1 .

Surgical Guidance via Multiplexed Molecular Imaging of Fresh Tissues Labeled with SERS-Coded Nanoparticles.

The imaging of dysregulated cell-surface receptors (or biomarkers) is a potential means of identifying the presence of cancer with high sensitivity and specificity. However, due to heterogeneities in the expression of protein biomarkers in tumors, molecular imaging technologies should ideally be capable of visualizing a multiplexed panel of cancer biomarkers. Recently, surface-enhanced Raman-scattering (SERS) nanoparticles (NPs) have attracted wide interest due to their potential for sensitive and multiplexed biomarker detection. In this review, we focus on the most recent advances in tumor imaging using SERS-coded NPs. A brief introduction of the structure and optical properties of SERS NPs is provided, followed by a detailed discussion of key imaging issues such as the administration of NPs in tissue (topical versus systemic), the optical configuration and imaging approach of Raman imaging systems, spectral demultiplexing methods for quantifying NP concentrations, and the disambiguation of specific vs. nonspecific sources of contrast through ratiometric imaging of targeted and untargeted (control) NP pairs. Finally, future challenges and directions are briefly outlined.

A Raman-based endoscopic strategy for multiplexed molecular imaging.

Endoscopic imaging is an invaluable diagnostic tool allowing minimally invasive access to tissues deep within the body. It has played a key role in screening colon cancer and is credited with preventing deaths through the detection and removal of precancerous polyps. However, conventional white-light endoscopy offers physicians structural information without the biochemical information that would be advantageous for early detection and is essential for molecular typing. To address this unmet need, we have developed a unique accessory, noncontact, fiber optic-based Raman spectroscopy device that has the potential to provide real-time, multiplexed functional information during routine endoscopy. This device is ideally suited for detection of functionalized surface-enhanced Raman scattering (SERS) nanoparticles as molecular imaging contrast agents. This device was designed for insertion through a clinical endoscope and has the potential to detect and quantify the presence of a multiplexed panel of tumor-targeting SERS nanoparticles. Characterization of the Raman instrument was performed with SERS particles on excised human tissue samples, and it has shown unsurpassed sensitivity and multiplexing capabilities, detecting 326-fM concentrations of SERS nanoparticles and unmixing 10 variations of colocalized SERS nanoparticles. Another unique feature of our noncontact Raman endoscope is that it has been designed for efficient use over a wide range of working distances from 1 to 10 mm. This is necessary to accommodate for imperfect centering during endoscopy and the nonuniform surface topology of human tissue. Using this endoscope as a key part of a multiplexed detection approach could allow endoscopists to distinguish between normal and precancerous tissues rapidly and to identify flat lesions that are otherwise missed.

Comparing high-resolution microscopy techniques for potential intraoperative use in guiding low-grade glioma resections.

BACKGROUND AND OBJECTIVES: Fluorescence image-guided surgery (FIGS), with contrast provided by 5-ALA-induced PpIX, has been shown to enable a higher extent of resection of high-grade gliomas. However, conventional FIGS with low-power microscopy lacks the sensitivity to aid in low-grade glioma (LGG) resection because PpIX signal is weak and sparse in such tissues. Intraoperative high-resolution microscopy of PpIX fluorescence has been proposed as a method to guide LGG resection, where sub-cellular resolution allows for the visualization of sparse and punctate mitochondrial PpIX production in tumor cells. Here, we assess the performance of three potentially portable high-resolution microscopy techniques that may be used for the intraoperative imaging of human LGG tissue samples with PpIX contrast: high-resolution fiber-optic microscopy (HRFM), high-resolution wide-field microscopy (WFM), and dual-axis confocal (DAC) microscopy. MATERIALS AND METHODS: Thick unsectioned human LGG tissue samples (n = 7) with 5-ALA-induced PpIX contrast were imaged using three imaging techniques (HRFM, WFM, DAC). The average signal-to-background ratio (SBR) was then calculated for each imaging modality (5 images per tissue, per modality). RESULTS: HRFM provides the ease of use and portability of a flexible fiber bundle, and is simple and inexpensive to build. However, in most cases (6/7), HRFM is not capable of detecting PpIX signal from LGGs due to high autofluorescence, generated by the fiber bundle under laser illumination at 405 nm, which overwhelms the PpIX signal and impedes its visualization. WFM is a camera-based method possessing high lateral resolution but poor axial resolution, resulting in sub-optimal image contrast. CONCLUSIONS: Consistent successful detection of PpIX signal throughout our human LGG tissue samples (n = 7), with an acceptable image contrast (SBR >2), was only achieved using DAC microscopy, which offers superior image resolution and contrast that is comparable to histology, but requires a laser-scanning mechanism to achieve optical sectioning.

Direct three-dimensional segmentation of prostate glands with nnU-Net.

Significance: In recent years, we and others have developed non-destructive methods to obtain three-dimensional (3D) pathology datasets of clinical biopsies and surgical specimens. For prostate cancer risk stratification (prognostication), standard-of-care Gleason grading is based on examining the morphology of prostate glands in thin 2D sections. This motivates us to perform 3D segmentation of prostate glands in our 3D pathology datasets for the purposes of computational analysis of 3D glandular features that could offer improved prognostic performance. Aim: To facilitate prostate cancer risk assessment, we developed a computationally efficient and accurate deep learning model for 3D gland segmentation based on open-top light-sheet microscopy datasets of human prostate biopsies stained with a fluorescent analog of hematoxylin and eosin (H&E). Approach: For 3D gland segmentation based on our H&E-analog 3D pathology datasets, we previously developed a hybrid deep learning and computer vision-based pipeline, called image translation-assisted segmentation in 3D (ITAS3D), which required a complex two-stage procedure and tedious manual optimization of parameters. To simplify this procedure, we use the 3D gland-segmentation masks previously generated by ITAS3D as training datasets for a direct end-to-end deep learning-based segmentation model, nnU-Net. The inputs to this model are 3D pathology datasets of prostate biopsies rapidly stained with an inexpensive fluorescent analog of H&E and the outputs are 3D semantic segmentation masks of the gland epithelium, gland lumen, and surrounding stromal compartments within the tissue. Results: nnU-Net demonstrates remarkable accuracy in 3D gland segmentations even with limited training data. Moreover, compared with the previous ITAS3D pipeline, nnU-Net operation is simpler and faster, and it can maintain good accuracy even with lower-resolution inputs. Conclusions: Our trained DL-based 3D segmentation model will facilitate future studies to demonstrate the value of computational 3D pathology for guiding critical treatment decisions for patients with prostate cancer.

An end-to-end workflow for nondestructive 3D pathology.

Recent advances in 3D pathology offer the ability to image orders of magnitude more tissue than conventional pathology methods while also providing a volumetric context that is not achievable with 2D tissue sections, and all without requiring destructive tissue sectioning. Generating high-quality 3D pathology datasets on a consistent basis, however, is not trivial and requires careful attention to a series of details during tissue preparation, imaging and initial data processing, as well as iterative optimization of the entire process. Here, we provide an end-to-end procedure covering all aspects of a 3D pathology workflow (using light-sheet microscopy as an illustrative imaging platform) with sufficient detail to perform well-controlled preclinical and clinical studies. Although 3D pathology is compatible with diverse staining protocols and computationally generated color palettes for visual analysis, this protocol focuses on the use of a fluorescent analog of hematoxylin and eosin, which remains the most common stain used for gold-standard pathological reports. We present our guidelines for a broad range of end users (e.g., biologists, clinical researchers and engineers) in a simple format. The end-to-end workflow requires 3-6 d to complete, bearing in mind that data analysis may take longer.

Nondestructive 3D Pathology with Light-Sheet Fluorescence Microscopy for Translational Research and Clinical Assays.

In recent years, there has been a revived appreciation for the importance of spatial context and morphological phenotypes for both understanding disease progression and guiding treatment decisions. Compared with conventional 2D histopathology, which is the current gold standard of medical diagnostics, nondestructive 3D pathology offers researchers and clinicians the ability to visualize orders of magnitude more tissue within their natural volumetric context. This has been enabled by rapid advances in tissue-preparation methods, high-throughput 3D microscopy instrumentation, and computational tools for processing these massive feature-rich data sets. Here, we provide a brief overview of many of these technical advances along with remaining challenges to be overcome. We also speculate on the future of 3D pathology as applied in translational investigations, preclinical drug development, and clinical decision-support assays.

Nondestructive 3D Pathology Image Atlas of Barrett Esophagus With Open-Top Light-Sheet Microscopy.

CONTEXT.­: Anatomic pathologists render diagnosis on tissue samples sectioned onto glass slides and viewed under a bright-field microscope. This approach is destructive to the sample, which can limit its use for ancillary assays that can inform patient management. Furthermore, the subjective interpretation of a relatively small number of 2D tissue sections per sample contributes to low interobserver agreement among pathologists for the assessment (diagnosis and grading) of various lesions. OBJECTIVE.­: To evaluate 3D pathology data sets of thick formalin-fixed Barrett esophagus specimens imaged nondestructively with open-top light-sheet (OTLS) microscopy. DESIGN.­: Formalin-fixed, paraffin-embedded Barrett esophagus samples (N = 15) were deparaffinized, stained with a fluorescent analog of hematoxylin-eosin, optically cleared, and imaged nondestructively with OTLS microscopy. The OTLS microscopy images were subsequently compared with archived hematoxylin-eosin histology sections from each sample. RESULTS.­: Barrett esophagus samples, both small endoscopic forceps biopsies and endoscopic mucosal resections, exhibited similar resolvable structures between OTLS microscopy and conventional light microscopy with up to a ×20 objective (×200 overall magnification). The 3D histologic images generated by OTLS microscopy can enable improved discrimination of cribriform and well-formed gland morphologies. In addition, a much larger amount of tissue is visualized with OTLS microscopy, which enables improved assessment of clinical specimens exhibiting high spatial heterogeneity. CONCLUSIONS.­: In esophageal specimens, OTLS microscopy can generate images comparable in quality to conventional light microscopy, with the advantages of providing 3D information for enhanced evaluation of glandular morphologies and enabling much more of the tissue specimen to be visualized nondestructively.

Introduction to the Biophotonics Congress 2022 feature issue.

A feature issue is being presented by a team of guest editors containing papers based on studies presented at the Optica Biophotonics Congress: Biomedical Optics held on April 24-27, 2022 in Fort Lauderdale, Florida, USA.

Innovations in Remote Teaching of Engineering Design Teams.

The University of Washington's Engineering Innovation in Health program is a yearlong engineering design course sequence where senior undergraduate and graduate engineering students across different disciplines work in teams with health professionals to address their unmet needs. With the onset of the COVID-19 pandemic, these team- and project-based courses shifted from an in-person to remote course environment. Here, we share innovative teaching strategies for a team-based, remote course environment. We show how this shift affected productivity by comparing survey results from before (in person) and during (remote) the pandemic. Preliminary results show that overall project outcomes and productivity were as high or, in some cases, higher during the pandemic than prior to the pandemic. These findings suggest that the innovative remote teaching strategies implemented by the teaching team provided effective options in the absence of certain hands-on experiences that are considered critical to engineering capstone design courses. A discussion on these teaching strategies in the context beyond the pandemic are considered in the discussion.

Weakly Supervised AI for Efficient Analysis of 3D Pathology Samples.

Human tissue consists of complex structures that display a diversity of morphologies, forming a tissue microenvironment that is, by nature, three-dimensional (3D). However, the current standard-of-care involves slicing 3D tissue specimens into two-dimensional (2D) sections and selecting a few for microscopic evaluation1,2, with concomitant risks of sampling bias and misdiagnosis3-6. To this end, there have been intense efforts to capture 3D tissue morphology and transition to 3D pathology, with the development of multiple high-resolution 3D imaging modalities7-18. However, these tools have had little translation to clinical practice as manual evaluation of such large data by pathologists is impractical and there is a lack of computational platforms that can efficiently process the 3D images and provide patient-level clinical insights. Here we present Modality-Agnostic Multiple instance learning for volumetric Block Analysis (MAMBA), a deep-learning-based platform for processing 3D tissue images from diverse imaging modalities and predicting patient outcomes. Archived prostate cancer specimens were imaged with open-top light-sheet microscopy12-14 or microcomputed tomography15,16 and the resulting 3D datasets were used to train risk-stratification networks based on 5-year biochemical recurrence outcomes via MAMBA. With the 3D block-based approach, MAMBA achieves an area under the receiver operating characteristic curve (AUC) of 0.86 and 0.74, superior to 2D traditional single-slice-based prognostication (AUC of 0.79 and 0.57), suggesting superior prognostication with 3D morphological features. Further analyses reveal that the incorporation of greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, suggesting that there is value in capturing larger extents of spatially heterogeneous 3D morphology. With the rapid growth and adoption of 3D spatial biology and pathology techniques by researchers and clinicians, MAMBA provides a general and efficient framework for 3D weakly supervised learning for clinical decision support and can help to reveal novel 3D morphological biomarkers for prognosis and therapeutic response.