Deep learning automated pathology in ex vivo microscopy.

Standard histopathology is currently the gold standard for assessment of margin status in Mohs surgical removal of skin cancer. Ex vivo confocal microscopy (XVM) is potentially faster, less costly and inherently 3D/digital compared to standard histopathology. Despite these advantages, XVM use is not widespread due, in part, to the need for pathologists to retrain to interpret XVM images. We developed artificial intelligence (AI)-driven XVM pathology by implementing algorithms that render intuitive XVM pathology images identical to standard histopathology and produce automated tumor positivity maps. XVM images have fluorescence labeling of cellular and nuclear biology on the background of endogenous (unstained) reflectance contrast as a grounding counter-contrast. XVM images of 26 surgical excision specimens discarded after Mohs micrographic surgery were used to develop an XVM data pipeline with 4 stages: flattening, colorizing, enhancement and automated diagnosis. The first two stages were novel, deterministic image processing algorithms, and the second two were AI algorithms. Diagnostic sensitivity and specificity were calculated for basal cell carcinoma detection as proof of principal for the XVM image processing pipeline. The resulting diagnostic readouts mimicked the appearance of histopathology and found tumor positivity that required first collapsing the confocal stack to a 2D image optimized for cellular fluorescence contrast, then a dark field-to-bright field colorizing transformation, then either an AI image transformation for visual inspection or an AI diagnostic binary image segmentation of tumor obtaining a diagnostic sensitivity and specificity of 88% and 91% respectively. These results show that video-assisted micrographic XVM pathology could feasibly aid margin status determination in micrographic surgery of skin cancer.

Deep learning-level melanoma detection by interpretable machine learning and imaging biomarker cues.

SIGNIFICANCE: Melanoma is a deadly cancer that physicians struggle to diagnose early because they lack the knowledge to differentiate benign from malignant lesions. Deep machine learning approaches to image analysis offer promise but lack the transparency to be widely adopted as stand-alone diagnostics. AIM: We aimed to create a transparent machine learning technology (i.e., not deep learning) to discriminate melanomas from nevi in dermoscopy images and an interface for sensory cue integration. APPROACH: Imaging biomarker cues (IBCs) fed ensemble machine learning classifier (Eclass) training while raw images fed deep learning classifier training. We compared the areas under the diagnostic receiver operator curves. RESULTS: Our interpretable machine learning algorithm outperformed the leading deep-learning approach 75% of the time. The user interface displayed only the diagnostic imaging biomarkers as IBCs. CONCLUSIONS: From a translational perspective, Eclass is better than convolutional machine learning diagnosis in that physicians can embrace it faster than black box outputs. Imaging biomarkers cues may be used during sensory cue integration in clinical screening. Our method may be applied to other image-based diagnostic analyses, including pathology and radiology.

A 3D biofabricated cutaneous squamous cell carcinoma tissue model with multi-channel confocal microscopy imaging biomarkers to quantify antitumor effects of chemotherapeutics in tissue.

Cutaneous squamous cell carcinoma (cSCC) causes approximately 10,000 deaths annually in the U. S. Current therapies are largely ineffective against metastatic and locally advanced cSCC. There is a need to identify novel, effective, and less toxic small molecule cSCC therapeutics. We developed a 3-dimensional bioprinted skin (3DBPS) model of cSCC tumors together with a microscopy assay to test chemotherapeutic effects in tissue. The full thickness SCC tissue model was validated using hematoxylin and eosin (H&E) and immunohistochemical histological staining, confocal microscopy, and cDNA microarray analysis. A nondestructive, 3D fluorescence confocal imaging assay with tdTomato-labeled A431 SCC and ZsGreen-labeled keratinocytes was developed to test efficacy and general toxicity of chemotherapeutics. Fluorescence-derived imaging biomarkers indicated that 50% of cancer cells were killed in the tissue after 1µM 5-Fluorouracil 48-hour treatment, compared to a baseline of 12% for untreated controls. The imaging biomarkers also showed that normal keratinocytes were less affected by treatment (11% killed) than the untreated tissue, which had no significant killing effect. Data showed that 5-Fluorouracil selectively killed cSCC cells more than keratinocytes. Our 3DBPS assay platform provides cellular-level measurement of cell viability and can be adapted to achieve nondestructive high-throughput screening (HTS) in bio-fabricated tissues.

Rapid Generation of Somatic Mouse Mosaics with Locus-Specific, Stably Integrated Transgenic Elements.

In situ transgenesis methods such as viruses and electroporation can rapidly create somatic transgenic mice but lack control over copy number, zygosity, and locus specificity. Here we establish mosaic analysis by dual recombinase-mediated cassette exchange (MADR), which permits stable labeling of mutant cells expressing transgenic elements from precisely defined chromosomal loci. We provide a toolkit of MADR elements for combination labeling, inducible and reversible transgene manipulation, VCre recombinase expression, and transgenesis of human cells. Further, we demonstrate the versatility of MADR by creating glioma models with mixed reporter-identified zygosity or with "personalized" driver mutations from pediatric glioma. MADR is extensible to thousands of existing mouse lines, providing a flexible platform to democratize the generation of somatic mosaic mice. VIDEO ABSTRACT.

Hyperspectral imaging in automated digital dermoscopy screening for melanoma.

OBJECTIVES: Early melanoma detection decreases morbidity and mortality. Early detection classically involves dermoscopy to identify suspicious lesions for which biopsy is indicated. Biopsy and histological examination then diagnose benign nevi, atypical nevi, or cancerous growths. With current methods, a considerable number of unnecessary biopsies are performed as only 11% of all biopsied, suspicious lesions are actually melanomas. Thus, there is a need for more advanced noninvasive diagnostics to guide the decision of whether or not to biopsy. Artificial intelligence can generate screening algorithms that transform a set of imaging biomarkers into a risk score that can be used to classify a lesion as a melanoma or a nevus by comparing the score to a classification threshold. Melanoma imaging biomarkers have been shown to be spectrally dependent in Red, Green, Blue (RGB) color channels, and hyperspectral imaging may further enhance diagnostic power. The purpose of this study was to use the same melanoma imaging biomarkers previously described, but over a wider range of wavelengths to determine if, in combination with machine learning algorithms, this could result in enhanced melanoma detection. METHODS: We used the melanoma advanced imaging dermatoscope (mAID) to image pigmented lesions assessed by dermatologists as requiring a biopsy. The mAID is a 21-wavelength imaging device in the 350-950 nm range. We then generated imaging biomarkers from these hyperspectral dermoscopy images, and, with the help of artificial intelligence algorithms, generated a melanoma Q-score for each lesion (0 = nevus, 1 = melanoma). The Q-score was then compared to the histopathologic diagnosis. RESULTS: The overall sensitivity and specificity of hyperspectral dermoscopy in detecting melanoma when evaluated in a set of lesions selected by dermatologists as requiring biopsy was 100% and 36%, respectively. CONCLUSION: With widespread application, and if validated in larger clinical trials, this non-invasive methodology could decrease unnecessary biopsies and potentially increase life-saving early detection events. Lasers Surg. Med. 51:214-222, 2019. © 2019 The Authors. Lasers in Surgery and Medicine Published by Wiley Periodicals, Inc.

Line scanning, stage scanning confocal microscope (LSSSCM).

For rapid pathological assessment of large surgical tissue excisions with cellular resolution, we present a line scanning, stage scanning confocal microscope (LSSSCM). LSSSCM uses no scanning mirrors. Laser light is focused with a single cylindrical lens to a line of diffraction-limited width directly into the (Z) sample focal plane, which is parallel to and near the flattened specimen surface. Semi-confocal optical sections are derived from the linear array distribution (Y) and a single mechanical drive that moves the sample parallel to the focal plane and perpendicular to the focused line (X). LSSSCM demonstrates cellular resolution in the conditions of high nuclear density within micronodular basal cell carcinoma.

Digital imaging biomarkers feed machine learning for melanoma screening.

We developed an automated approach for generating quantitative image analysis metrics (imaging biomarkers) that are then analysed with a set of 13 machine learning algorithms to generate an overall risk score that is called a Q-score. These methods were applied to a set of 120 "difficult" dermoscopy images of dysplastic nevi and melanomas that were subsequently excised/classified. This approach yielded 98% sensitivity and 36% specificity for melanoma detection, approaching sensitivity/specificity of expert lesion evaluation. Importantly, we found strong spectral dependence of many imaging biomarkers in blue or red colour channels, suggesting the need to optimize spectral evaluation of pigmented lesions.

Use of Digitally Stained Multimodal Confocal Mosaic Images to Screen for Nonmelanoma Skin Cancer.

Importance: Confocal microscopy has the potential to provide rapid bedside pathologic analysis, but clinical adoption has been limited in part by the need for physician retraining to interpret grayscale images. Digitally stained confocal mosaics (DSCMs) mimic the colors of routine histologic specimens and may increase adaptability of this technology. Objective: To evaluate the accuracy and precision of 3 physicians using DSCMs before and after training to detect basal cell carcinoma (BCC) and squamous cell carcinoma (SCC) in Mohs micrographic surgery fresh-tissue specimens. Design: This retrospective study used 133 DSCMs from 64 Mohs tissue excisions, which included clear margins, residual BCC, or residual SCC. Discarded tissue from Mohs surgical excisions from the dermatologic surgery units at Memorial Sloan Kettering Cancer Center and Oregon Health & Science University were collected for confocal imaging from 2006 to 2011. Final data analysis and interpretation took place between 2014 and 2016. Two Mohs surgeons and a Mohs fellow, who were blinded to the correlating gold standard frozen section diagnoses, independently reviewed the DSCMs for residual nonmelanoma skin cancer (NMSC) before and after a brief training session (about 5 minutes). The 2 assessments were separated by a 6-month washout period. Main Outcomes and Measures: Diagnostic accuracy was characterized by sensitivity and specificity of detecting NMSC using DSCMs vs standard frozen histopathologic specimens. The diagnostic precision was calculated based on interobserver agreement and κ scores. Paired 2-sample t tests were used for comparative means analyses before and after training. Results: The average respective sensitivities and specificities of detecting NMSC were 90% (95% CI, 89%-91%) and 79% (95% CI, 52%-100%) before training and 99% (95% CI, 99%-99%) (P = .001) and 93% (95% CI, 90%-96%) (P = .18) after training; for BCC, they were 83% (95% CI, 59%-100%) and 92% (95% CI, 81%-100%) before training and 98% (95% CI, 98%-98%) (P = .18) and 97% (95% CI, 95%-100%) (P = .15) after training; for SCC, they were 73% (95% CI, 65%-81%) and 89% (95% CI, 72%-100%) before training and 100% (P = .004) and 98% (95% CI, 95%-100%) (P = .21) after training. The pretraining interobserver agreement was 72% (κ = 0.58), and the posttraining interobserver agreement was 98% (κ = 0.97) (P = .04). Conclusions and Relevance: Diagnostic use of DSCMs shows promising correlation to frozen histologic analysis, but image quality was affected by variations in image contrast and mosaic-stitching artifact. With training, physicians were able to read DSCMs with significantly improved accuracy and precision to detect NMSC.

Rapid screening of cancer margins in tissue with multimodal confocal microscopy.

BACKGROUND: Complete and accurate excision of cancer is guided by the examination of histopathology. However, preparation of histopathology is labor intensive and slow, leading to insufficient sampling of tissue and incomplete and/or inaccurate excision of margins. We demonstrate the potential utility of multimodal confocal mosaicing microscopy for rapid screening of cancer margins, directly in fresh surgical excisions, without the need for conventional embedding, sectioning, or processing. MATERIALS AND METHODS: A multimodal confocal mosaicing microscope was developed to image basal cell carcinoma margins in surgical skin excisions, with the resolution that shows nuclear detail. Multimodal contrast is with fluorescence for imaging nuclei and reflectance for cellular cytoplasm and dermal collagen. Thirty-five excisions of basal cell carcinomas from Mohs surgery were imaged, and the mosaics analyzed by comparison with the corresponding frozen pathology. RESULTS: Confocal mosaics are produced in about 9 min, displaying tissue in fields of view of 12 mm with ×2 magnification. A digital staining algorithm transforms black and white contrast to purple and pink, which simulates the appearance of standard histopathology. Mosaicing enables rapid digital screening, which mimics the examination of histopathology. CONCLUSIONS: Multimodal confocal mosaicing microscopy offers a technology platform to potentially enable real-time pathology at the bedside. The imaging may serve as an adjunct to conventional histopathology to expedite screening of margins and guide surgery toward more complete and accurate excision of cancer.

Optical fiber probe spectroscopy for laparoscopic monitoring of tissue oxygenation during esophagectomies.

Anastomotic complication is a major morbidity associated with esophagectomy. Gastric ischemia after conduit creation contributes to anastomotic complications, but a reliable method to assess oxygenation in the gastric conduit is lacking. We hypothesize that fiber optic spectroscopy can reliably assess conduit oxygenation, and that intraoperative gastric ischemia will correlate with the development of anastomotic complications. A simple optical fiber probe spectrometer is designed for nondestructive laparoscopic measurement of blood content and hemoglobin oxygen saturation in the stomach tissue microvasculature during human esophagectomies. In 22 patients, the probe measured the light transport in stomach tissue between two fibers spaced 3-mm apart (500- to 650-nm wavelength range). The stomach tissue site of measurement becomes the site of a gastroesophageal anastamosis following excision of the cancerous esophagus and surgical ligation of two of the three gastric arteries that provide blood perfusion to the anastamosis. Measurements are made at each of five steps throughout the surgery. The resting baseline saturation is 0.51±0.15 and decreases to 0.35±0.20 with ligation. Seven patients develop anastomotic complications, and a decreased saturation at either of the last two steps (completion of conduit and completion of anastamosis) is predictive of complication with a sensitivity of 0.71 when the specificity equaled 0.71.

Sensitivity and specificity for detecting basal cell carcinomas in Mohs excisions with confocal fluorescence mosaicing microscopy.

Recent studies have demonstrated the ability of confocal fluorescence mosaicing microscopy to rapidly detect basal cell carcinomas (BCCs) directly in thick and fresh Mohs surgical excisions. Mosaics of confocal images display large areas of tissue with high resolution and magnification equivalent to 2x, which is the standard magnification when examining pathology. Comparison of mosaics to Mohs frozen histopathology was shown to be excellent for all types of BCCs. However, comparisons in the previous studies were visual and qualitative. In this work, we report the results of a semiquantitative preclinical study in which 45 confocal mosaics are blindly evaluated for the presence (or absence) of BCC tumor. The evaluations are made by two clinicians: a senior Mohs surgeon with prior expertise in interpreting confocal images, and a novice Mohs fellow with limited experience. The blinded evaluation is compared to the gold standard of frozen histopathology. BCCs are detected with an overall sensitivity of 96.6%, specificity of 89.2%, positive predictive value of 93.0%, and negative predictive value of 94.7%. The results demonstrate the potential clinical utility of confocal mosaicing microscopy toward rapid surgical pathology at the bedside to expedite and guide surgery.

Feasibility of digitally stained multimodal confocal mosaics to simulate histopathology.

Fluorescence confocal mosaicing microscopy of tissue biopsies stained with acridine orange has been shown to accurately identify tumors and with an overall sensitivity of 96.6% and specificity of 89.2%. However, fluorescence shows only nuclear detail similar to hematoxylin in histopathology and does not show collagen or cytoplasm, which may provide necessary negative contrast information similar to eosin used in histopathology. Reflectance mode contrast is sensitive to collagen and cytoplasm without staining. To further improve sensitivity and specificity, digitally stained confocal mosaics combine confocal fluorescence and reflectance images in a multimodal pseudo-color image to mimic the appearance of histopathology with hematoxylin and eosin and facilitate the introduction of confocal microscopy into the clinical realm.

Confocal mosaicing microscopy in Mohs skin excisions: feasibility of rapid surgical pathology.

Mosaicing of confocal images enables observation of nuclear morphology in large areas of tissue. An application of interest is rapid detection of basal cell carcinomas (BCCs) in skin excisions during Mohs surgery. A mosaic is currently created in less than 9 min, whereas preparing frozen histology requires 20 to 45 min for an excision. In reflectance mosaics, using acetic acid as a contrast agent to brighten nuclei, large and densely nucleated BCC tumors were detectable in fields of view of 12 x 12 mm (which is equivalent to a 2x-magnified view as required by Mohs surgeons). However, small and sparsely nucleated tumors remained undetectable. Their diminutive size within the large field of view resulted in weak light backscatter and contrast relative to the bright surrounding normal dermis. In fluorescence, a nuclear-specific contrast agent may be used and light emission collected specifically from nuclei but almost none from the dermis. Acridine orange of concentration 1 mM stains nuclei in 20 s with high specificity and strongly enhances nuclear-to-dermis contrast of BCCs. Comparison of fluorescence mosaics to histology shows that both large and small tumors are detectable. The results demonstrate the feasibility of confocal mosaicing microscopy toward rapid surgical pathology to potentially expedite and guide surgery.

Noninvasive imaging of melanoma with reflectance mode confocal scanning laser microscopy in a murine model.

A reflectance-mode confocal scanning laser microscope (rCSLM) was developed for imaging early-stage melanoma in a living mouse model without the addition of exogenous contrast agents. Lesions were first located by surveying the dorsum with a polarized light camera, then imaged with the rCSLM. The images demonstrated two characteristics of melanoma in this animal model: (1) melanocytes and apparent tumor nests in the epidermis at the stratum spinosum in a state of pagetoid spread and (2) architectural disruption of the dermal-epidermal junction. The epidermal melanocytes and apparent tumor nests had a high melanin content, which caused their reflectance to be fivefold greater than the surrounding epidermis. The rCSLM images illustrate the difference between normal skin and sites with apparent melanoma. This imaging modality shows promise to track the progression of melanoma lesions in animal models.

Imaging melanoma in a murine model using reflectance-mode confocal scanning laser microscopy and polarized light imaging.

The light-scattering properties of cutaneous tissues provide optical contrast for imaging the presence and depth of pigmented melanoma in a highly pigmented murine model, the C57/B6 mouse. Early lesions are difficult to identify when viewing black lesions on a black mouse. Two methods were used to image early lesions in this model. (1) A reflectance-mode confocal scanning laser microscope (rCSLM) was built to provide horizontal images (x-y at depth z) and transverse images (x-z at position y) non-invasively in the living mouse. (2) A polarized light imaging (PLI) camera was built using a linearly polarized white light source that viewed the skin through an analyzing linear polarizer oriented either parallel or perpendicular to the illumination's polarization to yield two images, "PAR" and "PER," respectively. The difference image, PAR-PER, eliminated multiply scattered light and yielded an image of the superficial but subsurface tissues based only on photons scattered once or a few times so as to retain their polarization. rCSLM could image melanoma lesions developing below the epidermis. PLI could distinguish superficial from deeper melanoma lesions because the melanin of the superficial lesions attenuated the PAR-PER image, whereas deeper lesions failed to attenuate the PAR-PER image.