Artificial intelligence algorithms and 3D volumetric rendering for basal cell carcinoma detection and tumor depth assessment in RCM-OCT images: a pilot study.

The Reflectance Confocal Microscopy - Optical Coherence Tomography (RCM-OCT) device has shown utility in detecting and assessing depth of basal cell carcinoma (BCC) in vivo but is challenging for novices to interpret. Artificial intelligence (AI) applied to RCM-OCT could aid readers. We trained artificial intelligence (AI) models, using OCT rasters of biopsy-confirmed BCC, to detect and create 3D BCC rendering and automatically measure tumor depth. Trained AI models were applied to a separate test set containing rasters of BCC, benign lesions, and normal skin. Blinded reader analysis and tumor depth correlation with histopathology were conducted. BCC detection improved from viewing OCT rasters only (sensitivity 73.3%, specificity 45.5%) to viewing rasters with AI-generated BCC rendering (sensitivity 86.7%, specificity 48.5%). A Pearson Correlation r2 = 0.59 (p=0.02) was achieved for the tumor depth measurement between AI and histologic measured depths. Thus, addition of AI to the RCM-OCT device may expand its utility widely.

A Prospective Double-Blinded Comparison of Reflectance Confocal Microscopy with Conventional Histopathology for In Vivo Assessment in Oral Cancer.

PURPOSE: We investigated reflectance confocal microscopy (RCM) as a possible noninvasive approach for the diagnosis of cancer and real-time assessment of surgical margins. EXPERIMENTAL DESIGN: In a phase I study on 20 patients, we established the RCM imaging morphologic features that distinguish oral squamous cell carcinoma (OSCC) from normal tissue with a newly developed intraoral RCM probe. Our subsequent phase II prospective double-blinded study in 60 patients tested the diagnostic accuracy of RCM against histopathology. Five RCM videos from the tumor and five from normal surrounding mucosa were collected on each patient, followed by a 3-mm punch biopsy of the imaged area. An experienced RCM reader, who was blinded to biopsy location and histologic diagnosis, examined the videos from both regions and classified each as "tumor" or "not tumor" based on RCM features established in phase I. Hematoxylin and eosin slides from the biopsies were read by a pathologist who was blinded to RCM results. Using histology as the gold standard, we calculated the sensitivity and specificity of RCM. RESULTS: We report a high agreement between the blinded readers (95% for normal tissue and 81.7% for tumors), high specificity (98.3%) and negative predictive values (96.6%) for normal tissue identification, and high sensitivity (90%) and positive predictive values (88.2%) for tumor detection. CONCLUSIONS: RCM imaging is a promising technology for noninvasive in vivo diagnosis of OSCC and for real-time intraoperative evaluation of mucosal surgical margins. Its inherent constraint, however, stems from the diminished capability to evaluate structures located at more substantial depths within the tissue.

Expert Agreement on the Presence and Spatial Localization of Melanocytic Features in Dermoscopy.

Dermoscopy aids in melanoma detection; however, agreement on dermoscopic features, including those of high clinical relevance, remains poor. In this study, we attempted to evaluate agreement among experts on exemplar images not only for the presence of melanocytic-specific features but also for spatial localization. This was a cross-sectional, multicenter, observational study. Dermoscopy images exhibiting at least 1 of 31 melanocytic-specific features were submitted by 25 world experts as exemplars. Using a web-based platform that allows for image markup of specific contrast-defined regions (superpixels), 20 expert readers annotated 248 dermoscopic images in collections of 62 images. Each collection was reviewed by five independent readers. A total of 4,507 feature observations were performed. Good-to-excellent agreement was found for 14 of 31 features (45.2%), with eight achieving excellent agreement (Gwet's AC >0.75) and seven of them being melanoma-specific features. These features were peppering/granularity (0.91), shiny white streaks (0.89), typical pigment network (0.83), blotch irregular (0.82), negative network (0.81), irregular globules (0.78), dotted vessels (0.77), and blue-whitish veil (0.76). By utilizing an exemplar dataset, a good-to-excellent agreement was found for 14 features that have previously been shown useful in discriminating nevi from melanoma. All images are public (www.isic-archive.com) and can be used for education, scientific communication, and machine learning experiments.

Paradoxical effect of epinephrine on lesion redness and vascularity.

INTRODUCTION: Epinephrine is commonly used in combination with local anesthetic (lidocaine/epinephrine) due to its beneficial vasoconstrictive properties. Typically, pallor is appreciated after injection as a sign of effect; however, we observed that some cutaneous malignancies paradoxically revealed increased redness and vascularity after injection of lidocaine/epinephrine. In this study, we investigate this phenomenon among a series of biopsied lesions to identify characteristics of lesions associated with increased redness and/or vascularity. OBJECTIVES: To determine characteristics of lesions which become redder or more vascular after injection with lidocaine/epinephrine prior to biopsy. METHODS: This cross-sectional study consisted of a convenience sample of lesions scheduled for biopsy. Lesions were photographed prior to and 7 min after injection of lidocaine/epinephrine as a part of standard care. Two readers blinded to study objectives and histopathological diagnosis assessed lesions for changes in redness and vascular features. RESULTS: Fifty-four lesions from 47 patients-61.7% male, mean age 64.8 years, age-range 24-91 were included. Thirty-six lesions were biopsy confirmed malignant, with 5 in situ and 31 invasive malignancies; the remaining 18 lesions were benign. In comparison with non-malignant lesions, malignant lesions were associated with an increase in clinically appreciable vascular features after injection of lidocaine/epinephrine, X2 (1) = 21.600, p < 0.001. Further stratification into benign, in situ, and invasive lesions strengthened the association, X2 (1) = 23.272, p < 0.001. CONCLUSIONS: Combination lidocaine/epinephrine has been shown to paradoxically increase the visibility of vessels seen in cutaneous malignancies. This is consistent with prior literature suggesting aberrant adrenergic signaling in neoangiogenic vessels.

Improving Artificial Intelligence-Based Diagnosis on Pediatric Skin Lesions.

Artificial intelligence algorithms to classify melanoma are dependent on their training data, which limits generalizability. The objective of this study was to compare the performance of an artificial intelligence model trained on a standard adult-predominant dermoscopic dataset before and after the addition of additional pediatric training images. The performances were compared using held-out adult and pediatric test sets of images. We trained two models: one (model A) on an adult-predominant dataset (37,662 images from the International Skin Imaging Collaboration) and the other (model A+P) on an additional 1,536 pediatric images. We compared performance between the two models on adult and pediatric held-out test images separately using the area under the receiver operating characteristic curve. We then used Gradient-weighted Class Activation Maps and background skin masking to understand the contributions of the lesion versus background skin to algorithm decision making. Adding images from a pediatric population with different epidemiological and visual patterns to current reference standard datasets improved algorithm performance on pediatric images without diminishing performance on adult images. This suggests a way that dermatologic artificial intelligence models can be made more generalizable. The presence of background skin was important to the pediatric-specific improvement seen between models. Our study highlights the importance of carefully curated and labeled data from diverse inputs to improve the generalizability of AI models for dermatology, in this case applied to dermoscopic images of adult and pediatric lesions to improve melanoma detection.

Lentigo maligna melanoma mapping using reflectance confocal microscopy correlates with staged excision: A prospective study.

BACKGROUND: Lentigo maligna/lentigo maligna melanoma (LM/LMM) can present with subclinical extension that may be difficult to define preoperatively and lead to incomplete excision and potential recurrence. Preliminarily studies have used reflectance confocal microscopy (RCM) to assess LM/LMM margins. OBJECTIVE: To evaluate the correlation of LM/LMM subclinical extension defined by RCM compared with the gold standard histopathology. METHODS: Prospective study of LM/LMM patients referred for dermatologic surgery. RCM was performed at the clinically defined initial surgical margin followed by margin-controlled staged excision with paraffin-embedded tissue, and histopathology was correlated with RCM results. RESULTS: Seventy-two patients were included. Mean age was 66.8 years (standard deviation, 11.1; range, 38-89); 69.4% were men. Seventy of 72 lesions (97.2%) were located on the head and neck with mean largest clinical diameter of 1.3 cm (range, 0.3-5). Diagnostic accuracy for detection of residual melanoma in the tumor debulk (after biopsy) had a sensitivity of 96.7% and a specificity of 66.7% when compared with histopathology. RCM margin assessment revealed an overall agreement with final histopathology of 85.9% (κ = 0.71; P < .001). LIMITATIONS: No RCM imaging beyond initial planned margins was performed. CONCLUSION: RCM showed moderate to excellent overall agreement between RCM imaging of LM/LMM and histopathology of staged excision margins.

Quantitative collagen analysis using second harmonic generation images for the detection of basal cell carcinoma with ex vivo multiphoton microscopy.

Basal cell carcinoma (BCC) is the most common skin cancer, and its incidence is rising. Millions of benign biopsies are performed annually for BCC diagnosis, increasing morbidity, and healthcare costs. Non-invasive in vivo technologies such as multiphoton microscopy (MPM) can aid in diagnosing BCC, reducing the need for biopsies. Furthermore, the second harmonic generation (SHG) signal generated from MPM can classify and prognosticate cancers based on extracellular matrix changes, especially collagen type I. We explored the potential of MPM to differentiate collagen changes associated with different BCC subtypes compared to normal skin structures and benign lesions. Quantitative analysis such as frequency band energy analysis in Fourier domain, CurveAlign and CT-FIRE fibre analysis was performed on SHG images from 52 BCC and 12 benign lesions samples. Our results showed that collagen distribution is more aligned surrounding BCCs nests compared to the skin's normal structures (p < 0.001) and benign lesions (p < 0.001). Also, collagen was orientated more parallelly surrounding indolent BCC subtypes (superficial and nodular) versus those with more aggressive behaviour (infiltrative BCC) (p = 0.021). In conclusion, SHG signal from type I collagen can aid not only in the diagnosis of BCC but could be useful for prognosticating these tumors. Our initial results are limited to a small number of samples, requiring large-scale studies to validate them. These findings represent the groundwork for future in vivo MPM for diagnosis and prognosis of BCC.

Tertiary lymphoid structures accompanied by fibrillary matrix morphology impact anti-tumor immunity in basal cell carcinomas.

Tertiary lymphoid structures (TLS) are specialized lymphoid formations that serve as local repertoire of T- and B-cells at sites of chronic inflammation, autoimmunity, and cancer. While presence of TLS has been associated with improved response to immune checkpoint blockade therapies and overall outcomes in several cancers, its prognostic value in basal cell carcinoma (BCC) has not been investigated. Herein, we determined the prognostic impact of TLS by relating its prevalence and maturation with outcome measures of anti-tumor immunity, namely tumor infiltrating lymphocytes (TILs) and tumor killing. In 30 distinct BCCs, we show the presence of TLS was significantly enriched in tumors harboring a nodular component and more mature primary TLS was associated with TIL counts. Moreover, assessment of the fibrillary matrix surrounding tumors showed discrete morphologies significantly associated with higher TIL counts, critically accounting for heterogeneity in TIL count distribution within TLS maturation stages. Specifically, increased length of fibers and lacunarity of the matrix with concomitant reduction in density and alignment of fibers were present surrounding tumors displaying high TIL counts. Given the interest in inducing TLS formation as a therapeutic intervention as well as its documented prognostic value, elucidating potential impediments to the ability of TLS in driving anti-tumor immunity within the tumor microenvironment warrants further investigation. These results begin to address and highlight the need to integrate stromal features which may present a hindrance to TLS formation and/or effective function as a mediator of immunotherapy response.

In vivo tumor immune microenvironment phenotypes correlate with inflammation and vasculature to predict immunotherapy response.

Response to immunotherapies can be variable and unpredictable. Pathology-based phenotyping of tumors into 'hot' and 'cold' is static, relying solely on T-cell infiltration in single-time single-site biopsies, resulting in suboptimal treatment response prediction. Dynamic vascular events (tumor angiogenesis, leukocyte trafficking) within tumor immune microenvironment (TiME) also influence anti-tumor immunity and treatment response. Here, we report dynamic cellular-level TiME phenotyping in vivo that combines inflammation profiles with vascular features through non-invasive reflectance confocal microscopic imaging. In skin cancer patients, we demonstrate three main TiME phenotypes that correlate with gene and protein expression, and response to toll-like receptor agonist immune-therapy. Notably, phenotypes with high inflammation associate with immunostimulatory signatures and those with high vasculature with angiogenic and endothelial anergy signatures. Moreover, phenotypes with high inflammation and low vasculature demonstrate the best treatment response. This non-invasive in vivo phenotyping approach integrating dynamic vasculature with inflammation serves as a reliable predictor of response to topical immune-therapy in patients.

Analyzing the Spatial Randomness in the Distribution of Acquired Melanocytic Neoplasms.

On the basis of the clinical impression and current knowledge, acquired melanocytic nevi and melanomas may not occur in random localizations. The goal of this study was to identify whether their distribution on the back is random and whether the location of melanoma correlates with its adjacent lesions. Therefore, patient-level and lesion-level spatial analyses were performed using the Clark‒Evans test for complete spatial randomness. A total of 311 patients with three-dimensional total body photography (average age of 40.08 [30‒49] years; male/female ratio: 128/183) with 5,108 eligible lesions in total were included in the study (mean sum of eligible lesions per patient of 16.42 [3‒199]). The patient-level analysis revealed that the distributions of acquired melanocytic neoplasms were more likely to deviate toward clustering than dispersion (average z-score of ‒0.55 [95% confidence interval = ‒0.69 to ‒0.41; P < 0.001]). The lesion-level analysis indicated a higher portion of melanomas (n = 57 of 72, 79.2% [95% confidence interval = 69.4‒88.9%]) appearing in proximity to neighboring melanocytic neoplasms than to nevi (n = 2,281 of 5,036, 45.3% [95% confidence interval = 43.9‒46.7%]). In conclusion, the nevi and melanomas' distribution on the back tends toward clustering as opposed to dispersion. Furthermore, melanomas are more likely to appear proximally to their neighboring neoplasms than to nevi. These findings may justify various oncogenic theories and improve diagnostic methodology.

Combined PARP1-Targeted Nuclear Contrast and Reflectance Contrast Enhance Confocal Microscopic Detection of Basal Cell Carcinoma.

Reflectance confocal microscopy (RCM) with endogenous backscattered contrast can noninvasively image basal cell carcinomas (BCCs) in skin. However, BCCs present with high nuclear density, and the relatively weak backscattering from nuclei imposes a fundamental limit on contrast, detectability, and diagnostic accuracy. We investigated PARPi-FL, an exogenous nuclear poly(adenosine diphosphate ribose) polymerase (PARP1)-targeted fluorescent contrast agent, and fluorescence confocal microscopy toward improving BCC diagnosis. Methods: We tested PARP1 expression in 95 BCC tissues using immunohistochemistry, followed by PARPi-FL staining in 32 fresh surgical BCC specimens. The diagnostic accuracy of PARPi-FL contrast was evaluated in 83 surgical specimens. The optimal parameters for permeability of PARPi-FL through intact skin was tested ex vivo on 5 human skin specimens and in vivo in 3 adult Yorkshire pigs. Results: We found significantly higher PARP1 expression and PARPi-FL binding in BCCs than in normal skin structures. Blinded reading of RCM-and-fluorescence confocal microscopy images by 2 experts demonstrated a higher diagnostic accuracy for BCCs with combined fluorescence and reflectance contrast than for RCM alone. Optimal parameters (time and concentration) for PARPi-FL transepidermal permeation through intact skin were successfully determined. Conclusion: Combined fluorescence and reflectance contrast may improve noninvasive BCC diagnosis with confocal microscopy.

Deep Learning for Basal Cell Carcinoma Detection for Reflectance Confocal Microscopy.

Basal cell carcinoma (BCC) is the most common skin cancer, with over 2 million cases diagnosed annually in the United States. Conventionally, BCC is diagnosed by naked eye examination and dermoscopy. Suspicious lesions are either removed or biopsied for histopathological confirmation, thus lowering the specificity of noninvasive BCC diagnosis. Recently, reflectance confocal microscopy, a noninvasive diagnostic technique that can image skin lesions at cellular level resolution, has shown to improve specificity in BCC diagnosis and reduced the number needed to biopsy by 2-3 times. In this study, we developed and evaluated a deep learning-based artificial intelligence model to automatically detect BCC in reflectance confocal microscopy images. The proposed model achieved an area under the curve for the receiver operator characteristic curve of 89.7% (stack level) and 88.3% (lesion level), a performance on par with that of reflectance confocal microscopy experts. Furthermore, the model achieved an area under the curve of 86.1% on a held-out test set from international collaborators, demonstrating the reproducibility and generalizability of the proposed automated diagnostic approach. These results provide a clear indication that the clinical deployment of decision support systems for the detection of BCC in reflectance confocal microscopy images has the potential for optimizing the evaluation and diagnosis of patients with skin cancer.

Skin strata delineation in reflectance confocal microscopy images using recurrent convolutional networks with attention.

Reflectance confocal microscopy (RCM) is an effective non-invasive tool for cancer diagnosis. However, acquiring and reading RCM images requires extensive training and experience, and novice clinicians exhibit high discordance in diagnostic accuracy. Quantitative tools to standardize image acquisition could reduce both required training and diagnostic variability. To perform diagnostic analysis, clinicians collect a set of RCM mosaics (RCM images concatenated in a raster fashion to extend the field view) at 4-5 specific layers in skin, all localized in the junction between the epidermal and dermal layers (dermal-epidermal junction, DEJ), necessitating locating that junction before mosaic acquisition. In this study, we automate DEJ localization using deep recurrent convolutional neural networks to delineate skin strata in stacks of RCM images collected at consecutive depths. Success will guide to automated and quantitative mosaic acquisition thus reducing inter operator variability and bring standardization in imaging. Testing our model against an expert labeled dataset of 504 RCM stacks, we achieved [Formula: see text] classification accuracy and nine-fold reduction in the number of anatomically impossible errors compared to the previous state-of-the-art.

Deep Learning-Based Denoising in High-Speed Portable Reflectance Confocal Microscopy.

BACKGROUND AND OBJECTIVE: Portable confocal microscopy (PCM) is a low-cost reflectance confocal microscopy technique that can visualize cellular details of human skin in vivo. When PCM images are acquired with a short exposure time to reduce motion blur and enable real-time 3D imaging, the signal-to-noise ratio (SNR) is decreased significantly, which poses challenges in reliably analyzing cellular features. In this paper, we evaluated deep learning (DL)-based approach for reducing noise in PCM images acquired with a short exposure time. STUDY DESIGN/MATERIALS AND METHODS: Content-aware image restoration (CARE) network was trained with pairs of low-SNR input and high-SNR ground truth PCM images obtained from 309 distinctive regions of interest (ROIs). Low-SNR input images were acquired from human skin in vivo at the imaging speed of 180 frames/second. The high-SNR ground truth images were generated by registering 30 low-SNR input images obtained from the same ROI and summing them. The CARE network was trained using the Google Colaboratory Pro platform. The denoising performance of the trained CARE network was quantitatively and qualitatively evaluated by using image pairs from 45 unseen ROIs. RESULTS: CARE denoising improved the image quality significantly, increasing similarity with the ground truth image by 1.9 times, reducing noise by 2.35 times, and increasing SNR by 7.4 dB. Banding noise, prominent in input images, was significantly reduced in CARE denoised images. CARE denoising provided quantitatively and qualitatively better noise reduction than non-DL filtering methods. Qualitative image assessment by three confocal readers showed that CARE denoised images exhibited negligible noise more often than input images and non-DL filtered images. CONCLUSIONS: Results showed the potential of using a DL-based method for denoising PCM images obtained at a high imaging speed. The DL-based denoising method needs to be further trained and tested for PCM images obtained from disease-suspicious skin lesions.

Semantic segmentation of reflectance confocal microscopy mosaics of pigmented lesions using weak labels.

Reflectance confocal microscopy (RCM) is a non-invasive imaging tool that reduces the need for invasive histopathology for skin cancer diagnoses by providing high-resolution mosaics showing the architectural patterns of skin, which are used to identify malignancies in-vivo. RCM mosaics are similar to dermatopathology sections, both requiring extensive training to interpret. However, these modalities differ in orientation, as RCM mosaics are horizontal (parallel to the skin surface) while histopathology sections are vertical, and contrast mechanism, RCM with a single (reflectance) mechanism resulting in grayscale images and histopathology with multi-factor color-stained contrast. Image analysis and machine learning methods can potentially provide a diagnostic aid to clinicians to interpret RCM mosaics, eventually helping to ease the adoption and more efficiently utilizing RCM in routine clinical practice. However standard supervised machine learning may require a prohibitive volume of hand-labeled training data. In this paper, we present a weakly supervised machine learning model to perform semantic segmentation of architectural patterns encountered in RCM mosaics. Unlike more widely used fully supervised segmentation models that require pixel-level annotations, which are very labor-demanding and error-prone to obtain, here we focus on training models using only patch-level labels (e.g. a single field of view within an entire mosaic). We segment RCM mosaics into "benign" and "aspecific (nonspecific)" regions, where aspecific regions represent the loss of regular architecture due to injury and/or inflammation, pre-malignancy, or malignancy. We adopt Efficientnet, a deep neural network (DNN) proven to accurately accomplish classification tasks, to generate class activation maps, and use a Gaussian weighting kernel to stitch smaller images back into larger fields of view. The trained DNN achieved an average area under the curve of 0.969, and Dice coefficient of 0.778 showing the feasibility of spatial localization of aspecific regions in RCM images, and making the diagnostics decision model more interpretable to the clinicians.