International Skin Imaging Collaboration-Designated Diagnoses (ISIC-DX): Consensus terminology for lesion diagnostic labeling.

BACKGROUND: A common terminology for diagnosis is critically important for clinical communication, education, research and artificial intelligence. Prevailing lexicons are limited in fully representing skin neoplasms. OBJECTIVES: To achieve expert consensus on diagnostic terms for skin neoplasms and their hierarchical mapping. METHODS: Diagnostic terms were extracted from textbooks, publications and extant diagnostic codes. Terms were hierarchically mapped to super-categories (e.g. 'benign') and cellular/tissue-differentiation categories (e.g. 'melanocytic'), and appended with pertinent-modifiers and synonyms. These terms were evaluated using a modified-Delphi consensus approach. Experts from the International-Skin-Imaging-Collaboration (ISIC) were surveyed on agreement with terms and their hierarchical mapping; they could suggest modifying, deleting or adding terms. Consensus threshold was >75% for the initial rounds and >50% for the final round. RESULTS: Eighteen experts completed all Delphi rounds. Of 379 terms, 356 (94%) reached consensus in round one. Eleven of 226 (5%) benign-category terms, 6/140 (4%) malignant-category terms and 6/13 (46%) indeterminate-category terms did not reach initial agreement. Following three rounds, final consensus consisted of 362 terms mapped to 3 super-categories and 41 cellular/tissue-differentiation categories. CONCLUSIONS: We have created, agreed upon, and made public a taxonomy for skin neoplasms and their hierarchical mapping. Further study will be needed to evaluate the utility and completeness of the lexicon.

Diagnosis of invasive cutaneous squamous cell carcinoma, imaging and staging.

The assessment of patients with a lesion raising the suspicion of an invasive cutaneous squamous cell carcinoma (cSCC) is a frequent clinical scenario. The management of patients with cSCC is a multistep approach, starting with the correct diagnosis. The two main diagnostic goals are to differentiate from other possible diagnoses and correctly recognize the lesion as cSCC, and then to determine the tumor spread (perform staging), that is if the patient has a common primary cSCC or a locally advanced cSCC, or a metastatic cSCC (with in-transit, regional lymph nodal, or rarely distant metastasis). The multistep diagnostic approach begins with the clinical characteristics of the primary cSCC, it is complemented with features with dermoscopy and, if available, reflectance confocal microscopy and is confirmed with histopathology. The tumor spread is assessed by physical examination and, in some cases, ultrasound and/or computed tomography or magnetic resonance imaging, mainly to investigate for regional lymph node metastasis or for local infiltration into deeper structures. In the last step, the clinical, histologic and radiologic findings are incorporated into staging systems.

Use of Dermoscopy among Greek Dermatologists in Everyday Clinical Practice: A National Questionnaire-Based Study.

Background: Dermatoscopy has been established as an important diagnostic tool for a wide range of skin diseases. This study aims to evaluate the use of dermatoscopy in clinical practice among Greek dermatologists. Methods: A nationwide questionnaire-based survey was conducted collecting data on the frequency of dermatoscopic examinations, the types of lesions examined, training and educational resources, as well as factors influencing the choice to incorporate dermatoscopy into daily clinical routines. Results: A total of 366 Greek dermatologists participated in the survey. Most of the respondents reported the daily use of dermatoscopy in their practice. Pigmented and non-pigmented lesions, inflammatory diseases, cutaneous infectious, hair disorders, and nail lesions were the most common indications for dermatoscopy. Factors influencing the utilization of dermatoscopy included increased diagnostic accuracy, enhanced patient care, better patient communication and general compliance, and improved satisfaction among dermatologists. Conclusions: This national questionnaire-based study demonstrates that dermatoscopy has become an integral part of daily dermatological practice in Greece. The findings highlight the significance of structured training and education to promote dermoscopy's effective and routine use. Incorporating dermatoscopy into clinical practice not only improves diagnostic precision but also enhances patient care, contributing to the overall quality of dermatological services in Greece.

Expert Agreement on the Presence and Spatial Localization of Melanocytic Features in Dermoscopy.

Dermoscopy aids in melanoma detection; however, agreement on dermoscopic features, including those of high clinical relevance, remains poor. In this study, we attempted to evaluate agreement among experts on exemplar images not only for the presence of melanocytic-specific features but also for spatial localization. This was a cross-sectional, multicenter, observational study. Dermoscopy images exhibiting at least 1 of 31 melanocytic-specific features were submitted by 25 world experts as exemplars. Using a web-based platform that allows for image markup of specific contrast-defined regions (superpixels), 20 expert readers annotated 248 dermoscopic images in collections of 62 images. Each collection was reviewed by five independent readers. A total of 4,507 feature observations were performed. Good-to-excellent agreement was found for 14 of 31 features (45.2%), with eight achieving excellent agreement (Gwet's AC >0.75) and seven of them being melanoma-specific features. These features were peppering/granularity (0.91), shiny white streaks (0.89), typical pigment network (0.83), blotch irregular (0.82), negative network (0.81), irregular globules (0.78), dotted vessels (0.77), and blue-whitish veil (0.76). By utilizing an exemplar dataset, a good-to-excellent agreement was found for 14 features that have previously been shown useful in discriminating nevi from melanoma. All images are public (www.isic-archive.com) and can be used for education, scientific communication, and machine learning experiments.

Perifollicular linear projections: A dermatoscopic criterion for the diagnosis of lentigo maligna on the face.

BACKGROUND: Lentigo maligna (LM) can mimic benign, flat, pigmented lesions and can be challenging to diagnose. OBJECTIVE: To describe a new dermatoscopic feature termed "perifollicular linear projections (PLP)" as a diagnostic criterion for LM on the face. METHODS: Retrospective study on reflectance confocal microscopy and dermatoscopy images of flat facial pigmented lesions originating from 2 databases. PLP were defined as short, linear, pigmented projections emanating from hair follicles. Dermatoscopy readers were blinded to the final histopathologic diagnosis. RESULTS: From 83 consecutive LMs, 21/83 (25.3%) displayed "bulging of hair follicles" on reflectance confocal microscopy and 18 of these 21 (85.7%), displayed PLP on dermatoscopy. From a database of 2873 consecutively imaged and biopsied lesions, 252 flat-pigmented facial lesions were included. PLP was seen in 47/76 melanomas (61.8%), compared with 7/176 lesions (3.9%) with other diagnosis (P < .001). The sensitivity was 61.8% (95% CI, 49.9%-72.7%), specificity 96.0% (95% CI, 92.9%-98.4%). PLP was independently associated with LM diagnosis on multivariate analysis (OR 26.1 [95% CI, 9.6%-71.0]). LIMITATIONS: Retrospective study. CONCLUSION: PLP is a newly described dermatoscopic criterion that may add specificity and sensitivity to the early diagnosis of LM located on the face. We postulate that PLP constitutes an intermediary step in the LM progression model.

Seven Plus One Steps to Assess Pigmented Nail Bands (Melanonychia Striata Longitudinalis).

Melanonychia striata longitudinalis might involve one or more fingers and/or toes and might result from several different causes, including benign and malignant tumors, trauma, infections, and activation of melanocytes that might be reactive or related to the pigmentary trait, drugs and some rare syndromes. This broad differential diagnosis renders the clinical assessment of melanonychia striata particularly challenging. Nail matrix melanoma is relatively rare, occurs almost always in adults involves more frequently the first toe or thumb. The most common nail unit cancer, squamous cell carcinoma / Bowen disease (SCC) of the nail matrix is seldom pigmented. Histopathologic examination remains the gold standard for melanoma and SCC diagnosis, but excisional or partial biopsies from the nail matrix require training and is not routinely performed by the majority of clinicians. Furthermore, the histopathologic evaluation of melanocytic lesions of the nail matrix is particularly challenging, since early melanoma has only bland histopathologic alterations. Dermatoscopy of the nail plate and its free edge significantly improves the clinical diagnosis, since specific patterns have been associated to each one of the causes of melanonychia. Based on knowledge generated and published in the last decades, we propose herein a stepwise diagnostic approach for melanonychia striata longitudinalis: 1) Hemorrhage first 2) Age matters 3) Number of nails matters 4) Free edge matters 5) Brown or gray? 6) Size matters 7) Regular or irregular and, finally, "follow back".

The Natural Evolution of Nevi with Peripheral Globules.

BACKGROUND: Peripheral globules (PG) in melanocytic lesions represent a concerning dermoscopic feature since they might be present in growing nevi and melanomas. Their natural evolution has not been fully elucidated, and an age-based management approach has been recommended. OBJECTIVES: The aim of this study was to calculate the growth rate of lesions with PG and investigate possible association with age, sex, location, and the global dermoscopic pattern. METHODS: We retrospectively selected the lesions of interest from a cohort of Caucasian patients who underwent sequential digital dermoscopy monitoring. Lesions with PG distributed at 75% or more of their circumference with available follow-up images or histopathologic report were included. The surface area was automatically calculated with the help of an incorporated tool used in the acquisition of the images. The images were also evaluated by independent investigators for the presence of pre-defined criteria. Growth-curve models were used to assess the growth rate. The outcome variable was the area of nevi in mm2, and scatterplots with Lowess curves were used to present the mean change of nevi during follow-up. RESULTS: A total of 208 lesions from 98 patients with a median age of 36 years (range 15-75) were included. The median follow-up time was 18 months (range 4-48). The mean growth rate for all nevi was 0.16 mm2/month (95% CI, 0.14-0.18, p < 0.001), ranging from -0.29 to 0.61 mm2/month. The growth rate was higher in nevi with a homogeneous dermoscopic pattern (p < 0.001). The number of peripheral globules during follow-up varied from increasing to complete disappearance. None of the lesions developed any melanoma-specific structure at follow-up. CONCLUSION: Nevi with PG grew at a mean rate of 0.16 mm2/month, and the growth rate was independent of age, gender, or anatomic location. Nevi with homogeneous pattern demonstrated the highest growth rate in our cohort. None of the monitored nevi with PG developed melanoma-specific criteria at follow-up.

Eccrine Porocarcinoma: A Review of the Literature.

Eccrine porocarcinoma (EPC) constitutes a rare malignant adnexal tumor, which accounts for about 0.005-0.01% of all cutaneous malignancies. It may develop de novo or arise from an eccrine poroma, after a latency period of years or even decades. Accumulating data suggest that specific oncogenic drivers and signaling pathways may be implicated in its tumorigenesis, while recent data have demonstrated a high overall mutation rate attributed to UV exposure. Diagnosis may be challenging and should rely on the combination of clinical, dermoscopical, histopathological and immunohistochemical findings. The literature is controversial regarding tumor behavior and prognosis and, therefore, there is no consensus on its surgical management, utility of lymph-node biopsy and further adjuvant or systemic treatment. However, recent advances in tumorigenesis of EPC may aid in the development of novel treatment strategies, which could improve survival of advanced or metastatic disease, such as immunotherapy. This review presents an update of the epidemiology, pathogenesis and clinical presentation of EPC and summarizes current data on diagnostic evaluation and management of this rare cutaneous malignancy.

Clinical and Dermoscopic Characteristics of Cutaneous Sarcomas: A Literature Review.

Under the umbrella of cutaneous sarcomas (CS) we include a heterogeneous group of rare, malignant, mesenchymal neoplasia, such as dermatofibrosarcoma protuberans, atypical fibroxanthoma, cutaneous undifferentiated pleomorphic sarcoma, cutaneous angiosarcoma and leiomyosarcoma. Clinical presentation and histopathological examination are the cornerstone of CS diagnosis and classification. There are scarce data in the literature in regards to the clinical and dermatoscopic characteristics of CS and the role of dermatoscopy in their early identification. We performed a literature review, aiming to summarize current data on the clinical and dermatoscopic presentation of the most common types of cutaneous sarcomas that may facilitate early diagnosis and prompt management. Based on the available published data, CS are characterized by mostly unspecific dermatoscopic patterns. Dermatofibrosarcoma protuberans, Kaposi's sarcoma, and in a lesser degree, cutaneous angiosarcoma, may display distinct dermatoscopic features, facilitating their early clinical recognition. In conclusion, dermatoscopy, in conjunction with the overall clinical context, may aid towards suspicion of CS.

Dermoscopic Clues of Histopathologically Aggressive Basal Cell Carcinoma Subtypes.

Background: The group of histopathologically aggressive BCC subtypes includes morpheaform, micronodular, infiltrative and metatypical BCC. Since these tumors are at increased risk of recurring, micrographically controlled surgery is considered the best therapeutic option. Although dermoscopy significantly improves the clinical recognition of BCC, scarce evidence exists on their dermoscopic criteria. Aim: To investigate the dermoscopic characteristics of histopathologically aggressive BCC subtypes. Materials and Methods: Dermoscopic images of morpheaform, micronodular, infiltrative and metatypical BCC were analyzed for the presence of predefined variables. Descriptive and analytical statistics were performed. Results: Most histopathologically aggressive BCCs were located on the head and neck. Infiltrative was the most common subtype. All subtypes, except micronodular BCC, rarely displayed dermoscopic pigmentation. The most frequent dermoscopic features of infiltrative BCC were arborizing vessels (67.1%), shiny white structures (48.6%) and ulceration (52.9%). The features prevailing in morpheaform BCC were arborizing vessels (68.4%), ulceration (n = 12, 63.2%) and white porcelain areas (47.4%). Micronodular BCC was typified by milky red structureless areas (53.8%), arborizing vessels (53.8%), short fine telangiectasias (50%), ulceration (46.2%) and blue structures (57.7%). The most common findings in metatypical BCC were arborizing vessels (77.8%), shiny white structures (66.7%), ulceration (62.9%) and keratin mass (29.6%). Limitations: Study population of only white skin and relatively small sample size in some groups. Conclusions: Our study provided data on the clinical, dermoscopic and epidemiological characteristics of histopathologically aggressive BCCs.

Dermoscopy of Actinic Keratosis: Is There a True Differentiation between Non-Pigmented and Pigmented Lesions?

Dermoscopic features of actinic keratosis (AK) have been widely studied, but there is still little evidence for their diagnostic accuracy. Our study investigates whether established dermoscopic criteria are reliable predictors in differentiating non-pigmented actinic keratosis (NPAK) from pigmented actinic keratosis (PAK). For this purpose, dermoscopic images of 83 clinically diagnosed AK (45 NPAK, 38PAK) were examined, and the sensitivity (Se), specificity (Sp), positive predictive value (PPV), and negative predictive value (NPV) were assessed. Features with statistical significance were the red pseudo-network (p = 0.02) for NPAK and the pigmented pseudo-network (p < 0.001) with a pigment intensity value even less than 10% for PAK (p = 0.001). Pigmented pseudo-network (Se: 89%, Sp: 77%, PPV: 77%, NPV: 89%) with a pigment intensity value of more than 10% (Se: 90%, Sp: 86%, PPV: 79%, NPV: 93%) had excellent diagnostic accuracy for PAK. Scale and widened follicular openings with yellowish dots surrounded by white circles were equally represented in both variants of AK. Linear wavy vessels and shiny streaks were more prominently observed in NPAK, as were rosettes in PAK, but these results failed to meet statistical significance. The red starburst pattern was near statistical significance for PAK. Therefore, pigmentation is the strongest dermoscopic predictor for the differentiation between NPAK and PAK.

Paradoxical effect of epinephrine on lesion redness and vascularity.

INTRODUCTION: Epinephrine is commonly used in combination with local anesthetic (lidocaine/epinephrine) due to its beneficial vasoconstrictive properties. Typically, pallor is appreciated after injection as a sign of effect; however, we observed that some cutaneous malignancies paradoxically revealed increased redness and vascularity after injection of lidocaine/epinephrine. In this study, we investigate this phenomenon among a series of biopsied lesions to identify characteristics of lesions associated with increased redness and/or vascularity. OBJECTIVES: To determine characteristics of lesions which become redder or more vascular after injection with lidocaine/epinephrine prior to biopsy. METHODS: This cross-sectional study consisted of a convenience sample of lesions scheduled for biopsy. Lesions were photographed prior to and 7 min after injection of lidocaine/epinephrine as a part of standard care. Two readers blinded to study objectives and histopathological diagnosis assessed lesions for changes in redness and vascular features. RESULTS: Fifty-four lesions from 47 patients-61.7% male, mean age 64.8 years, age-range 24-91 were included. Thirty-six lesions were biopsy confirmed malignant, with 5 in situ and 31 invasive malignancies; the remaining 18 lesions were benign. In comparison with non-malignant lesions, malignant lesions were associated with an increase in clinically appreciable vascular features after injection of lidocaine/epinephrine, X2 (1) = 21.600, p < 0.001. Further stratification into benign, in situ, and invasive lesions strengthened the association, X2 (1) = 23.272, p < 0.001. CONCLUSIONS: Combination lidocaine/epinephrine has been shown to paradoxically increase the visibility of vessels seen in cutaneous malignancies. This is consistent with prior literature suggesting aberrant adrenergic signaling in neoangiogenic vessels.

Agreement Between Experts and an Untrained Crowd for Identifying Dermoscopic Features Using a Gamified App: Reader Feasibility Study.

BACKGROUND: Dermoscopy is commonly used for the evaluation of pigmented lesions, but agreement between experts for identification of dermoscopic structures is known to be relatively poor. Expert labeling of medical data is a bottleneck in the development of machine learning (ML) tools, and crowdsourcing has been demonstrated as a cost- and time-efficient method for the annotation of medical images. OBJECTIVE: The aim of this study is to demonstrate that crowdsourcing can be used to label basic dermoscopic structures from images of pigmented lesions with similar reliability to a group of experts. METHODS: First, we obtained labels of 248 images of melanocytic lesions with 31 dermoscopic "subfeatures" labeled by 20 dermoscopy experts. These were then collapsed into 6 dermoscopic "superfeatures" based on structural similarity, due to low interrater reliability (IRR): dots, globules, lines, network structures, regression structures, and vessels. These images were then used as the gold standard for the crowd study. The commercial platform DiagnosUs was used to obtain annotations from a nonexpert crowd for the presence or absence of the 6 superfeatures in each of the 248 images. We replicated this methodology with a group of 7 dermatologists to allow direct comparison with the nonexpert crowd. The Cohen κ value was used to measure agreement across raters. RESULTS: In total, we obtained 139,731 ratings of the 6 dermoscopic superfeatures from the crowd. There was relatively lower agreement for the identification of dots and globules (the median κ values were 0.526 and 0.395, respectively), whereas network structures and vessels showed the highest agreement (the median κ values were 0.581 and 0.798, respectively). This pattern was also seen among the expert raters, who had median κ values of 0.483 and 0.517 for dots and globules, respectively, and 0.758 and 0.790 for network structures and vessels. The median κ values between nonexperts and thresholded average-expert readers were 0.709 for dots, 0.719 for globules, 0.714 for lines, 0.838 for network structures, 0.818 for regression structures, and 0.728 for vessels. CONCLUSIONS: This study confirmed that IRR for different dermoscopic features varied among a group of experts; a similar pattern was observed in a nonexpert crowd. There was good or excellent agreement for each of the 6 superfeatures between the crowd and the experts, highlighting the similar reliability of the crowd for labeling dermoscopic images. This confirms the feasibility and dependability of using crowdsourcing as a scalable solution to annotate large sets of dermoscopic images, with several potential clinical and educational applications, including the development of novel, explainable ML tools.

Lentigo maligna melanoma mapping using reflectance confocal microscopy correlates with staged excision: A prospective study.

BACKGROUND: Lentigo maligna/lentigo maligna melanoma (LM/LMM) can present with subclinical extension that may be difficult to define preoperatively and lead to incomplete excision and potential recurrence. Preliminarily studies have used reflectance confocal microscopy (RCM) to assess LM/LMM margins. OBJECTIVE: To evaluate the correlation of LM/LMM subclinical extension defined by RCM compared with the gold standard histopathology. METHODS: Prospective study of LM/LMM patients referred for dermatologic surgery. RCM was performed at the clinically defined initial surgical margin followed by margin-controlled staged excision with paraffin-embedded tissue, and histopathology was correlated with RCM results. RESULTS: Seventy-two patients were included. Mean age was 66.8 years (standard deviation, 11.1; range, 38-89); 69.4% were men. Seventy of 72 lesions (97.2%) were located on the head and neck with mean largest clinical diameter of 1.3 cm (range, 0.3-5). Diagnostic accuracy for detection of residual melanoma in the tumor debulk (after biopsy) had a sensitivity of 96.7% and a specificity of 66.7% when compared with histopathology. RCM margin assessment revealed an overall agreement with final histopathology of 85.9% (κ = 0.71; P < .001). LIMITATIONS: No RCM imaging beyond initial planned margins was performed. CONCLUSION: RCM showed moderate to excellent overall agreement between RCM imaging of LM/LMM and histopathology of staged excision margins.

Dermoscopy of Linear Basal Cell Carcinomas, a Potential Mimicker of Linear Lesions: a Descriptive Case-series.

Introduction: Among the various widely recognized basal cell carcinoma (BCC) clinical patterns, linear basal cell carcinoma (LBCC) is an uncommon morphologic variant of BCC. Objectives: Describe the clinical and dermoscopic characteristics of LBCC. Methods: Retrospective study including LBCC cases from 5 dermatology centers in North and South America. Biopsy-proven primary BCCs, that presented with at least 3:1 length:width ratio on physical examination, irrespective of tumor subtype or location, were included. Clinical and dermoscopic analysis were performed by 2 experts in dermoscopy. Results: Eighteen cases of LBCC met our inclusion criteria and were included in the study. Median age at diagnosis was 86.0 years, 10 patients (58.8%) were males. Regarding anatomic location, 11/18 (61.1%) were located on the head and neck, 5/18 (27.7%) cases were found on the trunk, and 2 on lower extremities (11.1%). Under dermoscopy, 15/18 (83.3%) of LBCC were pigmented. All tumors displayed at least one of the BCC-specific dermoscopic criteria the most common being blue-grey globules (72.2%). Conclusions: Dermoscopy might be useful in the differentiation of LBCC from other diagnoses presenting as linear lesions such as scars, scratches/erosions, and tattoos, among others. Some of these lesions might be confused by naked eye examination alone.

Artificial Intelligence in Dermatology: Challenges and Perspectives.

Artificial intelligence (AI) based on machine learning and convolutional neuron networks (CNN) is rapidly becoming a realistic prospect in dermatology. Non-melanoma skin cancer is the most common cancer worldwide and melanoma is one of the deadliest forms of cancer. Dermoscopy has improved physicians' diagnostic accuracy for skin cancer recognition but unfortunately it remains comparatively low. AI could provide invaluable aid in the early evaluation and diagnosis of skin cancer. In the last decade, there has been a breakthrough in new research and publications in the field of AI. Studies have shown that CNN algorithms can classify skin lesions from dermoscopic images with superior or at least equivalent performance compared to clinicians. Even though AI algorithms have shown very promising results for the diagnosis of skin cancer in reader studies, their generalizability and applicability in everyday clinical practice remain elusive. Herein we attempted to summarize the potential pitfalls and challenges of AI that were underlined in reader studies and pinpoint strategies to overcome limitations in future studies. Finally, we tried to analyze the advantages and opportunities that lay ahead for a better future for dermatology and patients, with the potential use of AI in our practices.

Artificial intelligence (AI) is the development of computer systems able to perform tasks that normally require human intelligence, such as visual perception, speech recognition, and translation between languages. The research on the use of AI in dermatology includes the ability of a machine to correctly classify a skin lesion. Skin cancer is the most common cancer worldwide and melanoma is the deadliest form of skin cancer. All skin cancers have a better prognosis when detected early in their development, hence their early detection is of paramount importance. Dermatologists use a dermatoscope­a specialized magnifying lens to improve their diagnostic capacity. However, even with the use of the dermatoscope, their ability to recognize skin cancer is far from perfect. AI has the ability to learn from dermoscopic images and subsequently provide an image-based diagnosis. Several studies compared the performance of machines and humans in classifying skin lesions from these images and showed that machines can classify skin lesions as good (and sometimes better) than humans. However, the application of AI in everyday clinical practice remains a challenge. In this article, we attempt to summarize the limitations and challenges that researchers found in their studies, and we provide strategies to improve the design of future studies. Finally, we describe the advantages and opportunities that could lay ahead for a better future for dermatology and patients.

The degradation of performance of a state-of-the-art skin image classifier when applied to patient-driven internet search.

Model Dermatology ( https://modelderm.com ; Build2021) is a publicly testable neural network that can classify 184 skin disorders. We aimed to investigate whether our algorithm can classify clinical images of an Internet community along with tertiary care center datasets. Consecutive images from an Internet skin cancer community ('RD' dataset, 1,282 images posted between 25 January 2020 to 30 July 2021; https://reddit.com/r/melanoma ) were analyzed retrospectively, along with hospital datasets (Edinburgh dataset, 1,300 images; SNU dataset, 2,101 images; TeleDerm dataset, 340 consecutive images). The algorithm's performance was equivalent to that of dermatologists in the curated clinical datasets (Edinburgh and SNU datasets). However, its performance deteriorated in the RD and TeleDerm datasets because of insufficient image quality and the presence of out-of-distribution disorders, respectively. For the RD dataset, the algorithm's Top-1/3 accuracy (39.2%/67.2%) and AUC (0.800) were equivalent to that of general physicians (36.8%/52.9%). It was more accurate than that of the laypersons using random Internet searches (19.2%/24.4%). The Top-1/3 accuracy was affected by inadequate image quality (adequate = 43.2%/71.3% versus inadequate = 32.9%/60.8%), whereas participant performance did not deteriorate (adequate = 35.8%/52.7% vs. inadequate = 38.4%/53.3%). In this report, the algorithm performance was significantly affected by the change of the intended settings, which implies that AI algorithms at dermatologist-level, in-distribution setting, may not be able to show the same level of performance in with out-of-distribution settings.

Validation of artificial intelligence prediction models for skin cancer diagnosis using dermoscopy images: the 2019 International Skin Imaging Collaboration Grand Challenge.

BACKGROUND: Previous studies of artificial intelligence (AI) applied to dermatology have shown AI to have higher diagnostic classification accuracy than expert dermatologists; however, these studies did not adequately assess clinically realistic scenarios, such as how AI systems behave when presented with images of disease categories that are not included in the training dataset or images drawn from statistical distributions with significant shifts from training distributions. We aimed to simulate these real-world scenarios and evaluate the effects of image source institution, diagnoses outside of the training set, and other image artifacts on classification accuracy, with the goal of informing clinicians and regulatory agencies about safety and real-world accuracy. METHODS: We designed a large dermoscopic image classification challenge to quantify the performance of machine learning algorithms for the task of skin cancer classification from dermoscopic images, and how this performance is affected by shifts in statistical distributions of data, disease categories not represented in training datasets, and imaging or lesion artifacts. Factors that might be beneficial to performance, such as clinical metadata and external training data collected by challenge participants, were also evaluated. 25 331 training images collected from two datasets (in Vienna [HAM10000] and Barcelona [BCN20000]) between Jan 1, 2000, and Dec 31, 2018, across eight skin diseases, were provided to challenge participants to design appropriate algorithms. The trained algorithms were then tested for balanced accuracy against the HAM10000 and BCN20000 test datasets and data from countries not included in the training dataset (Turkey, New Zealand, Sweden, and Argentina). Test datasets contained images of all diagnostic categories available in training plus other diagnoses not included in training data (not trained category). We compared the performance of the algorithms against that of 18 dermatologists in a simulated setting that reflected intended clinical use. FINDINGS: 64 teams submitted 129 state-of-the-art algorithm predictions on a test set of 8238 images. The best performing algorithm achieved 58·8% balanced accuracy on the BCN20000 data, which was designed to better reflect realistic clinical scenarios, compared with 82·0% balanced accuracy on HAM10000, which was used in a previously published benchmark. Shifted statistical distributions and disease categories not included in training data contributed to decreases in accuracy. Image artifacts, including hair, pen markings, ulceration, and imaging source institution, decreased accuracy in a complex manner that varied based on the underlying diagnosis. When comparing algorithms to expert dermatologists (2460 ratings on 1269 images), algorithms performed better than experts in most categories, except for actinic keratoses (similar accuracy on average) and images from categories not included in training data (26% correct for experts vs 6% correct for algorithms, p<0·0001). For the top 25 submitted algorithms, 47·1% of the images from categories not included in training data were misclassified as malignant diagnoses, which would lead to a substantial number of unnecessary biopsies if current state-of-the-art AI technologies were clinically deployed. INTERPRETATION: We have identified specific deficiencies and safety issues in AI diagnostic systems for skin cancer that should be addressed in future diagnostic evaluation protocols to improve safety and reliability in clinical practice. FUNDING: Melanoma Research Alliance and La Marató de TV3.

Deep Learning for Basal Cell Carcinoma Detection for Reflectance Confocal Microscopy.

Basal cell carcinoma (BCC) is the most common skin cancer, with over 2 million cases diagnosed annually in the United States. Conventionally, BCC is diagnosed by naked eye examination and dermoscopy. Suspicious lesions are either removed or biopsied for histopathological confirmation, thus lowering the specificity of noninvasive BCC diagnosis. Recently, reflectance confocal microscopy, a noninvasive diagnostic technique that can image skin lesions at cellular level resolution, has shown to improve specificity in BCC diagnosis and reduced the number needed to biopsy by 2-3 times. In this study, we developed and evaluated a deep learning-based artificial intelligence model to automatically detect BCC in reflectance confocal microscopy images. The proposed model achieved an area under the curve for the receiver operator characteristic curve of 89.7% (stack level) and 88.3% (lesion level), a performance on par with that of reflectance confocal microscopy experts. Furthermore, the model achieved an area under the curve of 86.1% on a held-out test set from international collaborators, demonstrating the reproducibility and generalizability of the proposed automated diagnostic approach. These results provide a clear indication that the clinical deployment of decision support systems for the detection of BCC in reflectance confocal microscopy images has the potential for optimizing the evaluation and diagnosis of patients with skin cancer.