GI.1 Norovirus Neutralizing Antibody Levels Are Correlated with GI.1 Histo-blood Group Antigen-Blocking Antibody Levels.

BACKGROUND: The in vitro cultivation of human noroviruses allows a comparison of antibody levels measured in neutralization and histoblood group antigen (HBGA)-blocking assays. METHODS: Serum samples collected during the evaluation of an investigational norovirus vaccine (HIL-214 [formerly TAK-214]) were assayed for neutralizing antibody levels against the vaccine's prototype Norwalk virus/GI.1 (P1) virus strain. Results were compared to those previously determined using HBGA-blocking assays. RESULTS: Neutralizing antibody seroresponses were observed in 83% of 24 vaccinated adults, and antibody levels were highly correlated (r=0.81, P<0.001) with those measured by HBGA-blocking. CONCLUSIONS: GI.1-specific HBGA-blocking antibodies are a surrogate for neutralization of GI.1 norovirus.

Position statement of the EADV Artificial Intelligence (AI) Task Force on AI-assisted smartphone apps and web-based services for skin disease.

BACKGROUND: As the use of smartphones continues to surge globally, mobile applications (apps) have become a powerful tool for healthcare engagement. Prominent among these are dermatology apps powered by Artificial Intelligence (AI), which provide immediate diagnostic guidance and educational resources for skin diseases, including skin cancer. OBJECTIVE: This article, authored by the EADV AI Task Force, seeks to offer insights and recommendations for the present and future deployment of AI-assisted smartphone applications (apps) and web-based services for skin diseases with emphasis on skin cancer detection. METHODS: An initial position statement was drafted on a comprehensive literature review, which was subsequently refined through two rounds of digital discussions and meticulous feedback by the EADV AI Task Force, ensuring its accuracy, clarity and relevance. RESULTS: Eight key considerations were identified, including risks associated with inaccuracy and improper user education, a decline in professional skills, the influence of non-medical commercial interests, data security, direct and indirect costs, regulatory approval and the necessity of multidisciplinary implementation. Following these considerations, three main recommendations were formulated: (1) to ensure user trust, app developers should prioritize transparency in data quality, accuracy, intended use, privacy and costs; (2) Apps and web-based services should ensure a uniform user experience for diverse groups of patients; (3) European authorities should adopt a rigorous and consistent regulatory framework for dermatology apps to ensure their safety and accuracy for users. CONCLUSIONS: The utilisation of AI-assisted smartphone apps and web-based services in diagnosing and treating skin diseases has the potential to greatly benefit patients in their dermatology journeys. By prioritising innovation, fostering collaboration and implementing effective regulations, we can ensure the successful integration of these apps into clinical practice.

International Skin Imaging Collaboration-Designated Diagnoses (ISIC-DX): Consensus terminology for lesion diagnostic labeling.

BACKGROUND: A common terminology for diagnosis is critically important for clinical communication, education, research and artificial intelligence. Prevailing lexicons are limited in fully representing skin neoplasms. OBJECTIVES: To achieve expert consensus on diagnostic terms for skin neoplasms and their hierarchical mapping. METHODS: Diagnostic terms were extracted from textbooks, publications and extant diagnostic codes. Terms were hierarchically mapped to super-categories (e.g. 'benign') and cellular/tissue-differentiation categories (e.g. 'melanocytic'), and appended with pertinent-modifiers and synonyms. These terms were evaluated using a modified-Delphi consensus approach. Experts from the International-Skin-Imaging-Collaboration (ISIC) were surveyed on agreement with terms and their hierarchical mapping; they could suggest modifying, deleting or adding terms. Consensus threshold was >75% for the initial rounds and >50% for the final round. RESULTS: Eighteen experts completed all Delphi rounds. Of 379 terms, 356 (94%) reached consensus in round one. Eleven of 226 (5%) benign-category terms, 6/140 (4%) malignant-category terms and 6/13 (46%) indeterminate-category terms did not reach initial agreement. Following three rounds, final consensus consisted of 362 terms mapped to 3 super-categories and 41 cellular/tissue-differentiation categories. CONCLUSIONS: We have created, agreed upon, and made public a taxonomy for skin neoplasms and their hierarchical mapping. Further study will be needed to evaluate the utility and completeness of the lexicon.

Comparison of Microneutralization and Histo-Blood Group Antigen-Blocking Assays for Functional Norovirus Antibody Detection.

BACKGROUND: The development of an in vitro cultivation system for human noroviruses allows the measurement of neutralizing antibody levels. METHODS: Serum neutralizing antibody levels were determined using a GII.4/Sydney/2012-like virus in human intestinal enteroids in samples collected before and 4 weeks after administration of an investigational norovirus vaccine and were compared with those measured in histo-blood group antigen (HBGA)-blocking assays. RESULTS: Neutralizing antibody seroresponses were observed in 71% of 24 vaccinated adults, and antibody levels were highly correlated (r = 0.82, P < .001) with those measured by HBGA blocking. CONCLUSIONS: HBGA-blocking antibodies are a surrogate for neutralization in human noroviruses. CLINICAL TRIALS REGISTRATION: NCT02475278.

Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic study.

BACKGROUND: Whether machine-learning algorithms can diagnose all pigmented skin lesions as accurately as human experts is unclear. The aim of this study was to compare the diagnostic accuracy of state-of-the-art machine-learning algorithms with human readers for all clinically relevant types of benign and malignant pigmented skin lesions. METHODS: For this open, web-based, international, diagnostic study, human readers were asked to diagnose dermatoscopic images selected randomly in 30-image batches from a test set of 1511 images. The diagnoses from human readers were compared with those of 139 algorithms created by 77 machine-learning labs, who participated in the International Skin Imaging Collaboration 2018 challenge and received a training set of 10 015 images in advance. The ground truth of each lesion fell into one of seven predefined disease categories: intraepithelial carcinoma including actinic keratoses and Bowen's disease; basal cell carcinoma; benign keratinocytic lesions including solar lentigo, seborrheic keratosis and lichen planus-like keratosis; dermatofibroma; melanoma; melanocytic nevus; and vascular lesions. The two main outcomes were the differences in the number of correct specific diagnoses per batch between all human readers and the top three algorithms, and between human experts and the top three algorithms. FINDINGS: Between Aug 4, 2018, and Sept 30, 2018, 511 human readers from 63 countries had at least one attempt in the reader study. 283 (55·4%) of 511 human readers were board-certified dermatologists, 118 (23·1%) were dermatology residents, and 83 (16·2%) were general practitioners. When comparing all human readers with all machine-learning algorithms, the algorithms achieved a mean of 2·01 (95% CI 1·97 to 2·04; p<0·0001) more correct diagnoses (17·91 [SD 3·42] vs 19·92 [4·27]). 27 human experts with more than 10 years of experience achieved a mean of 18·78 (SD 3·15) correct answers, compared with 25·43 (1·95) correct answers for the top three machine algorithms (mean difference 6·65, 95% CI 6·06-7·25; p<0·0001). The difference between human experts and the top three algorithms was significantly lower for images in the test set that were collected from sources not included in the training set (human underperformance of 11·4%, 95% CI 9·9-12·9 vs 3·6%, 0·8-6·3; p<0·0001). INTERPRETATION: State-of-the-art machine-learning classifiers outperformed human experts in the diagnosis of pigmented skin lesions and should have a more important role in clinical practice. However, a possible limitation of these algorithms is their decreased performance for out-of-distribution images, which should be addressed in future research. FUNDING: None.

Dermoscopy and dermatopathology correlates of cutaneous neoplasms.

Dermoscopy is increasingly used by clinicians (dermatologists, family physicians, podiatrists, doctors of osteopathic medicine, etc) to inform clinical management decisions. Dermoscopic findings or images provided to pathologists offer important insight into the clinician's diagnostic and management thought process. However, with limited dermoscopic training in dermatopathology, dermoscopic descriptions and images provided in the requisition form provide little value to pathologists. Most dermoscopic structures have direct histopathologic correlates, and therefore dermoscopy can act as an excellent communication bridge between the clinician and the pathologist. In the first article in this continuing medical education series, we review dermoscopic features and their histopathologic correlates.

Usefulness of dermoscopy to improve the clinical and histopathologic diagnosis of skin cancers.

Multiple studies have shown that dermoscopy increases the sensitivity and specificity for the detection of skin cancers compared with examination by the naked eye. Dermoscopy can also lead to the detection of thinner and smaller cancers. In addition, dermoscopy leads to the more precise selection of lesions requiring excision. In essence, dermoscopy helps clinicians differentiate benign from malignant lesions through the presence or absence of specific dermoscopic structures. Therefore, because most dermoscopic structures have direct histopathologic correlates, dermoscopy can allow the prediction of certain histologic findings present in skin cancers, thus helping select management and treatment options for select types of skin cancers. Visualizing dermoscopic structures in the ex vivo specimens can also be beneficial. It can improve the histologic diagnostic accuracy by targeted step-sectioning in areas of concern, which can be marked by the clinician before sending the specimen to the pathologist, or by the pathologist on the excised specimen in the laboratory. In addition, ex vivo dermoscopy can also be used to select tumor areas with genetic importance because some dermoscopic structures have been related to mutations with theragnostic relevance. In the second article in this continuing medical education series, we review the impact of dermoscopy on the diagnostic accuracy of skin cancer, how dermoscopy can affect the histopathologic examination, and which dermoscopic features may be more relevant in terms of histologic and genetic prediction.

Critical aspects to achieve a high-quality melanoma clinic.

PURPOSE OF REVIEW: With incidence of melanoma growing worldwide and new therapies prolonging the survival of patients with advanced disease, complex medical care is needed. RECENT FINDINGS: Best care of complicated melanoma cases is achieved in specialized referral centers. Aims to provide optimized melanoma therapy, best patient-reported treatment outcome, and successful clinical and translational research, necessitate a dedicated interdisciplinary team. SUMMARY: We report on critical aspects of the interaction between patients, medical care givers, clinical trial and biobanking teams, and emphasize the importance of interdisciplinary tumor boards. Specialized skin cancer nurses and local patient advocacy groups should be involved in patient care and could be the binding link between the patients and the treatment team.

Accuracy of dermatoscopy for the diagnosis of nonpigmented cancers of the skin.

BACKGROUND: Nonpigmented skin cancer is common, and diagnosis with the unaided eye is error prone. OBJECTIVE: To investigate whether dermatoscopy improves the diagnostic accuracy for nonpigmented (amelanotic) cutaneous neoplasms. METHODS: We collected a sample of 2072 benign and malignant neoplastic lesions and inflammatory conditions and presented close-up images taken with and without dermatoscopy to 95 examiners with different levels of experience. RESULTS: The area under the curve was significantly higher with than without dermatoscopy (0.68 vs 0.64, P < .001). Among 51 possible diagnoses, the correct diagnosis was selected in 33.1% of cases with and 26.4% of cases without dermatoscopy (P < .001). For experts, the frequencies of correct specific diagnoses of a malignant lesion improved from 40.2% without to 51.3% with dermatoscopy. For all malignant neoplasms combined, the frequencies of appropriate management strategies increased from 78.1% without to 82.5% with dermatoscopy. LIMITATIONS: The study deviated from a real-life clinical setting and was potentially affected by verification and selection bias. CONCLUSIONS: Dermatoscopy improves the diagnosis and management of nonpigmented skin cancer and should be used as an adjunct to examination with the unaided eye.

Differential Diagnosis of Seborrheic Keratosis: Clinical and Dermoscopic Features.

Seborrheic keratosis (SK) is a benign epidermal keratinocytic tumor that is extremely common, particularly in individuals over the age of 50. Most individuals with SK will have more than one lesion and the presence of over 10 lesions in the same person is not uncommon. Although the clinical morphology of most SK with their stuck-on, symmetric, keratotic, and waxy appearance makes them easy to identify, many manifest a morphology resembling melanoma or squamous cell carcinoma. One can argue that such cases will ultimately not prove to be problematic since a simple biopsy will easily reveal their benign nature and eliminate any concerns. However, the cost and morbidity associated with the biopsy of benign lesions should not be underestimated. Methods to improve our in vivo ability to correctly identify SK will prove beneficial not only to the health care system in general but to the individual patient specifically. The issue of greater concern resides with skin cancers that mimic SK or when skin cancers arise in association with SK. Needless to say, in vivo methods to help identify malignancy and differentiate them from benign lesions would be welcomed by all. Fortunately, we do now have in vivo imaging methods such as dermoscopy that can improve the clinician's diagnostic accuracy. In this article, we summarize the current knowledge regarding the clinical and dermoscopic features of SK, and provide clues to aid in their diagnosis.

J Drugs Dermatol. 2017;16(9):835-842.

Clinical and dermoscopic characteristics of melanomas on nonfacial chronically sun-damaged skin.

BACKGROUND: Melanomas on chronically sun-damaged skin (CSDS) can be difficult to identify and often manifest morphologic features that overlap with benign lesions. OBJECTIVE: We describe and analyze the clinical and dermoscopic characteristics of melanomas on nonfacial CSDS. METHODS: Melanoma cases on nonfacial CSDS were retrospectively identified from the biopsy specimen logs of 6 melanoma clinics. Clinical and dermoscopic images were combined into 1 database. Demographics, clinical, dermoscopic, and histopathologic information were analyzed. Descriptive frequencies were calculated. RESULTS: One hundred eighty-six cases met the inclusion criteria: 142 melanomas in situ (76%) and 39 invasive (21%; mean thickness, 0.49 mm). Lentigo maligna was the most common histopathologic subtype (n = 76; 40.9%). The most frequent dermoscopic structures were granularity (n = 126; 67.7%) and angulated lines (n = 82; 44%). Vascular structures were more frequent in invasive melanomas (56% vs 12% of in situ melanomas). Most manifested 1 of 3 dermoscopic patterns: patchy peripheral pigmented islands, angulated lines, and tan structureless with granularity pattern. LIMITATIONS: This was a retrospective study, and evaluators were not blinded to the diagnosis. In addition, interobserver concordance and sensitivity and specificity for dermoscopic structures were not evaluated. CONCLUSION: Outlier lesions manifesting dermoscopic structures, such as granularity, angulated lines, or vessels and any of the 3 described dermoscopic patterns should raise suspicion for melanoma.

Automatized self-supervised learning for skin lesion screening.

Melanoma, the deadliest form of skin cancer, has seen a steady increase in incidence rates worldwide, posing a significant challenge to dermatologists. Early detection is crucial for improving patient survival rates. However, performing total body screening (TBS), i.e., identifying suspicious lesions or ugly ducklings (UDs) by visual inspection, can be challenging and often requires sound expertise in pigmented lesions. To assist users of varying expertise levels, an artificial intelligence (AI) decision support tool was developed. Our solution identifies and characterizes UDs from real-world wide-field patient images. It employs a state-of-the-art object detection algorithm to locate and isolate all skin lesions present in a patient's total body images. These lesions are then sorted based on their level of suspiciousness using a self-supervised AI approach, tailored to the specific context of the patient under examination. A clinical validation study was conducted to evaluate the tool's performance. The results demonstrated an average sensitivity of 95% for the top-10 AI-identified UDs on skin lesions selected by the majority of experts in pigmented skin lesions. The study also found that the tool increased dermatologists' confidence when formulating a diagnosis, and the average majority agreement with the top-10 AI-identified UDs reached 100% when assisted by our tool. With the development of this AI-based decision support tool, we aim to address the shortage of specialists, enable faster consultation times for patients, and demonstrate the impact and usability of AI-assisted screening. Future developments will include expanding the dataset to include histologically confirmed melanoma and validating the tool for additional body regions.

Expert Agreement on the Presence and Spatial Localization of Melanocytic Features in Dermoscopy.

Dermoscopy aids in melanoma detection; however, agreement on dermoscopic features, including those of high clinical relevance, remains poor. In this study, we attempted to evaluate agreement among experts on exemplar images not only for the presence of melanocytic-specific features but also for spatial localization. This was a cross-sectional, multicenter, observational study. Dermoscopy images exhibiting at least 1 of 31 melanocytic-specific features were submitted by 25 world experts as exemplars. Using a web-based platform that allows for image markup of specific contrast-defined regions (superpixels), 20 expert readers annotated 248 dermoscopic images in collections of 62 images. Each collection was reviewed by five independent readers. A total of 4,507 feature observations were performed. Good-to-excellent agreement was found for 14 of 31 features (45.2%), with eight achieving excellent agreement (Gwet's AC >0.75) and seven of them being melanoma-specific features. These features were peppering/granularity (0.91), shiny white streaks (0.89), typical pigment network (0.83), blotch irregular (0.82), negative network (0.81), irregular globules (0.78), dotted vessels (0.77), and blue-whitish veil (0.76). By utilizing an exemplar dataset, a good-to-excellent agreement was found for 14 features that have previously been shown useful in discriminating nevi from melanoma. All images are public (www.isic-archive.com) and can be used for education, scientific communication, and machine learning experiments.

Dermoscopy of acral melanoma: a multicenter study on behalf of the international dermoscopy society.

BACKGROUND: Most studies on dermoscopy of acral lesions were conducted in Asian populations. In this study, we analyzed these features in a predominantly Caucasian population. OBJECTIVE: Estimate the prevalence of dermoscopic features in acral lesions, and assess their level of agreement between observers. METHODS: In this retrospective multicenter study, 167 acral lesions (66 melanomas) were evaluated for 13 dermoscopic patterns by 26 physicians, via a secured Internet platform. RESULTS: Parallel furrow pattern, bizarre pattern, and diffuse pigmentation with variable shades of brown had the highest prevalence. The agreement for lesion patterns between physicians was variable. Agreement was dependent on the level of diagnostic difficulty. CONCLUSION: Lesions with a diameter >1 cm were more likely to be melanoma. We found as well that a benign pattern can be seen in parts of melanomas. For this reason one should evaluate an acral lesion for the presence of malignant patterns first.

Dermoscopy for the pediatric dermatologist part III: dermoscopy of melanocytic lesions.

Melanocytic nevi encompass a variety of lesions, including blue, Spitz, congenital, and acquired nevi. These nevi can occasionally manifest clinical morphologies resembling melanoma, and the presence of such nevi in children can elicit anxiety in patients, parents, and clinicians. Dermoscopy has been shown to increase the diagnostic accuracy for melanoma and to help differentiate melanoma from nevi, ultimately aiding in the decision-making process as to whether to perform a biopsy. Dermoscopy is the perfect instrument to use during the evaluation of pigmented skin lesions in children because it is painless and provides important information for the clinician that can assist in formulating appropriate management decisions. This review highlights the most common benign dermoscopic patterns encountered in nevi and discuss the 10 most common dermoscopic structures seen in melanomas. Lesions manifesting a benign dermoscopic pattern and lacking any melanoma-specific structures do not need to be excised and can safely be monitored. In contrast, melanomas will invariably deviate from the benign nevus patterns and will usually manifest at least 1 of the 10 melanoma-specific structures: atypical network, negative network, streaks, crystalline structures, atypical dots and globules, irregular blotch, blue-white veil, regression structures, peripheral brown structureless areas, and atypical vessels. It is important to be cognizant of the fact that melanomas in childhood usually do not manifest the clinical ABCD features. Instead, they are often symmetric, amelanotic, nodular lesions. Although the clinical appearance may not be alarming, with dermoscopy they will invariably manifest at least one melanoma-specific structure, the most common being atypical vascular structures and crystalline structures.

Dermoscopy for the pediatric dermatologist part I: dermoscopy of pediatric infectious and inflammatory skin lesions and hair disorders.

The dermoscope allows physicians to examine the macroscopic and microscopic primary morphology of skin lesions, identify subtle clinical clues, confirm naked-eye clinical diagnoses, and monitor treatment progress while posing little threat to the young patient. This review summarizes important dermoscopic structures seen in infectious and inflammatory skin conditions and hair disorders in children. Scabies, pediculosis, phthiriasis, molluscum contagiosum, tinea nigra, and verrucae are well characterized dermoscopically by delta-shaped structures, ovoid-shaped nits, the crab louse, red corona, brown strands or spicules, and multiple densely packed papilla with a central black dot surrounded by a whitish halo, respectively. These dermoscopic structures will be discussed, focusing on the dermoscopic morphologies and dermoscopic sensitivity for diagnosis and its utility in monitoring treatment progress. Dermoscopy has also been shown to significantly improve the clinician's diagnostic and monitoring accuracy of inflammatory skin lesions such as psoriasis, which is characterized dermoscopically by uniformly distributed dotted blood vessels, and lichen planus, which is characterized by whitish lines on a purple to reddish background. Dermoscopy of the hair and scalp (trichoscopy) facilitates the differential diagnosis of hair diseases in children, including alopecia areata, trichotillomania, and tinea capitis. It can also assist in the diagnosis of multiple genetic hair shaft disorders, such as monilethrix, trichorrhexis invaginata, trichorrhexis nodosa, pili torti, and pili annulati.

Dermoscopy for the pediatric dermatologist, part ii: dermoscopy of genetic syndromes with cutaneous manifestations and pediatric vascular lesions.

Genetic syndromes including basal cell nevus syndrome (BSNS), xeroderma pigmentosum (XP), and epidermodysplasia verruciformis (EV) predispose the individual to skin cancer. Basal cell carcinomas (BCCs) often develop in patients with BCNS and XP. One of the aims of surveillance examination in these patients is to detect BCC while the tumors are still small and easy to manage. Dermoscopy, by allowing the visualization of arborizing vessels, ovoid nests, nonaggregated blue-gray globules, and spoke-wheel and leaf-like structures, can facilitate in the early detection of BCC. Patients with XP are also at risk for developing squamous cell carcinoma (SCC). Dermoscopy can assist in the early detection of these cancers by allowing the observer to visualize focal glomerular vessels, which is a common feature seen in SCC. This feature can also assist in detecting SCC developing in other syndromes such as EV and epidermolysis bullosa (EB). In addition to helping in the detection of BCC and SCC, dermoscopy can also help detect melanoma in individuals with XP and evaluate nevi developing in those with EB. This review will discuss how dermoscopy can be used in the management of patients with BSNS, XP, EV, and EB and will discuss the dermoscopic findings of vascular lesions, including pyogenic granuloma, hemangioma, port-wine stain, and lymphangioma circumscriptum.

Influence of time on dermoscopic diagnosis and management.

BACKGROUND/OBJECTIVES: Dermoscopy aids in clinical decision-making. However, time pressure is a common reason precluding its use. We evaluated the effect of time on lesion recognition and management decisions utilising clinical and dermoscopic images. METHOD: In all, 100 dermoscopic images were presented to 15 dermatologists with experience in dermoscopy and seven non-experts (dermatology residents). Each lesion was displayed thrice in succession. The dermoscopic image was initially presented for 1 s (t1). The same dermoscopic image was shown again without time constraints (t2) and then a final time with additional images of the clinical context (t3). Participants provided a diagnosis, their level of confidence and biopsy predilection after evaluating each image. RESULTS: For benign lesions, both groups rarely changed their diagnosis. However, an improvement in the number of correct benign diagnoses was observed when the lesion was shown in a clinical context. For malignant lesions, both groups improved when more time and clinical context was given; nevertheless, non-experts were more likely to change the diagnosis towards the correct one as more time was given and tended to perform more biopsies, in particular of benign lesions. Limitations were a small number of participants and an artificial study setting. CONCLUSION: Dermoscopy uses analytical and non-analytical reasoning approaches. We suggest that non-analytical reasoning is employed when rapid clinical decisions need to be made, especially during the evaluation of benign lesions. We conclude that dermoscopy is relatively rapid and non-time-consuming technique that adds relevant information and guides clinicians towards appropriate management decisions.

Agreement Between Experts and an Untrained Crowd for Identifying Dermoscopic Features Using a Gamified App: Reader Feasibility Study.

BACKGROUND: Dermoscopy is commonly used for the evaluation of pigmented lesions, but agreement between experts for identification of dermoscopic structures is known to be relatively poor. Expert labeling of medical data is a bottleneck in the development of machine learning (ML) tools, and crowdsourcing has been demonstrated as a cost- and time-efficient method for the annotation of medical images. OBJECTIVE: The aim of this study is to demonstrate that crowdsourcing can be used to label basic dermoscopic structures from images of pigmented lesions with similar reliability to a group of experts. METHODS: First, we obtained labels of 248 images of melanocytic lesions with 31 dermoscopic "subfeatures" labeled by 20 dermoscopy experts. These were then collapsed into 6 dermoscopic "superfeatures" based on structural similarity, due to low interrater reliability (IRR): dots, globules, lines, network structures, regression structures, and vessels. These images were then used as the gold standard for the crowd study. The commercial platform DiagnosUs was used to obtain annotations from a nonexpert crowd for the presence or absence of the 6 superfeatures in each of the 248 images. We replicated this methodology with a group of 7 dermatologists to allow direct comparison with the nonexpert crowd. The Cohen κ value was used to measure agreement across raters. RESULTS: In total, we obtained 139,731 ratings of the 6 dermoscopic superfeatures from the crowd. There was relatively lower agreement for the identification of dots and globules (the median κ values were 0.526 and 0.395, respectively), whereas network structures and vessels showed the highest agreement (the median κ values were 0.581 and 0.798, respectively). This pattern was also seen among the expert raters, who had median κ values of 0.483 and 0.517 for dots and globules, respectively, and 0.758 and 0.790 for network structures and vessels. The median κ values between nonexperts and thresholded average-expert readers were 0.709 for dots, 0.719 for globules, 0.714 for lines, 0.838 for network structures, 0.818 for regression structures, and 0.728 for vessels. CONCLUSIONS: This study confirmed that IRR for different dermoscopic features varied among a group of experts; a similar pattern was observed in a nonexpert crowd. There was good or excellent agreement for each of the 6 superfeatures between the crowd and the experts, highlighting the similar reliability of the crowd for labeling dermoscopic images. This confirms the feasibility and dependability of using crowdsourcing as a scalable solution to annotate large sets of dermoscopic images, with several potential clinical and educational applications, including the development of novel, explainable ML tools.