International Skin Imaging Collaboration-Designated Diagnoses (ISIC-DX): Consensus terminology for lesion diagnostic labeling.

BACKGROUND: A common terminology for diagnosis is critically important for clinical communication, education, research and artificial intelligence. Prevailing lexicons are limited in fully representing skin neoplasms. OBJECTIVES: To achieve expert consensus on diagnostic terms for skin neoplasms and their hierarchical mapping. METHODS: Diagnostic terms were extracted from textbooks, publications and extant diagnostic codes. Terms were hierarchically mapped to super-categories (e.g. 'benign') and cellular/tissue-differentiation categories (e.g. 'melanocytic'), and appended with pertinent-modifiers and synonyms. These terms were evaluated using a modified-Delphi consensus approach. Experts from the International-Skin-Imaging-Collaboration (ISIC) were surveyed on agreement with terms and their hierarchical mapping; they could suggest modifying, deleting or adding terms. Consensus threshold was >75% for the initial rounds and >50% for the final round. RESULTS: Eighteen experts completed all Delphi rounds. Of 379 terms, 356 (94%) reached consensus in round one. Eleven of 226 (5%) benign-category terms, 6/140 (4%) malignant-category terms and 6/13 (46%) indeterminate-category terms did not reach initial agreement. Following three rounds, final consensus consisted of 362 terms mapped to 3 super-categories and 41 cellular/tissue-differentiation categories. CONCLUSIONS: We have created, agreed upon, and made public a taxonomy for skin neoplasms and their hierarchical mapping. Further study will be needed to evaluate the utility and completeness of the lexicon.

Outcomes of Decompressive Surgery for Patients With Severe Cerebral Venous Thrombosis: DECOMPRESS2 Observational Study.

BACKGROUND: Decompressive neurosurgery is recommended for patients with cerebral venous thrombosis (CVT) who have large parenchymal lesions and impending brain herniation. This recommendation is based on limited evidence. We report long-term outcomes of patients with CVT treated by decompressive neurosurgery in an international cohort. METHODS: DECOMPRESS2 (Decompressive Surgery for Patients With Cerebral Venous Thrombosis, Part 2) was a prospective, international cohort study. Consecutive patients with CVT treated by decompressive neurosurgery were evaluated at admission, discharge, 6 months, and 12 months. The primary outcome was death or severe disability (modified Rankin Scale scores, 5-6) at 12 months. The secondary outcomes included patient and caregiver opinions on the benefits of surgery. The association between baseline variables before surgery and the primary outcome was assessed by multivariable logistic regression. RESULTS: A total of 118 patients (80 women; median age, 38 years) were included from 15 centers in 10 countries from December 2011 to December 2019. Surgery (115 craniectomies and 37 hematoma evacuations) was performed within a median of 1 day after diagnosis. At last assessment before surgery, 68 (57.6%) patients were comatose, fixed dilated pupils were found unilaterally in 27 (22.9%) and bilaterally in 9 (7.6%). Twelve-month follow-up data were available for 113 (95.8%) patients. Forty-six (39%) patients were dead or severely disabled (modified Rankin Scale scores, 5-6), of whom 40 (33.9%) patients had died. Forty-two (35.6%) patients were independent (modified Rankin Scale scores, 0-2). Coma (odds ratio, 2.39 [95% CI, 1.03-5.56]) and fixed dilated pupil (odds ratio, 2.22 [95% CI, 0.90-4.92]) were predictors of death or severe disability. Of the survivors, 56 (78.9%) patients and 61 (87.1%) caregivers expressed a positive opinion on surgery. CONCLUSIONS: Two-thirds of patients with severe CVT were alive and more than one-third were independent 1 year after decompressive surgery. Among survivors, surgery was judged as worthwhile by 4 out of 5 patients and caregivers. These results support the recommendation to perform decompressive neurosurgery in patients with CVT with impending brain herniation.

Expert Agreement on the Presence and Spatial Localization of Melanocytic Features in Dermoscopy.

Dermoscopy aids in melanoma detection; however, agreement on dermoscopic features, including those of high clinical relevance, remains poor. In this study, we attempted to evaluate agreement among experts on exemplar images not only for the presence of melanocytic-specific features but also for spatial localization. This was a cross-sectional, multicenter, observational study. Dermoscopy images exhibiting at least 1 of 31 melanocytic-specific features were submitted by 25 world experts as exemplars. Using a web-based platform that allows for image markup of specific contrast-defined regions (superpixels), 20 expert readers annotated 248 dermoscopic images in collections of 62 images. Each collection was reviewed by five independent readers. A total of 4,507 feature observations were performed. Good-to-excellent agreement was found for 14 of 31 features (45.2%), with eight achieving excellent agreement (Gwet's AC >0.75) and seven of them being melanoma-specific features. These features were peppering/granularity (0.91), shiny white streaks (0.89), typical pigment network (0.83), blotch irregular (0.82), negative network (0.81), irregular globules (0.78), dotted vessels (0.77), and blue-whitish veil (0.76). By utilizing an exemplar dataset, a good-to-excellent agreement was found for 14 features that have previously been shown useful in discriminating nevi from melanoma. All images are public (www.isic-archive.com) and can be used for education, scientific communication, and machine learning experiments.

Prospective validation of dermoscopy-based open-source artificial intelligence for melanoma diagnosis (PROVE-AI study).

The use of artificial intelligence (AI) has the potential to improve the assessment of lesions suspicious of melanoma, but few clinical studies have been conducted. We validated the accuracy of an open-source, non-commercial AI algorithm for melanoma diagnosis and assessed its potential impact on dermatologist decision-making. We conducted a prospective, observational clinical study to assess the diagnostic accuracy of the AI algorithm (ADAE) in predicting melanoma from dermoscopy skin lesion images. The primary aim was to assess the reliability of ADAE's sensitivity at a predefined threshold of 95%. Patients who had consented for a skin biopsy to exclude melanoma were eligible. Dermatologists also estimated the probability of melanoma and indicated management choices before and after real-time exposure to ADAE scores. All lesions underwent biopsy. Four hundred thirty-five participants were enrolled and contributed 603 lesions (95 melanomas). Participants had a mean age of 59 years, 54% were female, and 96% were White individuals. At the predetermined 95% sensitivity threshold, ADAE had a sensitivity of 96.8% (95% CI: 91.1-98.9%) and specificity of 37.4% (95% CI: 33.3-41.7%). The dermatologists' ability to assess melanoma risk significantly improved after ADAE exposure (AUC 0.7798 vs. 0.8161, p = 0.042). Post-ADAE dermatologist decisions also had equivalent or higher net benefit compared to biopsying all lesions. We validated the accuracy of an open-source melanoma AI algorithm and showed its theoretical potential for improving dermatology experts' ability to evaluate lesions suspicious of melanoma. Larger randomized trials are needed to fully evaluate the potential of adopting this AI algorithm into clinical workflows.

Impact and centrality of attention dysregulation on cognition, anxiety, and low mood in adolescents.

Functional impairments in cognition are frequently thought to be a feature of individuals with depression or anxiety. However, documented impairments are both broad and inconsistent, with little known about when they emerge, whether they are causes or effects of affective symptoms, or whether specific cognitive systems are implicated. Here, we show, in the adolescent ABCD cohort (N = 11,876), that attention dysregulation is a robust factor underlying wide-ranging cognitive task impairments seen in adolescents with moderate to severe anxiety or low mood. We stratified individuals high in DSM-oriented depression or anxiety symptomology, and low in attention deficit hyperactivity disorder (ADHD), as well as vice versa - demonstrating that those high in depression or anxiety dimensions but low in ADHD symptoms not only exhibited normal task performance across several commonly studied cognitive paradigms, but out-performed controls in several domains, as well as in those low in both dimensions. Similarly, we showed that there were no associations between psychopathological dimensions and performance on an extensive cognitive battery after controlling for attention dysregulation. Further, corroborating previous research, the co-occurrence of attention dysregulation was associated with a wide range of other adverse outcomes, psychopathological features, and executive functioning (EF) impairments. To assess how attention dysregulation relates to and generates diverse psychopathology, we performed confirmatory and exploratory network analysis with different analytic approaches using Gaussian Graphical Models and Directed Acyclic Graphs to examine interactions between ADHD, anxiety, low mood, oppositional defiant disorder (ODD), social relationships, and cognition. Confirmatory centrality analysis indicated that features of attention dysregulation were indeed central and robustly connected to a wide range of psychopathological traits across different categories, scales, and time points. Exploratory network analysis indicated potentially important bridging traits and socioenvironmental influences in the relationships between ADHD symptoms and mood/anxiety disorders. Trait perfectionism was uniquely associated with both better cognitive performance and broad psychopathological dimensions. This work suggests that attentional dysregulation may moderate the breadth of EF, fluid, and crystalized cognitive task outcomes seen in adolescents with anxiety and low mood, and may be central to disparate pathological features, and thus a target for attenuating wide-ranging negative developmental outcomes.

Improving Artificial Intelligence-Based Diagnosis on Pediatric Skin Lesions.

Artificial intelligence algorithms to classify melanoma are dependent on their training data, which limits generalizability. The objective of this study was to compare the performance of an artificial intelligence model trained on a standard adult-predominant dermoscopic dataset before and after the addition of additional pediatric training images. The performances were compared using held-out adult and pediatric test sets of images. We trained two models: one (model A) on an adult-predominant dataset (37,662 images from the International Skin Imaging Collaboration) and the other (model A+P) on an additional 1,536 pediatric images. We compared performance between the two models on adult and pediatric held-out test images separately using the area under the receiver operating characteristic curve. We then used Gradient-weighted Class Activation Maps and background skin masking to understand the contributions of the lesion versus background skin to algorithm decision making. Adding images from a pediatric population with different epidemiological and visual patterns to current reference standard datasets improved algorithm performance on pediatric images without diminishing performance on adult images. This suggests a way that dermatologic artificial intelligence models can be made more generalizable. The presence of background skin was important to the pediatric-specific improvement seen between models. Our study highlights the importance of carefully curated and labeled data from diverse inputs to improve the generalizability of AI models for dermatology, in this case applied to dermoscopic images of adult and pediatric lesions to improve melanoma detection.

Agreement Between Experts and an Untrained Crowd for Identifying Dermoscopic Features Using a Gamified App: Reader Feasibility Study.

BACKGROUND: Dermoscopy is commonly used for the evaluation of pigmented lesions, but agreement between experts for identification of dermoscopic structures is known to be relatively poor. Expert labeling of medical data is a bottleneck in the development of machine learning (ML) tools, and crowdsourcing has been demonstrated as a cost- and time-efficient method for the annotation of medical images. OBJECTIVE: The aim of this study is to demonstrate that crowdsourcing can be used to label basic dermoscopic structures from images of pigmented lesions with similar reliability to a group of experts. METHODS: First, we obtained labels of 248 images of melanocytic lesions with 31 dermoscopic "subfeatures" labeled by 20 dermoscopy experts. These were then collapsed into 6 dermoscopic "superfeatures" based on structural similarity, due to low interrater reliability (IRR): dots, globules, lines, network structures, regression structures, and vessels. These images were then used as the gold standard for the crowd study. The commercial platform DiagnosUs was used to obtain annotations from a nonexpert crowd for the presence or absence of the 6 superfeatures in each of the 248 images. We replicated this methodology with a group of 7 dermatologists to allow direct comparison with the nonexpert crowd. The Cohen κ value was used to measure agreement across raters. RESULTS: In total, we obtained 139,731 ratings of the 6 dermoscopic superfeatures from the crowd. There was relatively lower agreement for the identification of dots and globules (the median κ values were 0.526 and 0.395, respectively), whereas network structures and vessels showed the highest agreement (the median κ values were 0.581 and 0.798, respectively). This pattern was also seen among the expert raters, who had median κ values of 0.483 and 0.517 for dots and globules, respectively, and 0.758 and 0.790 for network structures and vessels. The median κ values between nonexperts and thresholded average-expert readers were 0.709 for dots, 0.719 for globules, 0.714 for lines, 0.838 for network structures, 0.818 for regression structures, and 0.728 for vessels. CONCLUSIONS: This study confirmed that IRR for different dermoscopic features varied among a group of experts; a similar pattern was observed in a nonexpert crowd. There was good or excellent agreement for each of the 6 superfeatures between the crowd and the experts, highlighting the similar reliability of the crowd for labeling dermoscopic images. This confirms the feasibility and dependability of using crowdsourcing as a scalable solution to annotate large sets of dermoscopic images, with several potential clinical and educational applications, including the development of novel, explainable ML tools.

Theory-Based Approaches to Support Dermoscopic Image Interpretation Education: A Review of the Literature.

Introduction: Efficient interpretation of dermoscopic images relies on pattern recognition, and the development of expert-level proficiency typically requires extensive training and years of practice. While traditional methods of transferring knowledge have proven effective, technological advances may significantly improve upon these strategies and better equip dermoscopy learners with the pattern recognition skills required for real-world practice. Objectives: A narrative review of the literature was performed to explore emerging directions in medical image interpretation education that may enhance dermoscopy education. This article represents the first of a two-part review series on this topic. Methods: To promote innovation in dermoscopy education, the International Skin Imaging Collaborative (ISIC) assembled a 12-member Education Working Group that comprises international dermoscopy experts and educational scientists. Based on a preliminary literature review and their experiences as educators, the group developed and refined a list of innovative approaches through multiple rounds of discussion and feedback. For each approach, literature searches were performed for relevant articles. Results: Through a consensus-based approach, the group identified a number of emerging directions in image interpretation education. The following theory-based approaches will be discussed in this first part: whole-task learning, microlearning, perceptual learning, and adaptive learning. Conclusions: Compared to traditional methods, these theory-based approaches may enhance dermoscopy education by making learning more engaging and interactive and reducing the amount of time required to develop expert-level pattern recognition skills. Further exploration is needed to determine how these approaches can be seamlessly and successfully integrated to optimize dermoscopy education.

Instructional Strategies to Enhance Dermoscopic Image Interpretation Education: a Review of the Literature.

Introduction: In image interpretation education, many educators have shifted away from traditional methods that involve passive instruction and fragmented learning to interactive ones that promote active engagement and integrated knowledge. By training pattern recognition skills in an effective manner, these interactive approaches provide a promising direction for dermoscopy education. Objectives: A narrative review of the literature was performed to probe emerging directions in medical image interpretation education that may support dermoscopy education. This article represents the second of a two-part review series. Methods: To promote innovation in dermoscopy education, the International Skin Imaging Collaborative (ISIC) assembled an Education Working Group that comprises international dermoscopy experts and educational scientists. Based on a preliminary literature review and their experiences as educators, the group developed and refined a list of innovative approaches through multiple rounds of discussion and feedback. For each approach, literature searches were performed for relevant articles. Results: Through a consensus-based approach, the group identified a number of theory-based approaches, as discussed in the first part of this series. The group also acknowledged the role of motivation, metacognition, and early failures in optimizing the learning process. Other promising teaching tools included gamification, social media, and perceptual and adaptive learning modules (PALMs). Conclusions: Over the years, many dermoscopy educators may have intuitively adopted these instructional strategies in response to learner feedback, personal observations, and changes in the learning environment. For dermoscopy training, PALMs may be especially valuable in that they provide immediate feedback and adapt the training schedule to the individual's performance.

Validation of artificial intelligence prediction models for skin cancer diagnosis using dermoscopy images: the 2019 International Skin Imaging Collaboration Grand Challenge.

BACKGROUND: Previous studies of artificial intelligence (AI) applied to dermatology have shown AI to have higher diagnostic classification accuracy than expert dermatologists; however, these studies did not adequately assess clinically realistic scenarios, such as how AI systems behave when presented with images of disease categories that are not included in the training dataset or images drawn from statistical distributions with significant shifts from training distributions. We aimed to simulate these real-world scenarios and evaluate the effects of image source institution, diagnoses outside of the training set, and other image artifacts on classification accuracy, with the goal of informing clinicians and regulatory agencies about safety and real-world accuracy. METHODS: We designed a large dermoscopic image classification challenge to quantify the performance of machine learning algorithms for the task of skin cancer classification from dermoscopic images, and how this performance is affected by shifts in statistical distributions of data, disease categories not represented in training datasets, and imaging or lesion artifacts. Factors that might be beneficial to performance, such as clinical metadata and external training data collected by challenge participants, were also evaluated. 25 331 training images collected from two datasets (in Vienna [HAM10000] and Barcelona [BCN20000]) between Jan 1, 2000, and Dec 31, 2018, across eight skin diseases, were provided to challenge participants to design appropriate algorithms. The trained algorithms were then tested for balanced accuracy against the HAM10000 and BCN20000 test datasets and data from countries not included in the training dataset (Turkey, New Zealand, Sweden, and Argentina). Test datasets contained images of all diagnostic categories available in training plus other diagnoses not included in training data (not trained category). We compared the performance of the algorithms against that of 18 dermatologists in a simulated setting that reflected intended clinical use. FINDINGS: 64 teams submitted 129 state-of-the-art algorithm predictions on a test set of 8238 images. The best performing algorithm achieved 58·8% balanced accuracy on the BCN20000 data, which was designed to better reflect realistic clinical scenarios, compared with 82·0% balanced accuracy on HAM10000, which was used in a previously published benchmark. Shifted statistical distributions and disease categories not included in training data contributed to decreases in accuracy. Image artifacts, including hair, pen markings, ulceration, and imaging source institution, decreased accuracy in a complex manner that varied based on the underlying diagnosis. When comparing algorithms to expert dermatologists (2460 ratings on 1269 images), algorithms performed better than experts in most categories, except for actinic keratoses (similar accuracy on average) and images from categories not included in training data (26% correct for experts vs 6% correct for algorithms, p<0·0001). For the top 25 submitted algorithms, 47·1% of the images from categories not included in training data were misclassified as malignant diagnoses, which would lead to a substantial number of unnecessary biopsies if current state-of-the-art AI technologies were clinically deployed. INTERPRETATION: We have identified specific deficiencies and safety issues in AI diagnostic systems for skin cancer that should be addressed in future diagnostic evaluation protocols to improve safety and reliability in clinical practice. FUNDING: Melanoma Research Alliance and La Marató de TV3.

A patient-centric dataset of images and metadata for identifying melanomas using clinical context.

Prior skin image datasets have not addressed patient-level information obtained from multiple skin lesions from the same patient. Though artificial intelligence classification algorithms have achieved expert-level performance in controlled studies examining single images, in practice dermatologists base their judgment holistically from multiple lesions on the same patient. The 2020 SIIM-ISIC Melanoma Classification challenge dataset described herein was constructed to address this discrepancy between prior challenges and clinical practice, providing for each image in the dataset an identifier allowing lesions from the same patient to be mapped to one another. This patient-level contextual information is frequently used by clinicians to diagnose melanoma and is especially useful in ruling out false positives in patients with many atypical nevi. The dataset represents 2,056 patients (20.8% with at least one melanoma, 79.2% with zero melanomas) from three continents with an average of 16 lesions per patient, consisting of 33,126 dermoscopic images and 584 (1.8%) histopathologically confirmed melanomas compared with benign melanoma mimickers.

The Role of DICOM in Artificial Intelligence for Skin Disease.

There is optimism that artificial intelligence (AI) will result in positive clinical outcomes, which is driving research and investment in the use of AI for skin disease. At present, AI for skin disease is embedded in research and development and not practiced widely in clinical dermatology. Clinical dermatology is also undergoing a technological transformation in terms of the development and adoption of standards that optimizes the quality use of imaging. Digital Imaging and Communications in Medicine (DICOM) is the international standard for medical imaging. DICOM is a continually evolving standard. There is considerable effort being invested in developing dermatology-specific extensions to the DICOM standard. The ability to encode relevant metadata and afford interoperability with the digital health ecosystem (e.g., image repositories, electronic medical records) has driven the initial impetus in the adoption of DICOM for dermatology. DICOM has a dedicated working group whose role is to develop a mechanism to support AI workflows and encode AI artifacts. DICOM can improve AI workflows by encoding derived objects (e.g., secondary images, visual explainability maps, AI algorithm output) and the efficient curation of multi-institutional datasets for machine learning training, testing, and validation. This can be achieved using DICOM mechanisms such as standardized image formats and metadata, metadata-based image retrieval, and de-identification protocols. DICOM can address several important technological and workflow challenges for the implementation of AI. However, many other technological, ethical, regulatory, medicolegal, and workforce barriers will need to be addressed before DICOM and AI can be used effectively in dermatology.

Neural predictors and effects of cognitive behavioral therapy for depression: the role of emotional reactivity and regulation.

BACKGROUND: Cognitive behavioral therapy (CBT) is an effective treatment for many patients suffering from major depressive disorder (MDD), but predictors of treatment outcome are lacking, and little is known about its neural mechanisms. We recently identified longitudinal changes in neural correlates of conscious emotion regulation that scaled with clinical responses to CBT for MDD, using a negative autobiographical memory-based task. METHODS: We now examine the neural correlates of emotional reactivity and emotion regulation during viewing of emotionally salient images as predictors of treatment outcome with CBT for MDD, and the relationship between longitudinal change in functional magnetic resonance imaging (fMRI) responses and clinical outcomes. Thirty-two participants with current MDD underwent baseline MRI scanning followed by 14 sessions of CBT. The fMRI task measured emotional reactivity and emotion regulation on separate trials using standardized images from the International Affective Pictures System. Twenty-one participants completed post-treatment scanning. Last observation carried forward was used to estimate clinical outcome for non-completers. RESULTS: Pre-treatment emotional reactivity Blood Oxygen Level-Dependent (BOLD) signal within hippocampus including CA1 predicted worse treatment outcome. In contrast, better treatment outcome was associated with increased down-regulation of BOLD activity during emotion regulation from time 1 to time 2 in precuneus, occipital cortex, and middle frontal gyrus. CONCLUSIONS: CBT may modulate the neural circuitry of emotion regulation. The neural correlates of emotional reactivity may be more strongly predictive of CBT outcome. The finding that treatment outcome was predicted by BOLD signal in CA1 may suggest overgeneralized memory as a negative prognostic factor in CBT outcome.

Prestimulus Activity in the Cingulo-Opercular Network Predicts Memory for Naturalistic Episodic Experience.

Human memory is strongly influenced by brain states occurring before an event, yet we know little about the underlying mechanisms. We found that activity in the cingulo-opercular network (including bilateral anterior insula [aI] and anterior prefrontal cortex [aPFC]) seconds before an event begins can predict whether this event will subsequently be remembered. We then tested how activity in the cingulo-opercular network shapes memory performance. Our findings indicate that prestimulus cingulo-opercular activity affects memory performance by opposingly modulating subsequent activity in two sets of regions previously linked to encoding and retrieval of episodic information. Specifically, higher prestimulus cingulo-opercular activity was associated with a subsequent increase in activity in temporal regions previously linked to encoding and with a subsequent reduction in activity within a set of regions thought to play a role in retrieval and self-referential processing. Together, these findings suggest that prestimulus attentional states modulate memory for real-life events by enhancing encoding and possibly by dampening interference from competing memory substrates.