Summary of the National Cancer Institute 2023 Virtual Workshop on Medical Image De-identification-Part 2: Pathology Whole Slide Image De-identification, De-facing, the Role of AI in Image De-identification, and the NCI MIDI Datasets and Pipeline.

De-identification of medical images intended for research is a core requirement for data sharing initiatives, particularly as the demand for data for artificial intelligence (AI) applications grows. The Center for Biomedical Informatics and Information Technology (CBIIT) of the United States National Cancer Institute (NCI) convened a two half-day virtual workshop with the intent of summarizing the state of the art in de-identification technology and processes and exploring interesting aspects of the subject. This paper summarizes the highlights of the second day of the workshop, the recordings and presentations of which are publicly available for review. The topics covered included pathology whole slide image de-identification, de-facing, the role of AI in image de-identification, and the NCI Medical Image De-Identification Initiative (MIDI) datasets and pipeline.

Evolutionary Strategies AI Addresses Multiple Technical Challenges in Deep Learning Deployment: Proof-of-Principle Demonstration for Neuroblastoma Brain Metastasis Detection.

Two significant obstacles hinder the advancement of Radiology AI. The first is the challenge of overfitting, where small training data sets can result in unreliable outcomes. The second challenge is the need for more generalizability, the lack of which creates difficulties in implementing the technology across various institutions and practices. A recent innovation, deep neuroevolution (DNE), has been introduced to tackle the overfitting issue by training on small data sets and producing accurate predictions. However, the generalizability of DNE has yet to be proven. This paper strives to overcome this barrier by demonstrating that DNE can achieve satisfactory results in diverse external validation sets. The main innovation of the work is thus showing that DNE can generalize to varied outside data. Our example use case is predicting brain metastasis from neuroblastoma, emphasizing the importance of AI with limited data sets. Despite image collection and labeling advancements, rare diseases will always constrain data availability. We optimized a convolutional neural network (CNN) with DNE to demonstrate generalizability. We trained the CNN with 60 MRI images and tested it on a separate diverse collection of images from over 50 institutions. For comparison, we also trained with the more traditional stochastic gradient descent (SGD) method, with the two variants of (1) training from scratch and (2) transfer learning. Our results show that DNE demonstrates excellent generalizability with 97% accuracy on the heterogeneous testing set, while neither form of SGD could reach 60% accuracy. DNE's ability to generalize from small training sets to external and diverse testing sets suggests that it or similar approaches may play an integral role in improving the clinical performance of AI.

Lung Cancer Survival Trends in the Veterans Health Administration.

INTRODUCTION: Lung cancer survival is improving in the United States. We investigated whether there was a similar trend within the Veterans Health Administration (VHA), the largest integrated healthcare system in the United States. MATERIALS AND METHODS: Data from the Veterans Affairs Central Cancer Registry were analyzed for temporal survival trends using Kaplan-Meier estimates and linear regression. RESULTS: A total number of 54,922 Veterans were identified with lung cancer diagnosed from 2010 to 2017. Histologies were classified as non-small-cell lung cancer (NSCLC) (64.2%), small cell lung cancer (SCLC) (12.9%), and 'other' (22.9%). The proportion with stage I increased from 18.1% to 30.4%, while stage IV decreased from 38.9% to 34.6% (both P < .001). The 3-year overall survival (OS) improved for stage I (58.6% to 68.4%, P < .001), stage II (35.5% to 48.4%, P < .001), stage III (18.7% to 29.4%, P < .001), and stage IV (3.4% to 7.8%, P < .001). For NSCLC, the median OS increased from 12 to 21 months (P < .001), and the 3-year OS increased from 24.1% to 38.3% (P < .001). For SCLC, the median OS remained unchanged (8 to 9 months, P = .10), while the 3-year OS increased from 9.1% to 12.3% (P = .014). Compared to White Veterans, Black Veterans with NSCLC had similar OS (P = .81), and those with SCLC had higher OS (P = .003). CONCLUSION: Lung cancer survival is improving within the VHA. Compared to White Veterans, Black Veterans had similar or higher survival rates. The observed racial equity in outcomes within a geographically and socioeconomically diverse population warrants further investigation to better understand and replicate this achievement in other healthcare systems.

Expert Agreement on the Presence and Spatial Localization of Melanocytic Features in Dermoscopy.

Dermoscopy aids in melanoma detection; however, agreement on dermoscopic features, including those of high clinical relevance, remains poor. In this study, we attempted to evaluate agreement among experts on exemplar images not only for the presence of melanocytic-specific features but also for spatial localization. This was a cross-sectional, multicenter, observational study. Dermoscopy images exhibiting at least 1 of 31 melanocytic-specific features were submitted by 25 world experts as exemplars. Using a web-based platform that allows for image markup of specific contrast-defined regions (superpixels), 20 expert readers annotated 248 dermoscopic images in collections of 62 images. Each collection was reviewed by five independent readers. A total of 4,507 feature observations were performed. Good-to-excellent agreement was found for 14 of 31 features (45.2%), with eight achieving excellent agreement (Gwet's AC >0.75) and seven of them being melanoma-specific features. These features were peppering/granularity (0.91), shiny white streaks (0.89), typical pigment network (0.83), blotch irregular (0.82), negative network (0.81), irregular globules (0.78), dotted vessels (0.77), and blue-whitish veil (0.76). By utilizing an exemplar dataset, a good-to-excellent agreement was found for 14 features that have previously been shown useful in discriminating nevi from melanoma. All images are public (www.isic-archive.com) and can be used for education, scientific communication, and machine learning experiments.

A population-level digital histologic biomarker for enhanced prognosis of invasive breast cancer.

Breast cancer is a heterogeneous disease with variable survival outcomes. Pathologists grade the microscopic appearance of breast tissue using the Nottingham criteria, which are qualitative and do not account for noncancerous elements within the tumor microenvironment. Here we present the Histomic Prognostic Signature (HiPS), a comprehensive, interpretable scoring of the survival risk incurred by breast tumor microenvironment morphology. HiPS uses deep learning to accurately map cellular and tissue structures to measure epithelial, stromal, immune, and spatial interaction features. It was developed using a population-level cohort from the Cancer Prevention Study-II and validated using data from three independent cohorts, including the Prostate, Lung, Colorectal, and Ovarian Cancer trial, Cancer Prevention Study-3, and The Cancer Genome Atlas. HiPS consistently outperformed pathologists in predicting survival outcomes, independent of tumor-node-metastasis stage and pertinent variables. This was largely driven by stromal and immune features. In conclusion, HiPS is a robustly validated biomarker to support pathologists and improve patient prognosis.

Toward a generalizable machine learning workflow for neurodegenerative disease staging with focus on neurofibrillary tangles.

Machine learning (ML) has increasingly been used to assist and expand current practices in neuropathology. However, generating large imaging datasets with quality labels is challenging in fields which demand high levels of expertise. Further complicating matters is the often seen disagreement between experts in neuropathology-related tasks, both at the case level and at a more granular level. Neurofibrillary tangles (NFTs) are a hallmark pathological feature of Alzheimer disease, and are associated with disease progression which warrants further investigation and granular quantification at a scale not currently accessible in routine human assessment. In this work, we first provide a baseline of annotator/rater agreement for the tasks of Braak NFT staging between experts and NFT detection using both experts and novices in neuropathology. We use a whole-slide-image (WSI) cohort of neuropathology cases from Emory University Hospital immunohistochemically stained for Tau. We develop a workflow for gathering annotations of the early stage formation of NFTs (Pre-NFTs) and mature intracellular (iNFTs) and show ML models can be trained to learn annotator nuances for the task of NFT detection in WSIs. We utilize a model-assisted-labeling approach and demonstrate ML models can be used to aid in labeling large datasets efficiently. We also show these models can be used to extract case-level features, which predict Braak NFT stages comparable to expert human raters, and do so at scale. This study provides a generalizable workflow for various pathology and related fields, and also provides a technique for accomplishing a high-level neuropathology task with limited human annotations.

Survey of Neuroanatomic Sampling and Staining Procedures in Alzheimer Disease Research Center Brain Banks.

The collection of post-mortem brain tissue has been a core function of the Alzheimer Disease Research Center's (ADRCs) network located within the United States since its inception. Individual brain banks and centers follow detailed protocols to record, store, and manage complex datasets that include clinical data, demographics, and when post-mortem tissue is available, a detailed neuropathological assessment. Since each institution often has specific research foci, there can be variability in tissue collection and processing workflows. While published guidelines exist for select diseases, such as those put forth by the National Institute on Aging and Alzheimer Association (NIA-AA), it is of importance to denote the current practices across institutions. To this end a survey was developed and sent to United States based brain bank leaders, collecting data on brain region sampling, including anatomic landmarks used, staining (including antibodies used), as well as whole-slide-image scanning hardware. We distributed this survey to 40 brain banks and obtained a response rate of 95% (38 / 40). Most brain banks followed guidelines defined by the NIA-AA, having H&E staining in all recommended regions and targeted region-based amyloid beta, tau, and alpha-synuclein immunohistochemical staining. However, sampling consistency varied related to key anatomic landmarks/locations in select regions, such as the striatum, periventricular white matter, and parietal cortex. This study highlights the diversity and similarities amongst brain banks and discusses considerations when amalgamating data/samples across multiple centers. This survey aids in establishing benchmarks to enhance dialogues on divergent workflows in a feasible way.

A population-level computational histologic signature for invasive breast cancer prognosis.

Breast cancer is a heterogeneous disease with variable survival outcomes. Pathologists grade the microscopic appearance of breast tissue using the Nottingham criteria, which is qualitative and does not account for non-cancerous elements within the tumor microenvironment (TME). We present the Histomic Prognostic Signature (HiPS), a comprehensive, interpretable scoring of the survival risk incurred by breast TME morphology. HiPS uses deep learning to accurately map cellular and tissue structures in order to measure epithelial, stromal, immune, and spatial interaction features. It was developed using a population-level cohort from the Cancer Prevention Study (CPS)-II and validated using data from three independent cohorts, including the PLCO trial, CPS-3, and The Cancer Genome Atlas. HiPS consistently outperformed pathologists' performance in predicting survival outcomes, independent of TNM stage and pertinent variables. This was largely driven by stromal and immune features. In conclusion, HiPS is a robustly validated biomarker to support pathologists and improve prognosis.

A Trial of Adding Lung Protective Strategies to Existing Enhanced Recovery After Surgery Protocols and Its Effect on Improving Postoperative Lung Function.

Background: With this rising popularization of enhanced recovery after surgery (ERAS) protocols, it is important to ask if the current and developing pathways are fully comprehensive for the patient's perioperative experience. Many current pathways discuss aspects of care including fluid management, pain management, and anti-emetic medication regiments, but few delineate recommendations for lung protective strategies. The hypothesis was that intraoperative lung protective strategies would results in improved postoperative lung function. Methods: One hundred patients at the Medical University of South Carolina undergoing hepatobiliary and colorectal surgeries were randomized to receive intraoperative lung protective techniques or a standard intraoperative ventilation management. Three maximum vital capacity breaths were recorded preoperatively, and postoperatively 30 min, 1 h, and 2 h after anesthesia stop time. Average maximum capacity breaths from all four data collection interactions were analyzed between both study and control cohorts. Results: There was no significant difference in the preoperative inspiratory capacity between the control and the ERAS group (2,043.3 ± 628.4 mL vs. 2,012.2 ± 895.2 mL; P = 0.84). Additional data analysis showed no statistically significant difference between ERAS and control groups: total average of the inspiratory capacity volumes (1,253.5 ± 593.7 mL vs. 1,390.4 ± 964.9 mL; P = 0.47), preoperative oxygen saturation (97.76±2.3% vs. 98.04±1.7%; P = 0.50), the postoperative oxygen saturation (98.51±1.4% vs. 96.83±14.2%; P = 0.40), and change in inspiratory capacity (95% confidence interval (CI) (-211.2 - 366.6); P = 0.60). Conclusions: No statistically significant difference in postoperative inspiratory capacities were seen after the implementation of intraoperative lung protective strategies. The addition of other indicators of postoperative lung function like pneumonia incidence or length of inpatient stay while receiving oxygen treatment could provide a fuller picture in future studies, but a higher power will be needed.

Fast, light, and scalable: harnessing data-mined line annotations for automated tumor segmentation on brain MRI.

OBJECTIVES: While fully supervised learning can yield high-performing segmentation models, the effort required to manually segment large training sets limits practical utility. We investigate whether data mined line annotations can facilitate brain MRI tumor segmentation model development without requiring manually segmented training data. METHODS: In this retrospective study, a tumor detection model trained using clinical line annotations mined from PACS was leveraged with unsupervised segmentation to generate pseudo-masks of enhancing tumors on T1-weighted post-contrast images (9911 image slices; 3449 adult patients). Baseline segmentation models were trained and employed within a semi-supervised learning (SSL) framework to refine the pseudo-masks. Following each self-refinement cycle, a new model was trained and tested on a held-out set of 319 manually segmented image slices (93 adult patients), with the SSL cycles continuing until Dice score coefficient (DSC) peaked. DSCs were compared using bootstrap resampling. Utilizing the best-performing models, two inference methods were compared: (1) conventional full-image segmentation, and (2) a hybrid method augmenting full-image segmentation with detection plus image patch segmentation. RESULTS: Baseline segmentation models achieved DSC of 0.768 (U-Net), 0.831 (Mask R-CNN), and 0.838 (HRNet), improving with self-refinement to 0.798, 0.871, and 0.873 (each p < 0.001), respectively. Hybrid inference outperformed full image segmentation alone: DSC 0.884 (Mask R-CNN) vs. 0.873 (HRNet), p < 0.001. CONCLUSIONS: Line annotations mined from PACS can be harnessed within an automated pipeline to produce accurate brain MRI tumor segmentation models without manually segmented training data, providing a mechanism to rapidly establish tumor segmentation capabilities across radiology modalities. KEY POINTS: • A brain MRI tumor detection model trained using clinical line measurement annotations mined from PACS was leveraged to automatically generate tumor segmentation pseudo-masks. • An iterative self-refinement process automatically improved pseudo-mask quality, with the best-performing segmentation pipeline achieving a Dice score of 0.884 on a held-out test set. • Tumor line measurement annotations generated in routine clinical radiology practice can be harnessed to develop high-performing segmentation models without manually segmented training data, providing a mechanism to rapidly establish tumor segmentation capabilities across radiology modalities.

Striatal Subregion-selective Dysregulated Dopamine Receptor-mediated Intracellular Signaling in a Model of DOPA-responsive Dystonia.

Although the mechanisms underlying dystonia are largely unknown, dystonia is often associated with abnormal dopamine neurotransmission. DOPA-responsive dystonia (DRD) is a prototype disorder for understanding dopamine dysfunction in dystonia because it is caused by mutations in genes necessary for the synthesis of dopamine and alleviated by the indirect-acting dopamine agonist l-DOPA. Although adaptations in striatal dopamine receptor-mediated intracellular signaling have been studied extensively in models of Parkinson's disease, another movement disorders associated with dopamine deficiency, little is known about dopaminergic adaptations in dystonia. To identify the dopamine receptor-mediated intracellular signaling associated with dystonia, we used immunohistochemistry to quantify striatal protein kinase A activity and extracellular signal-related kinase (ERK) phosphorylation after dopaminergic challenges in a knockin mouse model of DRD. l-DOPA treatment induced the phosphorylation of both protein kinase A substrates and ERK largely in D1 dopamine receptor-expressing striatal neurons. As expected, this response was blocked by pretreatment with the D1 dopamine receptor antagonist SCH23390. The D2 dopamine receptor antagonist raclopride also significantly reduced the phosphorylation of ERK; this contrasts with models of parkinsonism in which l-DOPA-induced ERK phosphorylation is not mediated by D2 dopamine receptors. Further, the dysregulated signaling was dependent on striatal subdomains whereby ERK phosphorylation was largely confined to dorsomedial (associative) striatum while the dorsolateral (sensorimotor) striatum was unresponsive. This complex interaction between striatal functional domains and dysregulated dopamine-receptor mediated responses has not been observed in other models of dopamine deficiency, such as parkinsonism, suggesting that regional variation in dopamine-mediated neurotransmission may be a hallmark of dystonia.

An Automated Pipeline for Differential Cell Counts on Whole-Slide Bone Marrow Aspirate Smears.

The pathologic diagnosis of bone marrow disorders relies in part on the microscopic analysis of bone marrow aspirate (BMA) smears and the manual counting of marrow nucleated cells to obtain a differential cell count (DCC). This manual process has significant limitations, including the analysis of only a small subset of optimal slide areas and nucleated cells, as well as interobserver variability due to differences in cell selection and classification. To address these shortcomings, we developed an automated machine learning-based pipeline for obtaining 11-component DCCs on whole-slide BMAs. This pipeline uses a sequential process of identifying optimal BMA regions with high proportions of marrow nucleated cells, detecting individual cells within these optimal areas, and classifying these cells into 1 of 11 DCC components. Convolutional neural network models were trained on 396,048 BMA region, 28,914 cell boundary, and 1,510,976 cell class images from manual annotations. The resulting automated pipeline produced 11-component DCCs that demonstrated a high statistical and diagnostic concordance with manual DCCs among a heterogeneous group of testing BMA slides with varying pathologies and cellularities. Additionally, we demonstrated that an automated analysis can reduce the intraslide variance in DCCs by analyzing the whole slide and marrow nucleated cells within all optimal regions. Finally, the pipeline outputs of region classification, cell detection, and cell classification can be visualized using whole-slide image analysis software. This study demonstrates the feasibility of a fully automated pipeline for generating DCCs on scanned whole-slide BMA images, with the potential for improving the current standard of practice for utilizing BMA smears in the laboratory analysis of hematologic disorders.

The Association of Improved Overall Survival with NSAIDs in Non-Small Cell Lung Cancer Patients Receiving Immune Checkpoint Inhibitors.

BACKGROUND: Immune checkpoint inhibitors (ICI) are commonly used in the management of patients with advanced non-small cell lung cancer (NSCLC), but response is suboptimal. Preclinical data suggest ICI efficacy may be enhanced with concomitant nonsteroidal anti-inflammatory (NSAID) medications. PATIENTS AND METHODS: In this retrospective study, the Veterans Health Administration Corporate Data Warehouse was queried for patients diagnosed with NSCLC and treated with ICI from 2010 to 2018. Concomitant NSAID use was defined as NSAID dispensation by a VA pharmacy within 90 days of the any ICI infusion. To mitigate immortal time bias, patients who started NSAIDs 60 or more days after ICI initiation were excluded from analysis. Survival was measured from start of ICI. RESULTS: We identified 3634 patients with NSCLC receiving ICI; 2336 (64.3%) were exposed to concomitant NSAIDs. On multivariable analysis, NSAIDs were associated with better overall survival (HR = 0.90; 95% CI, 0.83-0.98; P = .010). When stratifying by NSAID type, diclofenac was the only NSAID with significant association with overall survival (HR = 0.75; 95% CI, 0.68-0.83; P < .001). Propensity score matching of the original cohort yielded 1251 patients per cohort balanced in characteristics. NSAIDs remained associated with improved overall survival (HR = 0.85; 95% CI, 0.78-0.92; P < .001). CONCLUSION: This study of Veterans with NSCLC treated with ICI demonstrated that concomitant NSAIDs are associated with longer OS. This may indicate that NSAIDs can enhance ICI-induced antitumor immunity and should prospectively validated.

The status of digital pathology and associated infrastructure within Alzheimer's Disease Centers.

Digital pathology (DP) has transformative potential, especially for Alzheimer disease and related disorders. However, infrastructure barriers may limit adoption. To provide benchmarks and insights into implementation barriers, a survey was conducted in 2019 within National Institutes of Health's Alzheimer's Disease Centers (ADCs). Questions covered infrastructure, funding sources, and data management related to digital pathology. Of the 35 ADCs to which the survey was sent, 33 responded. Most respondents (81%) stated that their ADC had digital slide scanner access, with the most frequent brand being Aperio/Leica (62.9%). Approximately a third of respondents stated there were fees to utilize the scanner. For DP and machine learning (ML) resources, 41% of respondents stated none was supported by their ADC. For scanner purchasing and operations, 50% of respondents stated they received institutional support. Some were unsure of the file size of scanned digital images (37%) and total amount of storage space files occupied (50%). Most (76%) were aware of other departments at their institution working with ML; a similar (76%) percentage were unaware of multiuniversity or industry partnerships. These results demonstrate many ADCs have access to a digital slide scanner; additional investigations are needed to further understand hurdles to implement DP and ML workflows.

Impact of concomitant fibrates on immunotherapy outcomes for advanced non-small cell lung cancer.

BACKGROUND: Peroxisome proliferator-activated receptor agonists such as fibrates restore oxidative metabolism in cytotoxic T-lymphocytes, thereby enhancing response to immune checkpoint inhibitors (ICI) in preclinical models. However, there is no evidence in humans on the clinical impact of fibrates as an adjunct to ICI. METHODS: In this cohort study of Veterans with non-small cell lung cancer (NSCLC) receiving ICI, fibrate exposure was defined as a prescription filled within 90 days of an ICI infusion. Overall survival (OS), measured from the start of ICI, was compared between exposed and unexposed Veterans. Cox multivariable analysis (MVA) was used to identify factors associated with OS. A sensitivity analysis of Veterans with stage IV NSCLC who received docetaxel without ICI was similarly performed. RESULTS: The ICI cohort included 3593 Veterans, of whom 301 (8.5%) coincidentally received a fibrate. Veterans receiving fibrates were more likely to be older, white, male, and married, and to have greater comorbidity burden, but less likely to receive chemotherapy. Coincidental fibrates were associated with improved OS both on MVA (HR 0.86, 95%CI 0.75-0.99) and in a matched subset (HR 0.75, 95%CI 0.63-0.90). In contrast, among the cohort of 968 Veterans treated with chemotherapy, fibrates did not have a significant impact on OS by MVA (HR 0.99, 95%CI 0.79-1.25) or in a matched subset (HR 1.02, 95%CI CI 0.75-1.39). CONCLUSIONS: Concomitant fibrates are associated with improved OS among NSCLC patients receiving ICI but not among those receiving chemotherapy. This hypothesis-generating observation supports a potential role for fibrates as an adjunct to immunotherapy.

Reexpansion Pulmonary Edema following Tube Thoracostomy in a Pediatric Patient with Anterior Mediastinal Mass.

Reexpansion pulmonary edema (RPE) is an exceedingly rare and potentially fatal complication of a rapidly reexpanded lung following evacuation of air or fluid from the pleural space secondary to conditions such as a mediastinal mass, pleural effusion, or pneumothorax. Clinical presentations can range from mild radiographic changes to acute respiratory failure and hemodynamic instability. The rapidly progressive nature of the disease makes it important for clinicians to appropriately diagnose and manage patients who develop RPE. We present a case of a child with a large malignant pleural effusion who developed severe RPE after tube thoracostomy and ultimately required venoarterial extracorporeal membrane oxygenation (VA-ECMO). The patient was 7-year-old Caucasian male with newly diagnosed ambiguous T cell myeloid leukemia. A chest computerized tomography (CT) demonstrated a large pleural effusion causing tracheal shift and left bronchus compression as well as an anterior mediastinal mass causing compression of the right atria and right ventricle. Tube thoracostomy was performed in the operating room (OR) with deep sedation. The procedure was complicated with hypoxemia, bradycardia, and pulseless cardiac arrest. After return of spontaneous circulation, the child continued to have refractory hypoxemia, profound hypotension, and frothy secretions. Endotracheal intubation was performed with a size 5.0 cuffed endotracheal tube. Chest radiograph demonstrated opacification of the left hemithorax with chest infiltrates. Patient required VA-ECMO for circulatory support. Supportive therapy of RPE was continued and decannulation was done on day three. Tracheal extubation was performed on day five.

NuCLS: A scalable crowdsourcing approach and dataset for nucleus classification and segmentation in breast cancer.

BACKGROUND: Deep learning enables accurate high-resolution mapping of cells and tissue structures that can serve as the foundation of interpretable machine-learning models for computational pathology. However, generating adequate labels for these structures is a critical barrier, given the time and effort required from pathologists. RESULTS: This article describes a novel collaborative framework for engaging crowds of medical students and pathologists to produce quality labels for cell nuclei. We used this approach to produce the NuCLS dataset, containing >220,000 annotations of cell nuclei in breast cancers. This builds on prior work labeling tissue regions to produce an integrated tissue region- and cell-level annotation dataset for training that is the largest such resource for multi-scale analysis of breast cancer histology. This article presents data and analysis results for single and multi-rater annotations from both non-experts and pathologists. We present a novel workflow that uses algorithmic suggestions to collect accurate segmentation data without the need for laborious manual tracing of nuclei. Our results indicate that even noisy algorithmic suggestions do not adversely affect pathologist accuracy and can help non-experts improve annotation quality. We also present a new approach for inferring truth from multiple raters and show that non-experts can produce accurate annotations for visually distinctive classes. CONCLUSIONS: This study is the most extensive systematic exploration of the large-scale use of wisdom-of-the-crowd approaches to generate data for computational pathology applications.

A Diagnosis of Choriocarcinoma in a Parturient Presenting With Intracranial Hemorrhage.

Choriocarcinoma in a viable pregnancy is uncommon. The diagnosis can easily be missed when there is an explanation for the clinical symptoms that the cancer can mimic. We present the case of a primigravid patient whose choriocarcinoma was initially missed as a result of seemingly obvious explanations for her atypical history and disease manifestation. The patient is a Caucasian female at 30 weeks and 5 days of gestation who presented with persistent headaches and new-onset tonic-clonic seizures found on brain magnetic resonance imaging (MRI) to have a left intracranial hematoma, a 5 mm midline shift, and multiple foci of restricted diffusion. Cerebral angiogram demonstrated arteriovenous malformations (AVMs). The fetus was emergently delivered 1 week into hospitalization for non-reassuring fetal heart tracings in the setting of maternal lethargy secondary to continued AVM hemorrhage. The patient's hospital course was complicated by four episodes of intracranial bleeding and edema requiring neurosurgical intervention. Three weeks after hospitalization she was discharged to a rehabilitation center, shortly after which placental biopsy demonstrated choriocarcinoma. MRI after readmission demonstrated extensive metastatic disease and human chorionic gonadotropin (hCG) levels were greater than 225,000 mIU/mL. Despite two additional neurosurgical procedures and extensive chemotherapy the patient died 3 months after initial presentation. Choriocarcinoma is extremely rare in viable pregnancies, but it should be considered when a parturient presents with intracranial bleeding. A high level of suspicion and serial serum hCG levels may lead to early and potentially life-saving multidrug chemotherapy. With a broader differential, earlier hCG measurement, and earlier treatment, our patient may have survived.

Validation of artificial intelligence prediction models for skin cancer diagnosis using dermoscopy images: the 2019 International Skin Imaging Collaboration Grand Challenge.

BACKGROUND: Previous studies of artificial intelligence (AI) applied to dermatology have shown AI to have higher diagnostic classification accuracy than expert dermatologists; however, these studies did not adequately assess clinically realistic scenarios, such as how AI systems behave when presented with images of disease categories that are not included in the training dataset or images drawn from statistical distributions with significant shifts from training distributions. We aimed to simulate these real-world scenarios and evaluate the effects of image source institution, diagnoses outside of the training set, and other image artifacts on classification accuracy, with the goal of informing clinicians and regulatory agencies about safety and real-world accuracy. METHODS: We designed a large dermoscopic image classification challenge to quantify the performance of machine learning algorithms for the task of skin cancer classification from dermoscopic images, and how this performance is affected by shifts in statistical distributions of data, disease categories not represented in training datasets, and imaging or lesion artifacts. Factors that might be beneficial to performance, such as clinical metadata and external training data collected by challenge participants, were also evaluated. 25 331 training images collected from two datasets (in Vienna [HAM10000] and Barcelona [BCN20000]) between Jan 1, 2000, and Dec 31, 2018, across eight skin diseases, were provided to challenge participants to design appropriate algorithms. The trained algorithms were then tested for balanced accuracy against the HAM10000 and BCN20000 test datasets and data from countries not included in the training dataset (Turkey, New Zealand, Sweden, and Argentina). Test datasets contained images of all diagnostic categories available in training plus other diagnoses not included in training data (not trained category). We compared the performance of the algorithms against that of 18 dermatologists in a simulated setting that reflected intended clinical use. FINDINGS: 64 teams submitted 129 state-of-the-art algorithm predictions on a test set of 8238 images. The best performing algorithm achieved 58·8% balanced accuracy on the BCN20000 data, which was designed to better reflect realistic clinical scenarios, compared with 82·0% balanced accuracy on HAM10000, which was used in a previously published benchmark. Shifted statistical distributions and disease categories not included in training data contributed to decreases in accuracy. Image artifacts, including hair, pen markings, ulceration, and imaging source institution, decreased accuracy in a complex manner that varied based on the underlying diagnosis. When comparing algorithms to expert dermatologists (2460 ratings on 1269 images), algorithms performed better than experts in most categories, except for actinic keratoses (similar accuracy on average) and images from categories not included in training data (26% correct for experts vs 6% correct for algorithms, p<0·0001). For the top 25 submitted algorithms, 47·1% of the images from categories not included in training data were misclassified as malignant diagnoses, which would lead to a substantial number of unnecessary biopsies if current state-of-the-art AI technologies were clinically deployed. INTERPRETATION: We have identified specific deficiencies and safety issues in AI diagnostic systems for skin cancer that should be addressed in future diagnostic evaluation protocols to improve safety and reliability in clinical practice. FUNDING: Melanoma Research Alliance and La Marató de TV3.