An Outbreak of Covid-19 on an Aircraft Carrier.

BACKGROUND: An outbreak of coronavirus disease 2019 (Covid-19) occurred on the U.S.S. Theodore Roosevelt, a nuclear-powered aircraft carrier with a crew of 4779 personnel. METHODS: We obtained clinical and demographic data for all crew members, including results of testing by real-time reverse-transcriptase polymerase chain reaction (rRT-PCR). All crew members were followed up for a minimum of 10 weeks, regardless of test results or the absence of symptoms. RESULTS: The crew was predominantly young (mean age, 27 years) and was in general good health, meeting U.S. Navy standards for sea duty. Over the course of the outbreak, 1271 crew members (26.6% of the crew) tested positive for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection by rRT-PCR testing, and more than 1000 infections were identified within 5 weeks after the first laboratory-confirmed infection. An additional 60 crew members had suspected Covid-19 (i.e., illness that met Council of State and Territorial Epidemiologists clinical criteria for Covid-19 without a positive test result). Among the crew members with laboratory-confirmed infection, 76.9% (978 of 1271) had no symptoms at the time that they tested positive and 55.0% had symptoms develop at any time during the clinical course. Among the 1331 crew members with suspected or confirmed Covid-19, 23 (1.7%) were hospitalized, 4 (0.3%) received intensive care, and 1 died. Crew members who worked in confined spaces appeared more likely to become infected. CONCLUSIONS: SARS-CoV-2 spread quickly among the crew of the U.S.S. Theodore Roosevelt. Transmission was facilitated by close-quarters conditions and by asymptomatic and presymptomatic infected crew members. Nearly half of those who tested positive for the virus never had symptoms.

Recommendations for Performance Evaluation of Machine Learning in Pathology: A Concept Paper From the College of American Pathologists.

CONTEXT.­: Machine learning applications in the pathology clinical domain are emerging rapidly. As decision support systems continue to mature, laboratories will increasingly need guidance to evaluate their performance in clinical practice. Currently there are no formal guidelines to assist pathology laboratories in verification and/or validation of such systems. These recommendations are being proposed for the evaluation of machine learning systems in the clinical practice of pathology. OBJECTIVE.­: To propose recommendations for performance evaluation of in vitro diagnostic tests on patient samples that incorporate machine learning as part of the preanalytical, analytical, or postanalytical phases of the laboratory workflow. Topics described include considerations for machine learning model evaluation including risk assessment, predeployment requirements, data sourcing and curation, verification and validation, change control management, human-computer interaction, practitioner training, and competency evaluation. DATA SOURCES.­: An expert panel performed a review of the literature, Clinical and Laboratory Standards Institute guidance, and laboratory and government regulatory frameworks. CONCLUSIONS.­: Review of the literature and existing documents enabled the development of proposed recommendations. This white paper pertains to performance evaluation of machine learning systems intended to be implemented for clinical patient testing. Further studies with real-world clinical data are encouraged to support these proposed recommendations. Performance evaluation of machine learning models is critical to verification and/or validation of in vitro diagnostic tests using machine learning intended for clinical practice.

Independent assessment of a deep learning system for lymph node metastasis detection on the Augmented Reality Microscope.

Several machine learning algorithms have demonstrated high predictive capability in the identification of cancer within digitized pathology slides. The Augmented Reality Microscope (ARM) has allowed these algorithms to be seamlessly integrated within the pathology workflow by overlaying their inferences onto its microscopic field of view in real time. We present an independent assessment of the LYmph Node Assistant (LYNA) models, state-of-the-art algorithms for the identification of breast cancer metastases in lymph node biopsies, optimized for usage on the ARM. We assessed the models on 40 whole slide images at the commonly used objective magnifications of 10×, 20×, and 40×. We analyzed their performance across clinically relevant subclasses of tissue, including breast cancer, lymphocytes, histiocytes, blood, and fat. Each model obtained overall AUC values of approximately 0.98, accuracy values of approximately 0.94, and sensitivity values above 0.88 at classifying small regions of a field of view as benign or cancerous. Across tissue subclasses, the models performed most accurately on fat and blood, and least accurately on histiocytes, germinal centers, and sinus. The models also struggled with the identification of isolated tumor cells, especially at lower magnifications. After testing, we reviewed the discrepancies between model predictions and ground truth to understand the causes of error. We introduce a distinction between proper and improper ground truth for analysis in cases of uncertain annotations. Taken together, these methods comprise a novel approach for exploratory model analysis over complex anatomic pathology data in which precise ground truth is difficult to establish.

Real-time infection prediction with wearable physiological monitoring and AI to aid military workforce readiness during COVID-19.

Infectious threats, like the COVID-19 pandemic, hinder maintenance of a productive and healthy workforce. If subtle physiological changes precede overt illness, then proactive isolation and testing can reduce labor force impacts. This study hypothesized that an early infection warning service based on wearable physiological monitoring and predictive models created with machine learning could be developed and deployed. We developed a prototype tool, first deployed June 23, 2020, that delivered continuously updated scores of infection risk for SARS-CoV-2 through April 8, 2021. Data were acquired from 9381 United States Department of Defense (US DoD) personnel wearing Garmin and Oura devices, totaling 599,174 user-days of service and 201 million hours of data. There were 491 COVID-19 positive cases. A predictive algorithm identified infection before diagnostic testing with an AUC of 0.82. Barriers to implementation included adequate data capture (at least 48% data was needed) and delays in data transmission. We observe increased risk scores as early as 6 days prior to diagnostic testing (2.3 days average). This study showed feasibility of a real-time risk prediction score to minimize workforce impacts of infection.

Deep learning models for histologic grading of breast cancer and association with disease prognosis.

Histologic grading of breast cancer involves review and scoring of three well-established morphologic features: mitotic count, nuclear pleomorphism, and tubule formation. Taken together, these features form the basis of the Nottingham Grading System which is used to inform breast cancer characterization and prognosis. In this study, we develop deep learning models to perform histologic scoring of all three components using digitized hematoxylin and eosin-stained slides containing invasive breast carcinoma. We first evaluate model performance using pathologist-based reference standards for each component. To complement this typical approach to evaluation, we further evaluate the deep learning models via prognostic analyses. The individual component models perform at or above published benchmarks for algorithm-based grading approaches, achieving high concordance rates with pathologist grading. Further, prognostic performance using deep learning-based grading is on par with that of pathologists performing review of matched slides. By providing scores for each component feature, the deep-learning based approach also provides the potential to identify the grading components contributing most to prognostic value. This may enable optimized prognostic models, opportunities to improve access to consistent grading, and approaches to better understand the links between histologic features and clinical outcomes in breast cancer.

Accelerated Progression of Disseminated Coccidioidomycosis Following SARS-CoV-2 Infection: A Case Report.

We describe a patient with subclinical coccidioidomycosis who experienced rapid disease dissemination shortly after SARS-CoV-2 infection, suggesting host immune response dysregulation to coccidioidomycosis by SARS-CoV-2. We hypothesize that disrupted cell-mediated signaling may result after SARS-CoV-2 infection leading to functional exhaustion and CD8+ T-cell senescence with impairment in host cellular response to Coccidioides infection.

Introduction to Artificial Intelligence and Machine Learning for Pathology.

CONTEXT.­: Recent developments in machine learning have stimulated intense interest in software that may augment or replace human experts. Machine learning may impact pathology practice by offering new capabilities in analysis, interpretation, and outcomes prediction using images and other data. The principles of operation and management of machine learning systems are unfamiliar to pathologists, who anticipate a need for additional education to be effective as expert users and managers of the new tools. OBJECTIVE.­: To provide a background on machine learning for practicing pathologists, including an overview of algorithms, model development, and performance evaluation; to examine the current status of machine learning in pathology and consider possible roles and requirements for pathologists in local deployment and management of machine learning systems; and to highlight existing challenges and gaps in deployment methodology and regulation. DATA SOURCES.­: Sources include the biomedical and engineering literature, white papers from professional organizations, government reports, electronic resources, and authors' experience in machine learning. References were chosen when possible for accessibility to practicing pathologists without specialized training in mathematics, statistics, or software development. CONCLUSIONS.­: Machine learning offers an array of techniques that in recent published results show substantial promise. Data suggest that human experts working with machine learning tools outperform humans or machines separately, but the optimal form for this combination in pathology has not been established. Significant questions related to the generalizability of machine learning systems, local site verification, and performance monitoring remain to be resolved before a consensus on best practices and a regulatory environment can be established.

Determining breast cancer biomarker status and associated morphological features using deep learning.

Background: Breast cancer management depends on biomarkers including estrogen receptor, progesterone receptor, and human epidermal growth factor receptor 2 (ER/PR/HER2). Though existing scoring systems are widely used and well-validated, they can involve costly preparation and variable interpretation. Additionally, discordances between histology and expected biomarker findings can prompt repeat testing to address biological, interpretative, or technical reasons for unexpected results. Methods: We developed three independent deep learning systems (DLS) to directly predict ER/PR/HER2 status for both focal tissue regions (patches) and slides using hematoxylin-and-eosin-stained (H&E) images as input. Models were trained and evaluated using pathologist annotated slides from three data sources. Areas under the receiver operator characteristic curve (AUCs) were calculated for test sets at both a patch-level (>135 million patches, 181 slides) and slide-level (n = 3274 slides, 1249 cases, 37 sites). Interpretability analyses were performed using Testing with Concept Activation Vectors (TCAV), saliency analysis, and pathologist review of clustered patches. Results: The patch-level AUCs are 0.939 (95%CI 0.936-0.941), 0.938 (0.936-0.940), and 0.808 (0.802-0.813) for ER/PR/HER2, respectively. At the slide level, AUCs are 0.86 (95%CI 0.84-0.87), 0.75 (0.73-0.77), and 0.60 (0.56-0.64) for ER/PR/HER2, respectively. Interpretability analyses show known biomarker-histomorphology associations including associations of low-grade and lobular histology with ER/PR positivity, and increased inflammatory infiltrates with triple-negative staining. Conclusions: This study presents rapid breast cancer biomarker estimation from routine H&E slides and builds on prior advances by prioritizing interpretability of computationally learned features in the context of existing pathological knowledge.

Development and Validation of a Deep Learning Algorithm for Gleason Grading of Prostate Cancer From Biopsy Specimens.

Importance: For prostate cancer, Gleason grading of the biopsy specimen plays a pivotal role in determining case management. However, Gleason grading is associated with substantial interobserver variability, resulting in a need for decision support tools to improve the reproducibility of Gleason grading in routine clinical practice. Objective: To evaluate the ability of a deep learning system (DLS) to grade diagnostic prostate biopsy specimens. Design, Setting, and Participants: The DLS was evaluated using 752 deidentified digitized images of formalin-fixed paraffin-embedded prostate needle core biopsy specimens obtained from 3 institutions in the United States, including 1 institution not used for DLS development. To obtain the Gleason grade group (GG), each specimen was first reviewed by 2 expert urologic subspecialists from a multi-institutional panel of 6 individuals (years of experience: mean, 25 years; range, 18-34 years). A third subspecialist reviewed discordant cases to arrive at a majority opinion. To reduce diagnostic uncertainty, all subspecialists had access to an immunohistochemical-stained section and 3 histologic sections for every biopsied specimen. Their review was conducted from December 2018 to June 2019. Main Outcomes and Measures: The frequency of the exact agreement of the DLS with the majority opinion of the subspecialists in categorizing each tumor-containing specimen as 1 of 5 categories: nontumor, GG1, GG2, GG3, or GG4-5. For comparison, the rate of agreement of 19 general pathologists' opinions with the subspecialists' majority opinions was also evaluated. Results: For grading tumor-containing biopsy specimens in the validation set (n = 498), the rate of agreement with subspecialists was significantly higher for the DLS (71.7%; 95% CI, 67.9%-75.3%) than for general pathologists (58.0%; 95% CI, 54.5%-61.4%) (P < .001). In subanalyses of biopsy specimens from an external validation set (n = 322), the Gleason grading performance of the DLS remained similar. For distinguishing nontumor from tumor-containing biopsy specimens (n = 752), the rate of agreement with subspecialists was 94.3% (95% CI, 92.4%-95.9%) for the DLS and similar at 94.7% (95% CI, 92.8%-96.3%) for general pathologists (P = .58). Conclusions and Relevance: In this study, the DLS showed higher proficiency than general pathologists at Gleason grading prostate needle core biopsy specimens and generalized to an independent institution. Future research is necessary to evaluate the potential utility of using the DLS as a decision support tool in clinical workflows and to improve the quality of prostate cancer grading for therapy decisions.

Artificial Intelligence-Based Breast Cancer Nodal Metastasis Detection: Insights Into the Black Box for Pathologists.

CONTEXT.­: Nodal metastasis of a primary tumor influences therapy decisions for a variety of cancers. Histologic identification of tumor cells in lymph nodes can be laborious and error-prone, especially for small tumor foci. OBJECTIVE.­: To evaluate the application and clinical implementation of a state-of-the-art deep learning-based artificial intelligence algorithm (LYmph Node Assistant or LYNA) for detection of metastatic breast cancer in sentinel lymph node biopsies. DESIGN.­: Whole slide images were obtained from hematoxylin-eosin-stained lymph nodes from 399 patients (publicly available Camelyon16 challenge dataset). LYNA was developed by using 270 slides and evaluated on the remaining 129 slides. We compared the findings to those obtained from an independent laboratory (108 slides from 20 patients/86 blocks) using a different scanner to measure reproducibility. RESULTS.­: LYNA achieved a slide-level area under the receiver operating characteristic (AUC) of 99% and a tumor-level sensitivity of 91% at 1 false positive per patient on the Camelyon16 evaluation dataset. We also identified 2 "normal" slides that contained micrometastases. When applied to our second dataset, LYNA achieved an AUC of 99.6%. LYNA was not affected by common histology artifacts such as overfixation, poor staining, and air bubbles. CONCLUSIONS.­: Artificial intelligence algorithms can exhaustively evaluate every tissue patch on a slide, achieving higher tumor-level sensitivity than, and comparable slide-level performance to, pathologists. These techniques may improve the pathologist's productivity and reduce the number of false negatives associated with morphologic detection of tumor cells. We provide a framework to aid practicing pathologists in assessing such algorithms for adoption into their workflow (akin to how a pathologist assesses immunohistochemistry results).

Erratum: Publisher Correction: Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer.

[This corrects the article DOI: 10.1038/s41746-019-0112-2.].

Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer.

For prostate cancer patients, the Gleason score is one of the most important prognostic factors, potentially determining treatment independent of the stage. However, Gleason scoring is based on subjective microscopic examination of tumor morphology and suffers from poor reproducibility. Here we present a deep learning system (DLS) for Gleason scoring whole-slide images of prostatectomies. Our system was developed using 112 million pathologist-annotated image patches from 1226 slides, and evaluated on an independent validation dataset of 331 slides. Compared to a reference standard provided by genitourinary pathology experts, the mean accuracy among 29 general pathologists was 0.61 on the validation set. The DLS achieved a significantly higher diagnostic accuracy of 0.70 (p = 0.002) and trended towards better patient risk stratification in correlations to clinical follow-up data. Our approach could improve the accuracy of Gleason scoring and subsequent therapy decisions, particularly where specialist expertise is unavailable. The DLS also goes beyond the current Gleason system to more finely characterize and quantitate tumor morphology, providing opportunities for refinement of the Gleason system itself.