Masked pre-training of transformers for histology image analysis.

In digital pathology, whole-slide images (WSIs) are widely used for applications such as cancer diagnosis and prognosis prediction. Vision transformer (ViT) models have recently emerged as a promising method for encoding large regions of WSIs while preserving spatial relationships among patches. However, due to the large number of model parameters and limited labeled data, applying transformer models to WSIs remains challenging. In this study, we propose a pretext task to train the transformer model in a self-supervised manner. Our model, MaskHIT, uses the transformer output to reconstruct masked patches, measured by contrastive loss. We pre-trained MaskHIT model using over 7000 WSIs from TCGA and extensively evaluated its performance in multiple experiments, covering survival prediction, cancer subtype classification, and grade prediction tasks. Our experiments demonstrate that the pre-training procedure enables context-aware understanding of WSIs, facilitates the learning of representative histological features based on patch positions and visual patterns, and is essential for the ViT model to achieve optimal results on WSI-level tasks. The pre-trained MaskHIT surpasses various multiple instance learning approaches by 3% and 2% on survival prediction and cancer subtype classification tasks, and also outperforms recent state-of-the-art transformer-based methods. Finally, a comparison between the attention maps generated by the MaskHIT model with pathologist's annotations indicates that the model can accurately identify clinically relevant histological structures on the whole slide for each task.

Oligometastatic Melanoma Treated by Metastasectomy in Combination with Immune Checkpoint and BRAF Inhibitors: A Case Series.

BACKGROUND Early therapies for metastatic melanoma improved patient quality of life; however, median survival remained unaffected. Studies are showing that surgical excision with the combination of immune checkpoint inhibitor (ICI) therapy has better outcomes than systemic therapy alone. This single-center case series describes 7 patients with oligometastatic melanoma treated by metastasectomy in combination with ICI and BRAF inhibitors. CASE REPORT One female and 6 male patients are included in our study, with ages ranging from 34 to 82 years. Oligometastatic melanoma is defined was having no more than 5 metastatic regions. Each patient had an Eastern Cooperative Oncology Group (ECOG) performance status of 0 or 1. Patients received either ICI therapy with ipilimumab, nivolumab, and/or pembrolizumab, or targeted therapy with encorafenib and binimetinib, or a combination. Patients underwent metastasectomies with curative intent. The main outcome and measurements obtained were the duration of disease-free survival, based on radiographic evidence. The range of disease-free survival in our population was 13 to 67 months, with the lower end limited by patient death and the upper limit being the present day. CONCLUSIONS This case series reiterates survival benefit for patients who received metastasectomy after exhibiting good response to ICI therapy. ICI and/or BRAF inhibitor therapy combined with metastasectomy provides a possible curative option for patients who may have previously been relegated to palliative-focused care. By using a multimodal approach with oncologists and surgeons, we can challenge our understanding of what constitutes a resectable cancer.

Examining longitudinal markers of bladder cancer recurrence through a semiautonomous machine learning system for quantifying specimen atypia from urine cytology.

BACKGROUND: Urine cytology is generally considered the primary approach for screening for recurrence of bladder cancer. However, it is currently unclear how best to use cytological examinations for assessment and early detection of recurrence, beyond identifying a positive finding that requires more invasive methods to confirm recurrence and decide on therapeutic options. Because screening programs are frequent, and can be burdensome, finding quantitative means to reduce this burden for patients, cytopathologists, and urologists is an important endeavor and can improve both the efficiency and reliability of findings. Additionally, identifying ways to risk-stratify patients is crucial for improving quality of life while reducing the risk of future recurrence or progression of the cancer. METHODS: In this study, a computational machine learning tool, AutoParis-X, was leveraged to extract imaging features from urine cytology examinations longitudinally to study the predictive potential of urine cytology for assessing recurrence risk. This study examined how the significance of imaging predictors changes over time before and after surgery to determine which predictors and time periods are most relevant for assessing recurrence risk. RESULTS: Results indicate that imaging predictors extracted using AutoParis-X can predict recurrence as well or better than traditional cytological/histological assessments alone and that the predictiveness of these features is variable across time, with key differences in overall specimen atypia identified immediately before tumor recurrence. CONCLUSIONS: Further research will clarify how computational methods can be effectively used in high-volume screening programs to improve recurrence detection and complement traditional modes of assessment.

Large-scale validation study of an improved semiautonomous urine cytology assessment tool: AutoParis-X.

BACKGROUND: Adopting a computational approach for the assessment of urine cytology specimens has the potential to improve the efficiency, accuracy, and reliability of bladder cancer screening, which has heretofore relied on semisubjective manual assessment methods. As rigorous, quantitative criteria and guidelines have been introduced for improving screening practices (e.g., The Paris System for Reporting Urinary Cytology), algorithms to emulate semiautonomous diagnostic decision-making have lagged behind, in part because of the complex and nuanced nature of urine cytology reporting. METHODS: In this study, the authors report on the development and large-scale validation of a deep-learning tool, AutoParis-X, which can facilitate rapid, semiautonomous examination of urine cytology specimens. RESULTS: The results of this large-scale, retrospective validation study indicate that AutoParis-X can accurately determine urothelial cell atypia and aggregate a wide variety of cell-related and cluster-related information across a slide to yield an atypia burden score, which correlates closely with overall specimen atypia and is predictive of Paris system diagnostic categories. Importantly, this approach accounts for challenges associated with the assessment of overlapping cell cluster borders, which improve the ability to predict specimen atypia and accurately estimate the nuclear-to-cytoplasm ratio for cells in these clusters. CONCLUSIONS: The authors developed a publicly available, open-source, interactive web application that features a simple, easy-to-use display for examining urine cytology whole-slide images and determining the level of atypia in specific cells, flagging the most abnormal cells for pathologist review. The accuracy of AutoParis-X (and other semiautomated digital pathology systems) indicates that these technologies are approaching clinical readiness and necessitates full evaluation of these algorithms in head-to-head clinical trials.

Pancreatic acinar metaplasia at the gastroesophageal junction is associated with protective effect against intestinal metaplasia in patients with gastroesophageal reflux disease.

Anecdotal evidence suggests that pancreatic acinar metaplasia (PAM) and intestinal metaplasia (IM) overlap infrequently at the gastroesophageal junction/distal esophagus (GEJ/DE). The goal of this study was to evaluate the significance of PAM at GEJ/DE in relation to IM in patients with gastroesophageal reflux disease (GERD). Group 1 comprised 230 consecutive patients with GEJ/DE biopsies (80.6% with GERD symptoms). Group 2 comprised 151 patients with established GERD and GEJ/DE biopsies taken before Nissen fundoplication. Group 3 comprised 540 consecutive patients used for a follow-up study of PAM. PAM was present in 15.7%-15.9% and IM in 24.8%-31.1% of patients in groups 1 and 2, respectively. PAM-IM overlap was present in 2.2%-3.3%, respectively. Patients with PAM were, on average, 6-12 years younger than patients with IM, and were predominantly female (72.2%-75%), in contrast to patients with IM (47.3%-32%). In the unadjusted logistic regression model, patients with PAM were 69%-65% less likely to also have IM, as compared to patients without PAM. In the fully adjusted model, patients with PAM were 35%-61% less likely to also have IM, although the P-value was not significant. Follow-up analysis of patients with PAM from group 3 (n = 28) demonstrated the prevalence of IM and PAM in subsequent biopsies at 7.1% and 60.7%, respectively. No cases showed PAM-IM overlap on follow-up. The data suggests that PAM at the GEJ/DE is associated with protective effect against IM and thus could be useful as a marker of decreased susceptibility to IM.

MHAttnSurv: Multi-head attention for survival prediction using whole-slide pathology images.

Whole slide images (WSI) based survival prediction has attracted increasing interest in pathology. Despite this, extracting prognostic information from WSIs remains a challenging task due to their enormous size and the scarcity of pathologist annotations. Previous studies have utilized multiple instance learning approach to combine information from several randomly sampled patches, but this approach may not be adequate as different visual patterns may contribute unequally to prognosis prediction. In this study, we introduce a multi-head attention mechanism that allows each attention head to independently explore the utility of various visual patterns on a tumor slide, thereby enabling more comprehensive information extraction from WSIs. We evaluated our approach on four cancer types from The Cancer Genome Atlas database. Our model achieved an average c-index of 0.640, outperforming three existing state-of-the-art approaches for WSI-based survival prediction on these datasets. Visualization of attention maps reveals that the attention heads synergistically focus on different morphological patterns, providing additional evidence for the effectiveness of multi-head attention in survival prediction.

Graph Convolutional Neural Networks for Histologic Classification of Pancreatic Cancer.

CONTEXT.­: Pancreatic ductal adenocarcinoma has some of the worst prognostic outcomes among various cancer types. Detection of histologic patterns of pancreatic tumors is essential to predict prognosis and decide the treatment for patients. This histologic classification can have a large degree of variability even among expert pathologists. OBJECTIVE.­: To detect aggressive adenocarcinoma and less aggressive pancreatic tumors from nonneoplasm cases using a graph convolutional network-based deep learning model. DESIGN.­: Our model uses a convolutional neural network to extract detailed information from every small region in a whole slide image. Then, we use a graph architecture to aggregate the extracted features from these regions and their positional information to capture the whole slide-level structure and make the final prediction. RESULTS.­: We evaluated our model on an independent test set and achieved an F1 score of 0.85 for detecting neoplastic cells and ductal adenocarcinoma, significantly outperforming other baseline methods. CONCLUSIONS.­: If validated in prospective studies, this approach has a great potential to assist pathologists in identifying adenocarcinoma and other types of pancreatic tumors in clinical settings.

Detection of Colorectal Adenocarcinoma and Grading Dysplasia on Histopathologic Slides Using Deep Learning.

Colorectal cancer (CRC) is one of the most common types of cancer among men and women. The grading of dysplasia and the detection of adenocarcinoma are important clinical tasks in the diagnosis of CRC and shape the patients' follow-up plans. This study evaluated the feasibility of deep learning models for the classification of colorectal lesions into four classes: benign, low-grade dysplasia, high-grade dysplasia, and adenocarcinoma. To this end, a deep neural network was developed on a training set of 655 whole slide images of digitized colorectal resection slides from a tertiary medical institution; and the network was evaluated on an internal test set of 234 slides, as well as on an external test set of 606 adenocarcinoma slides from The Cancer Genome Atlas database. The model achieved an overall accuracy, sensitivity, and specificity of 95.5%, 91.0%, and 97.1%, respectively, on the internal test set, and an accuracy and sensitivity of 98.5% for adenocarcinoma detection task on the external test set. Results suggest that such deep learning models can potentially assist pathologists in grading colorectal dysplasia, detecting adenocarcinoma, prescreening, and prioritizing the reviewing of suspicious cases to improve the turnaround time for patients with a high risk of CRC. Furthermore, the high sensitivity on the external test set suggests the model's generalizability in detecting colorectal adenocarcinoma on whole slide images across different institutions.

Uncovering additional predictors of urothelial carcinoma from voided urothelial cell clusters through a deep learning-based image preprocessing technique.

BACKGROUND: Urine cytology is commonly used as a screening test for high-grade urothelial carcinoma for patients with risk factors or hematuria and is an essential step in longitudinal monitoring of patients with previous bladder cancer history. However, the semisubjective nature of current reporting systems for urine cytology (e.g., The Paris System) can hamper reproducibility. For instance, the incorporation of urothelial cell clusters into the classification schema is still an item of debate and perplexity among expert cytopathologists because several previous works have disputed their diagnostic relevance. METHODS: In this work, an automated preprocessing tool for urothelial cell cluster assessment was developed that divides urothelial cell clusters into meaningful components for downstream assessment (ie, population-based studies, workflow automation). RESULTS: In this work, an automated preprocessing tool for urothelial cell cluster assessment was developed that divides urothelial cell clusters into meaningful components for downstream assessment (ie, population-based studies, workflow automation). Results indicate that cell cluster atypia (i.e., defined by whether the cell cluster harbored multiple atypical cells, thresholded by a minimum number of cells), cell border overlap and smoothness, and total number of clusters are important markers of specimen atypia when considering assessment of urothelial cell clusters. CONCLUSIONS: Markers established through techniques to separate cell clusters may have wider applicability for the design and implementation of machine learning approaches for urine cytology assessment.

Predicting oncogene mutations of lung cancer using deep learning and histopathologic features on whole-slide images.

Lung cancer is a leading cause of death in both men and women globally. The recent development of tumor molecular profiling has opened opportunities for targeted therapies for lung adenocarcinoma (LUAD) patients. However, the lack of access to molecular profiling or cost and turnaround time associated with it could hinder oncologists' willingness to order frequent molecular tests, limiting potential benefits from precision medicine. In this study, we developed a weakly supervised deep learning model for predicting somatic mutations of LUAD patients based on formalin-fixed paraffin-embedded (FFPE) whole-slide images (WSIs) using LUAD subtypes-related histological features and recent advances in computer vision. Our study was performed on a total of 747 hematoxylin and eosin (H&E) stained FFPE LUAD WSIs and the genetic mutation data of 232 patients who were treated at Dartmouth-Hitchcock Medical Center (DHMC). We developed our convolutional neural network-based models to analyze whole slides and predict five major genetic mutations, i.e., BRAF, EGFR, KRAS, STK11, and TP53. We additionally used 111 cases from the LUAD dataset of the CPTAC-3 study for external validation. Our model achieved an AUROC of 0.799 (95% CI: 0.686-0.904) and 0.686 (95% CI: 0.620-0.752) for predicting EGFR genetic mutations on the DHMC and CPTAC-3 test sets, respectively. Predicting TP53 genetic mutations also showed promising outcomes. Our results demonstrated that H&E stained FFPE LUAD whole slides could be utilized to predict oncogene mutations, such as EGFR, indicating that somatic mutations could present subtle morphological characteristics in histology slides, where deep learning-based feature extractors can learn such latent information.

Incidence of Pancreatic Intraepithelial Neoplasia in an Autopsy Series.

OBJECTIVES: Pancreatic intraepithelial neoplasia (PanIN) is the currently preferred designation for putative preneoplastic changes in the pancreas. There are few data for the incidence of PanIN in the general population. Our goal was to determine the incidence of PanIN in a large group of pancreases obtained at autopsy. METHODS: Slides stained with hematoxylin and eosin were scanned to count PanIN. RESULTS: We found multiple PanINs in most pancreases and at least 1 in 86.4% of 154 pancreases when multiple slides (8-12) from each were examined. The average age at autopsy was 62 years, and 90% of the patients were 40 years or older. Several questions were raised by our observations. Should a minimum size be defined for classification as PanIN? Do PanINs occur in lesions that apparently arise from acinar to ductal metaplasia? Does squamous metaplasia in PanIN have any special significance, and do purely squamous lesions have preneoplastic significance? CONCLUSIONS: We conclude that the incidence of PanIN is higher than previously reported.

The Host-Microbiome Response to Hyperbaric Oxygen Therapy in Ulcerative Colitis Patients.

BACKGROUND & AIMS: Hyperbaric oxygen therapy (HBOT) is a promising treatment for moderate-to-severe ulcerative colitis. However, our current understanding of the host and microbial response to HBOT remains unclear. This study examined the molecular mechanisms underpinning HBOT using a multi-omic strategy. METHODS: Pre- and post-intervention mucosal biopsies, tissue, and fecal samples were collected from HBOT phase 2 clinical trials. Biopsies and fecal samples were subjected to shotgun metaproteomics, metabolomics, 16s rRNA sequencing, and metagenomics. Tissue was subjected to bulk RNA sequencing and digital spatial profiling (DSP) for single-cell RNA and protein analysis, and immunohistochemistry was performed. Fecal samples were also used for colonization experiments in IL10-/- germ-free UC mouse models. RESULTS: Proteomics identified negative associations between HBOT response and neutrophil azurophilic granule abundance. DSP identified an HBOT-specific reduction of neutrophil STAT3, which was confirmed by immunohistochemistry. HBOT decreased microbial diversity with a proportional increase in Firmicutes and a secondary bile acid lithocholic acid. A major source of the reduction in diversity was the loss of mucus-adherent taxa, resulting in increased MUC2 levels post-HBOT. Targeted database searching revealed strain-level associations between Akkermansia muciniphila and HBOT response status. Colonization of IL10-/- with stool obtained from HBOT responders resulted in lower colitis activity compared with non-responders, with no differences in STAT3 expression, suggesting complementary but independent host and microbial responses. CONCLUSIONS: HBOT reduces host neutrophil STAT3 and azurophilic granule activity in UC patients and changes in microbial composition and metabolism in ways that improve colitis activity. Intestinal microbiota, especially strain level variations in A muciniphila, may contribute to HBOT non-response.

Evaluation of an Artificial Intelligence-Augmented Digital System for Histologic Classification of Colorectal Polyps.

Importance: Colorectal polyps are common, and their histopathologic classification is used in the planning of follow-up surveillance. Substantial variation has been observed in pathologists' classification of colorectal polyps, and improved assessment by pathologists may be associated with reduced subsequent underuse and overuse of colonoscopy. Objective: To compare standard microscopic assessment with an artificial intelligence (AI)-augmented digital system that annotates regions of interest within digitized polyp tissue and predicts polyp type using a deep learning model to assist pathologists in colorectal polyp classification. Design, Setting, and Participants: In this diagnostic study conducted at a tertiary academic medical center and a community hospital in New Hampshire, 100 slides with colorectal polyp samples were read by 15 pathologists using a microscope and an AI-augmented digital system, with a washout period of at least 12 weeks between use of each modality. The study was conducted from February 10 to July 10, 2020. Main Outcomes and Measures: Accuracy and time of evaluation were used to compare pathologists' performance when a microscope was used with their performance when the AI-augmented digital system was used. Outcomes were compared using paired t tests and mixed-effects models. Results: In assessments of 100 slides with colorectal polyp specimens, use of the AI-augmented digital system significantly improved pathologists' classification accuracy compared with microscopic assessment from 73.9% (95% CI, 71.7%-76.2%) to 80.8% (95% CI, 78.8%-82.8%) (P < .001). The overall difference in the evaluation time per slide between the digital system (mean, 21.7 seconds; 95% CI, 20.8-22.7 seconds) and microscopic examination (mean, 13.0 seconds; 95% CI, 12.4-13.5 seconds) was -8.8 seconds (95% CI, -9.8 to -7.7 seconds), but this difference decreased as pathologists became more familiar and experienced with the digital system; the difference between the time of evaluation on the last set of 20 slides for all pathologists when using the microscope and the digital system was 4.8 seconds (95% CI, 3.0-6.5 seconds). Conclusions and Relevance: In this diagnostic study, an AI-augmented digital system significantly improved the accuracy of pathologic interpretation of colorectal polyps compared with microscopic assessment. If applied broadly to clinical practice, this tool may be associated with decreases in subsequent overuse and underuse of colonoscopy and thus with improved patient outcomes and reduced health care costs.

Resolution-based distillation for efficient histology image classification.

Developing deep learning models to analyze histology images has been computationally challenging, as the massive size of the images causes excessive strain on all parts of the computing pipeline. This paper proposes a novel deep learning-based methodology for improving the computational efficiency of histology image classification. The proposed approach is robust when used with images that have reduced input resolution, and it can be trained effectively with limited labeled data. Moreover, our approach operates at either the tissue- or slide-level, removing the need for laborious patch-level labeling. Our method uses knowledge distillation to transfer knowledge from a teacher model pre-trained at high resolution to a student model trained on the same images at a considerably lower resolution. Also, to address the lack of large-scale labeled histology image datasets, we perform the knowledge distillation in a self-supervised fashion. We evaluate our approach on three distinct histology image datasets associated with celiac disease, lung adenocarcinoma, and renal cell carcinoma. Our results on these datasets demonstrate that a combination of knowledge distillation and self-supervision allows the student model to approach and, in some cases, surpass the teacher model's classification accuracy while being much more computationally efficient. Additionally, we observe an increase in student classification performance as the size of the unlabeled dataset increases, indicating that there is potential for this method to scale further with additional unlabeled data. Our model outperforms the high-resolution teacher model for celiac disease in accuracy, F1-score, precision, and recall while requiring 4 times fewer computations. For lung adenocarcinoma, our results at 1.25× magnification are within 1.5% of the results for the teacher model at 10× magnification, with a reduction in computational cost by a factor of 64. Our model on renal cell carcinoma at 1.25× magnification performs within 1% of the teacher model at 5× magnification while requiring 16 times fewer computations. Furthermore, our celiac disease outcomes benefit from additional performance scaling with the use of more unlabeled data. In the case of 0.625× magnification, using unlabeled data improves accuracy by 4% over the tissue-level baseline. Therefore, our approach can improve the feasibility of deep learning solutions for digital pathology on standard computational hardware and infrastructures.

A large-scale internal validation study of unsupervised virtual trichrome staining technologies on nonalcoholic steatohepatitis liver biopsies.

Non-alcoholic steatohepatitis (NASH) is a fatty liver disease characterized by accumulation of fat in hepatocytes with concurrent inflammation and is associated with morbidity, cirrhosis and liver failure. After extraction of a liver core biopsy, tissue sections are stained with hematoxylin and eosin (H&E) to grade NASH activity, and stained with trichrome to stage fibrosis. Methods to computationally transform one stain into another on digital whole slide images (WSI) can lessen the need for additional physical staining besides H&E, reducing personnel, equipment, and time costs. Generative adversarial networks (GAN) have shown promise for virtual staining of tissue. We conducted a large-scale validation study of the viability of GANs for H&E to trichrome conversion on WSI (n = 574). Pathologists were largely unable to distinguish real images from virtual/synthetic images given a set of twelve Turing Tests. We report high correlation between staging of real and virtual stains ([Formula: see text]; 95% CI: 0.84-0.88). Stages assigned to both virtual and real stains correlated similarly with a number of clinical biomarkers and progression to End Stage Liver Disease (Hazard Ratio HR = 2.06, 95% CI: 1.36-3.12, p < 0.001 for real stains; HR = 2.02, 95% CI: 1.40-2.92, p < 0.001 for virtual stains). Our results demonstrate that virtual trichrome technologies may offer a software solution that can be employed in the clinical setting as a diagnostic decision aid.

CD8 T-Cell-Predominant Lymphocytic Esophagitis is One of the Major Patterns of Lymphocytic Inflammation in Gastroesophageal Reflux Disease.

CONTEXT.­: Published reports have suggested an association of lymphocytic esophagitis (LyE) with gastroesophageal reflux disease (GERD) and primary motility disorders and have also shown that GERD and motility disorders frequently overlap. These findings make it difficult to determine the true relationship between LyE and GERD, which may be confounded by the presence of motility disorders with LyE. OBJECTIVE.­: To characterize patterns of lymphocytic inflammation in patients with GERD who have no motility abnormalities. DESIGN.­: We identified 161 patients seen at our institution from 1998 to 2014 who were diagnosed with GERD, had normal esophageal motility, and available esophageal biopsies. LyE was defined as peripapillary lymphocytosis with rare or absent granulocytes. CD4 and CD8 immunophenotype of lymphocytes was evaluated using immunohistochemistry. RESULTS.­: We found increased intraepithelial lymphocytes in 13.7% of patients with GERD. Two major patterns and 1 minor pattern of lymphocytic inflammation were observed as follows: (1) LyE (in 6.8% [11 of 161] of patients and typically focal), (2) dispersed lymphocytes in an area of reflux esophagitis (in 5.6% [9 of 161] and typically diffuse), and (3) peripapillary lymphocytes in an area of reflux esophagitis (in 1.2% [2 of 161]). CD8 T cells significantly outnumbered CD4 T cells in 91% of patients with lymphocytic esophagitis and 100% of patients with dispersed lymphocytes (9 of 9) or peripapillary lymphocytes (2 of 2) in the area of reflux esophagitis. CONCLUSIONS.­: These findings suggest that LyE is one of the major patterns of lymphocytic inflammation in GERD. CD8 T-cell-predominant immunophenotype may be useful as a marker of GERD in the differential diagnosis of LyE.

Predicting colorectal polyp recurrence using time-to-event analysis of medical records.

Identifying patient characteristics that influence the rate of colorectal polyp recurrence can provide important insights into which patients are at higher risk for recurrence. We used natural language processing to extract polyp morphological characteristics from 953 polyp-presenting patients' electronic medical records. We used subsequent colonoscopy reports to examine how the time to polyp recurrence (731 patients experienced recurrence) is influenced by these characteristics as well as anthropometric features using Kaplan-Meier curves, Cox proportional hazards modeling, and random survival forest models. We found that the rate of recurrence differed significantly by polyp size, number, and location and patient smoking status. Additionally, right-sided colon polyps increased recurrence risk by 30% compared to left-sided polyps. History of tobacco use increased polyp recurrence risk by 20% compared to never-users. A random survival forest model showed an AUC of 0.65 and identified several other predictive variables, which can inform development of personalized polyp surveillance plans.

Evaluation of a Deep Neural Network for Automated Classification of Colorectal Polyps on Histopathologic Slides.

Importance: Histologic classification of colorectal polyps plays a critical role in screening for colorectal cancer and care of affected patients. An accurate and automated algorithm for the classification of colorectal polyps on digitized histopathologic slides could benefit practitioners and patients. Objective: To evaluate the performance and generalizability of a deep neural network for colorectal polyp classification on histopathologic slide images using a multi-institutional data set. Design, Setting, and Participants: This prognostic study used histopathologic slides collected from January 1, 2016, to June 31, 2016, from Dartmouth-Hitchcock Medical Center, Lebanon, New Hampshire, with 326 slides used for training, 157 slides for an internal data set, and 25 for a validation set. For the external data set, 238 slides for 179 distinct patients were obtained from 24 institutions across 13 US states. Data analysis was performed from April 9 to November 23, 2019. Main Outcomes and Measures: Accuracy, sensitivity, and specificity of the model to classify 4 major colorectal polyp types: tubular adenoma, tubulovillous or villous adenoma, hyperplastic polyp, and sessile serrated adenoma. Performance was compared with that of local pathologists' at the point of care identified from corresponding pathology laboratories. Results: For the internal evaluation on the 157 slides with ground truth labels from 5 pathologists, the deep neural network had a mean accuracy of 93.5% (95% CI, 89.6%-97.4%) compared with local pathologists' accuracy of 91.4% (95% CI, 87.0%-95.8%). On the external test set of 238 slides with ground truth labels from 5 pathologists, the deep neural network achieved an accuracy of 87.0% (95% CI, 82.7%-91.3%), which was comparable with local pathologists' accuracy of 86.6% (95% CI, 82.3%-90.9%). Conclusions and Relevance: The findings suggest that this model may assist pathologists by improving the diagnostic efficiency, reproducibility, and accuracy of colorectal cancer screenings.

Attention-Based Deep Neural Networks for Detection of Cancerous and Precancerous Esophagus Tissue on Histopathological Slides.

Importance: Deep learning-based methods, such as the sliding window approach for cropped-image classification and heuristic aggregation for whole-slide inference, for analyzing histological patterns in high-resolution microscopy images have shown promising results. These approaches, however, require a laborious annotation process and are fragmented. Objective: To evaluate a novel deep learning method that uses tissue-level annotations for high-resolution histological image analysis for Barrett esophagus (BE) and esophageal adenocarcinoma detection. Design, Setting, and Participants: This diagnostic study collected deidentified high-resolution histological images (N = 379) for training a new model composed of a convolutional neural network and a grid-based attention network. Histological images of patients who underwent endoscopic esophagus and gastroesophageal junction mucosal biopsy between January 1, 2016, and December 31, 2018, at Dartmouth-Hitchcock Medical Center (Lebanon, New Hampshire) were collected. Main Outcomes and Measures: The model was evaluated on an independent testing set of 123 histological images with 4 classes: normal, BE-no-dysplasia, BE-with-dysplasia, and adenocarcinoma. Performance of this model was measured and compared with that of the current state-of-the-art sliding window approach using the following standard machine learning metrics: accuracy, recall, precision, and F1 score. Results: Of the independent testing set of 123 histological images, 30 (24.4%) were in the BE-no-dysplasia class, 14 (11.4%) in the BE-with-dysplasia class, 21 (17.1%) in the adenocarcinoma class, and 58 (47.2%) in the normal class. Classification accuracies of the proposed model were 0.85 (95% CI, 0.81-0.90) for the BE-no-dysplasia class, 0.89 (95% CI, 0.84-0.92) for the BE-with-dysplasia class, and 0.88 (95% CI, 0.84-0.92) for the adenocarcinoma class. The proposed model achieved a mean accuracy of 0.83 (95% CI, 0.80-0.86) and marginally outperformed the sliding window approach on the same testing set. The F1 scores of the attention-based model were at least 8% higher for each class compared with the sliding window approach: 0.68 (95% CI, 0.61-0.75) vs 0.61 (95% CI, 0.53-0.68) for the normal class, 0.72 (95% CI, 0.63-0.80) vs 0.58 (95% CI, 0.45-0.69) for the BE-no-dysplasia class, 0.30 (95% CI, 0.11-0.48) vs 0.22 (95% CI, 0.11-0.33) for the BE-with-dysplasia class, and 0.67 (95% CI, 0.54-0.77) vs 0.58 (95% CI, 0.44-0.70) for the adenocarcinoma class. However, this outperformance was not statistically significant. Conclusions and Relevance: Results of this study suggest that the proposed attention-based deep neural network framework for BE and esophageal adenocarcinoma detection is important because it is based solely on tissue-level annotations, unlike existing methods that are based on regions of interest. This new model is expected to open avenues for applying deep learning to digital pathology.

Automated Detection of Celiac Disease on Duodenal Biopsy Slides: A Deep Learning Approach.

CONTEXT: Celiac disease (CD) prevalence and diagnosis have increased substantially in recent years. The current gold standard for CD confirmation is visual examination of duodenal mucosal biopsies. An accurate computer-aided biopsy analysis system using deep learning can help pathologists diagnose CD more efficiently. SUBJECTS AND METHODS: In this study, we trained a deep learning model to detect CD on duodenal biopsy images. Our model uses a state-of-the-art residual convolutional neural network to evaluate patches of duodenal tissue and then aggregates those predictions for whole-slide classification. We tested the model on an independent set of 212 images and evaluated its classification results against reference standards established by pathologists. RESULTS: Our model identified CD, normal tissue, and nonspecific duodenitis with accuracies of 95.3%, 91.0%, and 89.2%, respectively. The area under the receiver operating characteristic curve was >0.95 for all classes. CONCLUSIONS: We have developed an automated biopsy analysis system that achieves high performance in detecting CD on biopsy slides. Our system can highlight areas of interest and provide preliminary classification of duodenal biopsies before review by pathologists. This technology has great potential for improving the accuracy and efficiency of CD diagnosis.