Development and validation of a deep learning-based microsatellite instability predictor from prostate cancer whole-slide images.

Microsatellite instability-high (MSI-H) is a tumor-agnostic biomarker for immune checkpoint inhibitor therapy. However, MSI status is not routinely tested in prostate cancer, in part due to low prevalence and assay cost. As such, prediction of MSI status from hematoxylin and eosin (H&E) stained whole-slide images (WSIs) could identify prostate cancer patients most likely to benefit from confirmatory testing to evaluate their eligibility for immunotherapy and need for Lynch syndrome testing. Prostate biopsies and surgical resections from prostate cancer patients referred to our institution were analyzed. MSI status was determined by next-generation sequencing. Patients sequenced before a cutoff date formed an algorithm development set (n = 4015, MSI-H 1.8%) and a paired validation set (n = 173, MSI-H 19.7%) that consisted of two serial sections from each sample, one stained and scanned internally and the other at an external site. Patients sequenced after the cutoff date formed a temporally independent validation set (n = 1350, MSI-H 2.3%). Attention-based multiple instance learning models were trained to predict MSI-H from H&E WSIs. The predictor achieved area under the receiver operating characteristic curve values of 0.78 (95% CI [0.69-0.86]), 0.72 (95% CI [0.63-0.81]), and 0.72 (95% CI [0.62-0.82]) on the internally prepared, externally prepared, and temporal validation sets, respectively, showing effective predictability and generalization to both external staining/scanning processes and temporally independent samples. While MSI-H status is significantly correlated with Gleason score, the model remained predictive within each Gleason score subgroup.

Prediction of MET Overexpression in Lung Adenocarcinoma from Hematoxylin and Eosin Images.

Mesenchymal epithelial transition (MET) protein overexpression is a targetable event in non-small cell lung cancer and is the subject of active drug development. Challenges in identifying patients for these therapies include lack of access to validated testing, such as standardized immunohistochemistry assessment, and consumption of valuable tissue for a single gene/protein assay. Development of prescreening algorithms using routinely available digitized hematoxylin and eosin (H&E)-stained slides to predict MET overexpression could promote testing for those who will benefit most. Recent literature reports a positive correlation between MET protein overexpression and RNA expression. In this work, a large database of matched H&E slides and RNA expression data were leveraged to train a weakly supervised model to predict MET RNA overexpression directly from H&E images. This model was evaluated on an independent holdout test set of 300 overexpressed and 289 normal patients, demonstrating a receiver operating characteristic area under curve of 0.70 (95th percentile interval: 0.66 to 0.74) with stable performance characteristics across different patient clinical variables and robust to synthetic noise on the test set. These results suggest that H&E-based predictive models could be useful to prioritize patients for confirmatory testing of MET protein or MET gene expression status.

Artificial intelligence-augmented histopathologic review using image analysis to optimize DNA yield from formalin-fixed paraffin-embedded slides.

To achieve minimum DNA input requirements for next-generation sequencing (NGS), pathologists visually estimate macrodissection and slide count decisions. Unfortunately, misestimation may cause tissue waste and increased laboratory costs. We developed an artificial intelligence (AI)-augmented smart pathology review system (SmartPath) to empower pathologists with quantitative metrics for accurately determining tissue extraction parameters. SmartPath uses two deep learning architectures, a U-Net based network for cell segmentation and a multi-field-of-view convolutional network for tumor area segmentation, to extract features from digitized H&E-stained formalin-fixed paraffin-embedded slides. From the segmented tumor area, SmartPath suggests a macrodissection area. To predict DNA yield per slide, the extracted features from within the macrodissection area are correlated with known DNA yields to fit a regularized linear model (R = 0.85). Then, a pathologist-defined target yield divided by the predicted DNA yield per slide gives the number of slides to scrape. Following model development, an internal validation trial was conducted within the Tempus Labs molecular sequencing laboratory. We evaluated our system on 501 clinical colorectal cancer slides, where half received SmartPath-augmented review and half traditional pathologist review. The SmartPath cohort had 25% more DNA yields within a desired target range of 100-2000 ng. The number of extraction attempts was statistically unchanged between cohorts. The SmartPath system recommended fewer slides to scrape for large tissue sections, saving tissue in these cases. Conversely, SmartPath recommended more slides to scrape for samples with scant tissue sections, especially those with degraded DNA, helping prevent costly re-extraction due to insufficient extraction yield. A statistical analysis was performed to measure the impact of covariates on the results, offering insights on how to improve future applications of SmartPath. With these improvements, AI-augmented histopathologic review has the potential to decrease tissue waste, sequencing time, and laboratory costs by optimizing DNA yields, especially for samples with scant tissue and/or degraded DNA.

Integration of tumor extrinsic and intrinsic features associates with immunotherapy response in non-small cell lung cancer.

The efficacy of immune checkpoint blockade (ICB) varies greatly among metastatic non-small cell lung cancer (NSCLC) patients. Loss of heterozygosity at the HLA-I locus (HLA-LOH) has been identified as an important immune escape mechanism. However, despite HLA-I disruptions in their tumor, many patients have durable ICB responses. Here we seek to identify HLA-I-independent features associated with ICB response in NSCLC. We use single-cell profiling to identify tumor-infiltrating, clonally expanded CD4+ T cells that express a canonical cytotoxic gene program and NSCLC cells with elevated HLA-II expression. We postulate cytotoxic CD4+ T cells mediate anti-tumor activity via HLA-II on tumor cells and augment HLA-I-dependent cytotoxic CD8+ T cell interactions to drive ICB response in NSCLC. We show that integrating tumor extrinsic cytotoxic gene expression with tumor mutational burden is associated with longer time to progression in a real-world cohort of 123 NSCLC patients treated with ICB regimens, including those with HLA-LOH.

A pan-cancer organoid platform for precision medicine.

Patient-derived tumor organoids (TOs) are emerging as high-fidelity models to study cancer biology and develop novel precision medicine therapeutics. However, utilizing TOs for systems-biology-based approaches has been limited by a lack of scalable and reproducible methods to develop and profile these models. We describe a robust pan-cancer TO platform with chemically defined media optimized on cultures acquired from over 1,000 patients. Crucially, we demonstrate tumor genetic and transcriptomic concordance utilizing this approach and further optimize defined minimal media for organoid initiation and propagation. Additionally, we demonstrate a neural-network-based high-throughput approach for label-free, light-microscopy-based drug assays capable of predicting patient-specific heterogeneity in drug responses with applicability across solid cancers. The pan-cancer platform, molecular data, and neural-network-based drug assay serve as resources to accelerate the broad implementation of organoid models in precision medicine research and personalized therapeutic profiling programs.

Automatic localization of normal active organs in 3D PET scans.

PET imaging captures the metabolic activity of tissues and is commonly visually interpreted by clinicians for detecting cancer, assessing tumor progression, and evaluating response to treatment. To automate accomplishing these tasks, it is important to distinguish between normal active organs and activity due to abnormal tumor growth. In this paper, we propose a deep learning method to localize and detect normal active organs visible in a 3D PET scan field-of-view. Our method adapts the deep network architecture of YOLO to detect multiple organs in 2D slices and aggregates the results to produce semantically labeled 3D bounding boxes. We evaluate our method on 479 18F-FDG PET scans of 156 patients achieving an average organ detection precision of 75-98%, recall of 94-100%, average bounding box centroid localization error of less than 14 mm, wall localization error of less than 24 mm and a mean IOU of up to 72%.

Adversarial Stain Transfer for Histopathology Image Analysis.

It is generally recognized that color information is central to the automatic and visual analysis of histopathology tissue slides. In practice, pathologists rely on color, which reflects the presence of specific tissue components, to establish a diagnosis. Similarly, automatic histopathology image analysis algorithms rely on color or intensity measures to extract tissue features. With the increasing access to digitized histopathology images, color variation and its implications have become a critical issue. These variations are the result of not only a variety of factors involved in the preparation of tissue slides but also in the digitization process itself. Consequently, different strategies have been proposed to alleviate stain-related tissue inconsistencies in automatic image analysis systems. Such techniques generally rely on collecting color statistics to perform color matching across images. In this work, we propose a different approach for stain normalization that we refer to as stain transfer. We design a discriminative image analysis model equipped with a stain normalization component that transfers stains across datasets. Our model comprises a generative network that learns data set-specific staining properties and image-specific color transformations as well as a task-specific network (e.g., classifier or segmentation network). The model is trained end-to-end using a multi-objective cost function. We evaluate the proposed approach in the context of automatic histopathology image analysis on three data sets and two different analysis tasks: tissue segmentation and classification. The proposed method achieves superior results in terms of accuracy and quality of normalized images compared to various baselines.

A structured latent model for ovarian carcinoma subtyping from histopathology slides.

Accurate subtyping of ovarian carcinomas is an increasingly critical and often challenging diagnostic process. This work focuses on the development of an automatic classification model for ovarian carcinoma subtyping. Specifically, we present a novel clinically inspired contextual model for histopathology image subtyping of ovarian carcinomas. A whole slide image is modelled using a collection of tissue patches extracted at multiple magnifications. An efficient and effective feature learning strategy is used for feature representation of a tissue patch. The locations of salient, discriminative tissue regions are treated as latent variables allowing the model to explicitly ignore portions of the large tissue section that are unimportant for classification. These latent variables are considered in a structured formulation to model the contextual information represented from the multi-magnification analysis of tissues. A novel, structured latent support vector machine formulation is defined and used to combine information from multiple magnifications while simultaneously operating within the latent variable framework. The structural and contextual nature of our method addresses the challenges of intra-class variation and pathologists' workload, which are prevalent in histopathology image classification. Extensive experiments on a dataset of 133 patients demonstrate the efficacy and accuracy of the proposed method against state-of-the-art approaches for histopathology image classification. We achieve an average multi-class classification accuracy of 90%, outperforming existing works while obtaining substantial agreement with six clinicians tested on the same dataset.

Clinically-inspired automatic classification of ovarian carcinoma subtypes.

CONTEXT: It has been shown that ovarian carcinoma subtypes are distinct pathologic entities with differing prognostic and therapeutic implications. Histotyping by pathologists has good reproducibility, but occasional cases are challenging and require immunohistochemistry and subspecialty consultation. Motivated by the need for more accurate and reproducible diagnoses and to facilitate pathologists' workflow, we propose an automatic framework for ovarian carcinoma classification. MATERIALS AND METHODS: Our method is inspired by pathologists' workflow. We analyse imaged tissues at two magnification levels and extract clinically-inspired color, texture, and segmentation-based shape descriptors using image-processing methods. We propose a carefully designed machine learning technique composed of four modules: A dissimilarity matrix, dimensionality reduction, feature selection and a support vector machine classifier to separate the five ovarian carcinoma subtypes using the extracted features. RESULTS: This paper presents the details of our implementation and its validation on a clinically derived dataset of eighty high-resolution histopathology images. The proposed system achieved a multiclass classification accuracy of 95.0% when classifying unseen tissues. Assessment of the classifier's confusion (confusion matrix) between the five different ovarian carcinoma subtypes agrees with clinician's confusion and reflects the difficulty in diagnosing endometrioid and serous carcinomas. CONCLUSIONS: Our results from this first study highlight the difficulty of ovarian carcinoma diagnosis which originate from the intrinsic class-imbalance observed among subtypes and suggest that the automatic analysis of ovarian carcinoma subtypes could be valuable to clinician's diagnostic procedure by providing a second opinion.