Cervical cytology screening facilitated by an artificial intelligence microscope: A preliminary study.

BACKGROUND: Cervical cytology screening is usually laborious with a heavy workload and poor diagnostic consistency. The authors have developed an artificial intelligence (AI) microscope that can provide onsite diagnostic assistance for cervical cytology screening in real time. METHODS: A total of 2167 cervical cytology slides were selected from a cohort of 10,601 cases from Shenzhen Maternity and Child Healthcare Hospital, and the training data set consisted of 42,073 abnormal cervical epithelial cells. The recognition results of an AI technique were presented in a microscope eyepiece by an augmented reality technique. Potentially abnormal cells were highlighted with binary classification results in a 10× field of view (FOV) and with multiclassification results according to the Bethesda system in 20× and 40× FOVs. In addition, 486 slides were selected for the reader study to evaluate the performance of the AI microscope. RESULTS: In the reader study, which compared manual reading with AI assistance, the sensitivities for the detection of low-grade squamous intraepithelial lesions and high-grade squamous intraepithelial lesions were significantly improved from 0.837 to 0.923 (P < .001) and from 0.830 to 0.917 (P < .01), respectively; the κ score for atypical squamous cells of undetermined significance (ASCUS) was improved from 0.581 to 0.637; the averaged pairwise κ of consistency for multiclassification was improved from 0.649 to 0.706; the averaged pairwise κ of consistency for binary classification was improved from 0.720 to 0.798; and the averaged pairwise κ of ASCUS was improved from 0.557 to 0.639. CONCLUSIONS: The results of this study show that an AI microscope can provide real-time assistance for cervical cytology screening and improve the efficiency and accuracy of cervical cytology diagnosis.

Convolutional neural networks can accurately distinguish four histologic growth patterns of lung adenocarcinoma in digital slides.

During the diagnostic workup of lung adenocarcinomas (LAC), pathologists evaluate distinct histological tumor growth patterns. The percentage of each pattern on multiple slides bears prognostic significance. To assist with the quantification of growth patterns, we constructed a pipeline equipped with a convolutional neural network (CNN) and soft-voting as the decision function to recognize solid, micropapillary, acinar, and cribriform growth patterns, and non-tumor areas. Slides of primary LAC were obtained from Cedars-Sinai Medical Center (CSMC), the Military Institute of Medicine in Warsaw and the TCGA portal. Several CNN models trained with 19,924 image tiles extracted from 78 slides (MIMW and CSMC) were evaluated on 128 test slides from the three sites by F1-score and accuracy using manual tumor annotations by pathologist. The best CNN yielded F1-scores of 0.91 (solid), 0.76 (micropapillary), 0.74 (acinar), 0.6 (cribriform), and 0.96 (non-tumor) respectively. The overall accuracy of distinguishing the five tissue classes was 89.24%. Slide-based accuracy in the CSMC set (88.5%) was significantly better (p < 2.3E-4) than the accuracy in the MIMW (84.2%) and TCGA (84%) sets due to superior slide quality. Our model can work side-by-side with a pathologist to accurately quantify the percentages of growth patterns in tumors with mixed LAC patterns.

Spatial Mapping of Myeloid Cells and Macrophages by Multiplexed Tissue Staining.

An array of phenotypically diverse myeloid cells and macrophages (MC&M) resides in the tumor microenvironment, requiring multiplexed detection systems for visualization. Here we report an automated, multiplexed staining approach, named PLEXODY, that consists of five MC&M-related fluorescently-tagged antibodies (anti - CD68, - CD163, - CD206, - CD11b, and - CD11c), and three chromogenic antibodies, reactive with high- and low-molecular weight cytokeratins and CD3, highlighting tumor regions, benign glands and T cells. The staining prototype and image analysis methods which include a pixel/area-based quantification were developed using tissues from inflamed colon and tonsil and revealed a unique tissue-specific composition of 14 MC&M-associated pixel classes. As a proof-of-principle, PLEXODY was applied to three cases of pancreatic, prostate and renal cancers. Across digital images from these cancer types we observed 10 MC&M-associated pixel classes at frequencies greater than 3%. Cases revealed higher frequencies of single positive compared to multi-color pixels and a high abundance of CD68+/CD163+ and CD68+/CD163+/CD206+ pixels. Significantly more CD68+ and CD163+ vs. CD11b+ and CD11c+ pixels were in direct contact with tumor cells and T cells. While the greatest percentage (~70%) of CD68+ and CD163+ pixels was 0-20 microns away from tumor and T cell borders, CD11b+ and CD11c+ pixels were detected up to 240 microns away from tumor/T cell masks. Together, these data demonstrate significant differences in densities and spatial organization of MC&M-associated pixel classes, but surprising similarities between the three cancer types.

A novel machine learning approach reveals latent vascular phenotypes predictive of renal cancer outcome.

Gene expression signatures are commonly used as predictive biomarkers, but do not capture structural features within the tissue architecture. Here we apply a 2-step machine learning framework for quantitative imaging of tumor vasculature to derive a spatially informed, prognostic gene signature. The trained algorithms classify endothelial cells and generate a vascular area mask (VAM) in H&E micrographs of clear cell renal cell carcinoma (ccRCC) cases from The Cancer Genome Atlas (TCGA). Quantification of VAMs led to the discovery of 9 vascular features (9VF) that predicted disease-free-survival in a discovery cohort (n = 64, HR = 2.3). Correlation analysis and information gain identified a 14 gene expression signature related to the 9VF's. Two generalized linear models with elastic net regularization (14VF and 14GT), based on the 14 genes, separated independent cohorts of up to 301 cases into good and poor disease-free survival groups (14VF HR = 2.4, 14GT HR = 3.33). For the first time, we successfully applied digital image analysis and targeted machine learning to develop prognostic, morphology-based, gene expression signatures from the vascular architecture. This novel morphogenomic approach has the potential to improve previous methods for biomarker development.

Data integration from pathology slides for quantitative imaging of multiple cell types within the tumor immune cell infiltrate.

BACKGROUND: Immune cell infiltrates (ICI) of tumors are scored by pathologists around tumor glands. To obtain a better understanding of the immune infiltrate, individual immune cell types, their activation states and location relative to tumor cells need to be determined. This process requires precise identification of the tumor area and enumeration of immune cell subtypes separately in the stroma and inside tumor nests. Such measurements can be accomplished by a multiplex format using immunohistochemistry (IHC). METHOD: We developed a pipeline that combines immunohistochemistry (IHC) and digital image analysis. One slide was stained with pan-cytokeratin and CD45 and the other slide with CD8, CD4 and CD68. The tumor mask generated through pan-cytokeratin staining was transferred from one slide to the other using affine image co-registration. Bland-Altman plots and Pearson correlation were used to investigate differences between densities and counts of immune cell underneath the transferred versus manually annotated tumor masks. One-way ANOVA was used to compare the mask transfer error for tissues with solid and glandular tumor architecture. RESULTS: The overlap between manual and transferred tumor masks ranged from 20%-90% across all cases. The error of transferring the mask was 2- to 4-fold greater in tumor regions with glandular compared to solid growth pattern (p < 10-6). Analyzing data from a single slide, the Pearson correlation coefficients of cell type densities outside and inside tumor regions were highest for CD4 + T-cells (r = 0.8), CD8 + T-cells (r = 0.68) or CD68+ macrophages (r = 0.79). The correlation coefficient for CD45+ T- and B-cells was only 0.45. The transfer of the mask generated an error in the measurement of intra- and extra- tumoral CD68+, CD8+ or CD4+ counts (p < 10-10). CONCLUSIONS: In summary, we developed a general method to integrate data from IHC stained slides into a single dataset. Because of the transfer error between slides, we recommend applying the antibody for demarcation of the tumor on the same slide as the ICI antibodies.

Quantitative imaging for development of companion diagnostics to drugs targeting HGF/MET.

The limited clinical success of anti-HGF/MET drugs can be attributed to the lack of predictive biomarkers that adequately select patients for treatment. We demonstrate here that quantitative digital imaging of formalin fixed paraffin embedded tissues stained by immunohistochemistry can be used to measure signals from weakly staining antibodies and provides new opportunities to develop assays for detection of MET receptor activity. To establish a biomarker panel of MET activation, we employed seven antibodies measuring protein expression in the HGF/MET pathway in 20 cases and up to 80 cores from 18 human cancer types. The antibodies bind to epitopes in the extra (EC)- and intracellular (IC) domains of MET (MET4EC, SP44_METIC, D1C2_METIC), to MET-pY1234/pY1235, a marker of MET kinase activation, as well as to HGF, pSFK or pMAPK. Expression of HGF was determined in tumour cells (T_HGF) as well as in stroma surrounding cancer (St_HGF). Remarkably, MET4EC correlated more strongly with pMET (r = 0.47) than SP44_METIC (r = 0.21) or D1C2_METIC (r = 0.08) across 18 cancer types. In addition, correlation coefficients of pMET and T_HGF (r = 0.38) and pMET and pSFK (r = 0.56) were high. Prediction models of MET activation reveal cancer-type specific differences in performance of MET4EC, SP44_METIC and anti-HGF antibodies. Thus, we conclude that assays to predict the response to HGF/MET inhibitors require a cancer-type specific antibody selection and should be developed in those cancer types in which they are employed clinically.

Rapid 3-D delineation of cell nuclei for high-content screening platforms.

High-resolution three-dimensional (3-D) microscopy combined with multiplexing of fluorescent labels allows high-content analysis of large numbers of cell nuclei. The full automation of 3-D screening platforms necessitates image processing algorithms that can accurately and robustly delineate nuclei in images with little to no human intervention. Imaging-based high-content screening was originally developed as a powerful tool for drug discovery. However, cell confluency, complexity of nuclear staining as well as poor contrast between nuclei and background result in slow and unreliable 3-D image processing and therefore negatively affect the performance of studying a drug response. Here, we propose a new method, 3D-RSD, to delineate nuclei by means of 3-D radial symmetries and test it on high-resolution image data of human cancer cells treated by drugs. The nuclei detection performance was evaluated by means of manually generated ground truth from 2351 nuclei (27 confocal stacks). When compared to three other nuclei segmentation methods, 3D-RSD possessed a better true positive rate of 83.3% and F-score of 0.895±0.045 (p-value=0.047). Altogether, 3D-RSD is a method with a very good overall segmentation performance. Furthermore, implementation of radial symmetries offers good processing speed, and makes 3D-RSD less sensitive to staining patterns. In particular, the 3D-RSD method performs well in cell lines, which are often used in imaging-based HCS platforms and are afflicted by nuclear crowding and overlaps that hinder feature extraction.

Machine learning approaches to analyze histological images of tissues from radical prostatectomies.

Computerized evaluation of histological preparations of prostate tissues involves identification of tissue components such as stroma (ST), benign/normal epithelium (BN) and prostate cancer (PCa). Image classification approaches have been developed to identify and classify glandular regions in digital images of prostate tissues; however their success has been limited by difficulties in cellular segmentation and tissue heterogeneity. We hypothesized that utilizing image pixels to generate intensity histograms of hematoxylin (H) and eosin (E) stains deconvoluted from H&E images numerically captures the architectural difference between glands and stroma. In addition, we postulated that joint histograms of local binary patterns and local variance (LBPxVAR) can be used as sensitive textural features to differentiate benign/normal tissue from cancer. Here we utilized a machine learning approach comprising of a support vector machine (SVM) followed by a random forest (RF) classifier to digitally stratify prostate tissue into ST, BN and PCa areas. Two pathologists manually annotated 210 images of low- and high-grade tumors from slides that were selected from 20 radical prostatectomies and digitized at high-resolution. The 210 images were split into the training (n=19) and test (n=191) sets. Local intensity histograms of H and E were used to train a SVM classifier to separate ST from epithelium (BN+PCa). The performance of SVM prediction was evaluated by measuring the accuracy of delineating epithelial areas. The Jaccard J=59.5 ± 14.6 and Rand Ri=62.0 ± 7.5 indices reported a significantly better prediction when compared to a reference method (Chen et al., Clinical Proteomics 2013, 10:18) based on the averaged values from the test set. To distinguish BN from PCa we trained a RF classifier with LBPxVAR and local intensity histograms and obtained separate performance values for BN and PCa: JBN=35.2 ± 24.9, OBN=49.6 ± 32, JPCa=49.5 ± 18.5, OPCa=72.7 ± 14.8 and Ri=60.6 ± 7.6 in the test set. Our pixel-based classification does not rely on the detection of lumens, which is prone to errors and has limitations in high-grade cancers and has the potential to aid in clinical studies in which the quantification of tumor content is necessary to prognosticate the course of the disease. The image data set with ground truth annotation is available for public use to stimulate further research in this area.