Evaluating Cellularity Estimation Methods: Comparing AI Counting with Pathologists' Visual Estimates.

The development of next-generation sequencing (NGS) has enabled the discovery of cancer-specific driver gene alternations, making precision medicine possible. However, accurate genetic testing requires a sufficient amount of tumor cells in the specimen. The evaluation of tumor content ratio (TCR) from hematoxylin and eosin (H&E)-stained images has been found to vary between pathologists, making it an important challenge to obtain an accurate TCR. In this study, three pathologists exhaustively labeled all cells in 41 regions from 41 lung cancer cases as either tumor, non-tumor or indistinguishable, thus establishing a "gold standard" TCR. We then compared the accuracy of the TCR estimated by 13 pathologists based on visual assessment and the TCR calculated by an AI model that we have developed. It is a compact and fast model that follows a fully convolutional neural network architecture and produces cell detection maps which can be efficiently post-processed to obtain tumor and non-tumor cell counts from which TCR is calculated. Its raw cell detection accuracy is 92% while its classification accuracy is 84%. The results show that the error between the gold standard TCR and the AI calculation was significantly smaller than that between the gold standard TCR and the pathologist's visual assessment (p<0.05). Additionally, the robustness of AI models across institutions is a key issue and we demonstrate that the variation in AI was smaller than that in the average of pathologists when evaluated by institution. These findings suggest that the accuracy of tumor cellularity assessments in clinical workflows is significantly improved by the introduction of robust AI models, leading to more efficient genetic testing and ultimately to better patient outcomes.

Development of multiple AI pipelines that predict neoadjuvant chemotherapy response of breast cancer using H&E-stained tissues.

In recent years, the treatment of breast cancer has advanced dramatically and neoadjuvant chemotherapy (NAC) has become a common treatment method, especially for locally advanced breast cancer. However, other than the subtype of breast cancer, no clear factor indicating sensitivity to NAC has been identified. In this study, we attempted to use artificial intelligence (AI) to predict the effect of preoperative chemotherapy from hematoxylin and eosin images of pathological tissue obtained from needle biopsies prior to chemotherapy. Application of AI to pathological images typically uses a single machine-learning model such as support vector machines (SVMs) or deep convolutional neural networks (CNNs). However, cancer tissues are extremely diverse and learning with a realistic number of cases limits the prediction accuracy of a single model. In this study, we propose a novel pipeline system that uses three independent models each focusing on different characteristics of cancer atypia. Our system uses a CNN model to learn structural atypia from image patches and SVM and random forest models to learn nuclear atypia from fine-grained nuclear features extracted by image analysis methods. It was able to predict the NAC response with 95.15% accuracy on a test set of 103 unseen cases. We believe that this AI pipeline system will contribute to the adoption of personalized medicine in NAC therapy for breast cancer.

Prediction of non-muscle invasive bladder cancer recurrence using machine learning of quantitative nuclear features.

Non-muscle invasive bladder cancer (NMIBC) generally has a good prognosis; however, recurrence after transurethral resection (TUR), the standard primary treatment, is a major problem. Clinical management after TUR has been based on risk classification using clinicopathological factors, but these classifications are not complete. In this study, we attempted to predict early recurrence of NMIBC based on machine learning of quantitative morphological features. In general, structural, cellular, and nuclear atypia are evaluated to determine cancer atypia. However, since it is difficult to accurately quantify structural atypia from TUR specimens, in this study, we used only nuclear atypia and analyzed it using feature extraction followed by classification using Support Vector Machine and Random Forest machine learning algorithms. For the analysis, 125 patients diagnosed with NMIBC were used; data from 95 patients were randomly selected for the training set, and data from 30 patients were randomly selected for the test set. The results showed that the support vector machine-based model predicted recurrence within 2 years after TUR with a probability of 90% and the random forest-based model with probability of 86.7%. In the future, the system can be used to objectively predict NMIBC recurrence after TUR.

Prediction of early recurrence of hepatocellular carcinoma after resection using digital pathology images assessed by machine learning.

Hepatocellular carcinoma (HCC) is a representative primary liver cancer caused by long-term and repetitive liver injury. Surgical resection is generally selected as the radical cure treatment. Because the early recurrence of HCC after resection is associated with low overall survival, the prediction of recurrence after resection is clinically important. However, the pathological characteristics of the early recurrence of HCC have not yet been elucidated. We attempted to predict the early recurrence of HCC after resection based on digital pathologic images of hematoxylin and eosin-stained specimens and machine learning applying a support vector machine (SVM). The 158 HCC patients meeting the Milan criteria who underwent surgical resection were included in this study. The patients were categorized into three groups: Group I, patients with HCC recurrence within 1 year after resection (16 for training and 23 for test); Group II, patients with HCC recurrence between 1 and 2 years after resection (22 and 28); and Group III, patients with no HCC recurrence within 4 years after resection (31 and 38). The SVM-based prediction method separated the three groups with 89.9% (80/89) accuracy. Prediction of Groups I was consistent for all cases, while Group II was predicted to be Group III in one case, and Group III was predicted to be Group II in 8 cases. The use of digital pathology and machine learning could be used for highly accurate prediction of HCC recurrence after surgical resection, especially that for early recurrence. Currently, in most cases after HCC resection, regular blood tests and diagnostic imaging are used for follow-up observation; however, the use of digital pathology coupled with machine learning offers potential as a method for objective postoprative follow-up observation.

Automated histological classification of whole-slide images of gastric biopsy specimens.

BACKGROUND: Automated image analysis has been developed currently in the field of surgical pathology. The aim of the present study was to evaluate the classification accuracy of the e-Pathologist image analysis software. METHODS: A total of 3062 gastric biopsy specimens were consecutively obtained and stained. The specimen slides were anonymized and digitized. At least two experienced gastrointestinal pathologists evaluated each slide for pathological diagnosis. We compared the three-tier (positive for carcinoma or suspicion of carcinoma; caution for adenoma or suspicion of a neoplastic lesion; or negative for a neoplastic lesion) or two-tier (negative or non-negative) classification results of human pathologists and of the e-Pathologist. RESULTS: Of 3062 cases, 33.4% showed an abnormal finding. For the three-tier classification, the overall concordance rate was 55.6% (1702/3062). The kappa coefficient was 0.28 (95% CI, 0.26-0.30; fair agreement). For the negative biopsy specimens, the concordance rate was 90.6% (1033/1140), but for the positive biopsy specimens, the concordance rate was less than 50%. For the two-tier classification, the sensitivity, specificity, positive predictive value, and negative predictive value were 89.5% (95% CI, 87.5-91.4%), 50.7% (95% CI, 48.5-52.9%), 47.7% (95% CI, 45.4-49.9%), and 90.6% (95% CI, 88.8-92.2%), respectively. CONCLUSIONS: Although there are limitations and requirements for applying automated histopathological classification of gastric biopsy specimens in the clinical setting, the results of the present study are promising.

Automated histological classification of whole slide images of colorectal biopsy specimens.

BACKGROUND: An automated image analysis system, e-Pathologist, was developed to improve the quality of colorectal biopsy diagnostics in routine pathology practice. OBJECTIVE: The aim of the study was to evaluate the classification accuracy of the e-Pathologist image analysis software in the setting of routine pathology practice in two institutions. MATERIALS AND METHODS: In total, 1328 colorectal tissue specimens were consecutively obtained from two hospitals (1077 tissues from Tokyo hospital, and 251 tissues from East hospital) and the stained specimen slides were anonymized and digitized. At least two experienced gastrointestinal pathologists evaluated each slide for pathological diagnosis. We compared the 3-tier classification results (carcinoma or suspicion of carcinoma, adenoma, and lastly negative for a neoplastic lesion) between the human pathologists and that of e-Pathologist. RESULTS: For the Tokyo hospital specimens, all carcinoma tissues were correctly classified (n=112), and 9.9% (80/810) of the adenoma tissues were incorrectly classified as negative. For the East hospital specimens, 0 out of the 51 adenoma tissues were incorrectly classified as negative while 9.3% (11/118) of the carcinoma tissues were incorrectly classified as either adenoma, or negative. For the Tokyo and East hospital datasets, the undetected rate of carcinoma, undetected rate of adenoma, and over-detected proportion were 0% and 9.3%, 9.9% and 0%, and 36.1% and 27.1%, respectively. CONCLUSIONS: This image analysis system requires some improvements; however, it has the potential to assist pathologists in quality improvement of routine pathological practice in the not too distant future.

Pathological Diagnosis of Gastric Cancers with a Novel Computerized Analysis System.

BACKGROUND: Recent studies of molecular biology have provided great advances for diagnostic molecular pathology. Automated diagnostic systems with computerized scanning for sampled cells in fluids or smears are now widely utilized. Automated analysis of tissue sections is, however, very difficult because they exhibit a complex mixture of overlapping malignant tumor cells, benign host-derived cells, and extracellular materials. Thus, traditional histological diagnosis is still the most powerful method for diagnosis of diseases. METHODS: We have developed a novel computer-assisted pathology system for rapid, automated histological analysis of hematoxylin and eosin (H and E)-stained sections. It is a multistage recognition system patterned after methods that human pathologists use for diagnosis but harnessing machine learning and image analysis. The system first analyzes an entire H and E-stained section (tissue) at low resolution to search suspicious areas for cancer and then the selected areas are analyzed at high resolution to confirm the initial suspicion. RESULTS: After training the pathology system with gastric tissues samples, we examined its performance using other 1905 gastric tissues. The system's accuracy in detecting malignancies was shown to be almost equal to that of conventional diagnosis by expert pathologists. CONCLUSIONS: Our novel computerized analysis system provides a support for histological diagnosis, which is useful for screening and quality control. We consider that it could be extended to be applicable to many other carcinomas after learning normal and malignant forms of various tissues. Furthermore, we expect it to contribute to the development of more objective grading systems, immunohistochemical staining systems, and fluorescent-stained image analysis systems.

Quantitative diagnosis of breast tumors by morphometric classification of microenvironmental myoepithelial cells using a machine learning approach.

Machine learning systems have recently received increased attention for their broad applications in several fields. In this study, we show for the first time that histological types of breast tumors can be classified using subtle morphological differences of microenvironmental myoepithelial cell nuclei without any direct information about neoplastic tumor cells. We quantitatively measured 11661 nuclei on the four histological types: normal cases, usual ductal hyperplasia and low/high grade ductal carcinoma in situ (DCIS). Using a machine learning system, we succeeded in classifying the four histological types with 90.9% accuracy. Electron microscopy observations suggested that the activity of typical myoepithelial cells in DCIS was lowered. Through these observations as well as meta-analytic database analyses, we developed a paracrine cross-talk-based biological mechanism of DCIS progressing to invasive cancer. Our observations support novel approaches in clinical computational diagnostics as well as in therapy development against progression.

A novel method for morphological pleomorphism and heterogeneity quantitative measurement: Named cell feature level co-occurrence matrix.

BACKGROUND: Recent developments in molecular pathology and genetic/epigenetic analysis of cancer tissue have resulted in a marked increase in objective and measurable data. In comparison, the traditional morphological analysis approach to pathology diagnosis, which can connect these molecular data and clinical diagnosis, is still mostly subjective. Even though the advent and popularization of digital pathology has provided a boost to computer-aided diagnosis, some important pathological concepts still remain largely non-quantitative and their associated data measurements depend on the pathologist's sense and experience. Such features include pleomorphism and heterogeneity. METHODS AND RESULTS: In this paper, we propose a method for the objective measurement of pleomorphism and heterogeneity, using the cell-level co-occurrence matrix. Our method is based on the widely used Gray-level co-occurrence matrix (GLCM), where relations between neighboring pixel intensity levels are captured into a co-occurrence matrix, followed by the application of analysis functions such as Haralick features. In the pathological tissue image, through image processing techniques, each nucleus can be measured and each nucleus has its own measureable features like nucleus size, roundness, contour length, intra-nucleus texture data (GLCM is one of the methods). In GLCM each nucleus in the tissue image corresponds to one pixel. In this approach the most important point is how to define the neighborhood of each nucleus. We define three types of neighborhoods of a nucleus, then create the co-occurrence matrix and apply Haralick feature functions. In each image pleomorphism and heterogeneity are then determined quantitatively. For our method, one pixel corresponds to one nucleus feature, and we therefore named our method Cell Feature Level Co-occurrence Matrix (CFLCM). We tested this method for several nucleus features. CONCLUSION: CFLCM is showed as a useful quantitative method for pleomorphism and heterogeneity on histopathological image analysis.

Quantitative nucleic features are effective for discrimination of intraductal proliferative lesions of the breast.

BACKGROUND: Intraductal proliferative lesions (IDPLs) of the breast are recognized as a risk factor for subsequent invasive carcinoma development. Although opportunities for IDPL diagnosis have increased, these lesions are difficult to diagnose correctly, especially atypical ductal hyperplasia (ADH) and low-grade ductal carcinoma in situ (LG-DCIS). In order to define the difference between these lesions, many molecular pathological approaches have been performed. However, still we do not have a molecular marker and objective histological index about IDPLs of the breast. METHODS: We generated full digital pathology archives from 175 female IDPL patients, including usual ductal hyperplasia (UDH), ADH, LG-DCIS, intermediate-grade (IM)-DCIS, and high-grade (HG)-DCIS. After total 2,035,807 nucleic segmentations were extracted, we evaluated nuclear features using step-wise linear discriminant analysis (LDA) and a support vector machine. RESULTS: High diagnostic accuracy (81.8-99.3%) was achieved between pathologists' diagnoses and two-group LDA predictions from nucleic features for IDPL discrimination. Grouping of nuclear features as size and shape-related or intranuclear texture-related revealed that the latter group was more important when distinguishing between normal duct, UDH, ADH, and LG-DCIS. However, these two groups were equally important when discriminating between LG-DCIS and HG-DCIS. The Mahalanobis distances between each group showed that the smallest distance values occurred between LG-DCIS and IM-DCIS and between ADH and Normal. On the other hand, the distance value between ADH and LG-DCIS was larger than this distance. CONCLUSIONS: In this study, we have presented a practical and useful digital pathological method that incorporates nuclear morphological and textural features for IDPL prediction. We expect that this novel algorithm is used for the automated diagnosis assisting system for breast cancer.

Classification of mitotic figures with convolutional neural networks and seeded blob features.

BACKGROUND: The mitotic figure recognition contest at the 2012 International Conference on Pattern Recognition (ICPR) challenges a system to identify all mitotic figures in a region of interest of hematoxylin and eosin stained tissue, using each of three scanners (Aperio, Hamamatsu, and multispectral). METHODS: Our approach combines manually designed nuclear features with the learned features extracted by convolutional neural networks (CNN). The nuclear features capture color, texture, and shape information of segmented regions around a nucleus. The use of a CNN handles the variety of appearances of mitotic figures and decreases sensitivity to the manually crafted features and thresholds. RESULTS: On the test set provided by the contest, the trained system achieves F1 scores up to 0.659 on color scanners and 0.589 on multispectral scanner. CONCLUSIONS: We demonstrate a powerful technique combining segmentation-based features with CNN, identifying the majority of mitotic figures with a fair precision. Further, we show that the approach accommodates information from the additional focal planes and spectral bands from a multi-spectral scanner without major redesign.

Mitotic figure recognition: agreement among pathologists and computerized detector.

Despite the prognostic importance of mitotic count as one of the components of the Bloom-Richardson grade, several studies have found that pathologists' agreement on the mitotic grade is fairly modest. Collecting a set of more than 4,200 candidate mitotic figures, we evaluate pathologists' agreement on individual figures, and train a computerized system for mitosis detection, comparing its performance to the classifications of three pathologists. The system's and the pathologists' classifications are based on evaluation of digital micrographs of hematoxylin and eosin stained breast tissue. On figures where the majority of pathologists agree on a classification, we compare the performance of the trained system to that of the individual pathologists. We find that the level of agreement of the pathologists ranges from slight to moderate, with strong biases, and that the system performs competitively in rating the ground truth set. This study is a step towards automatic mitosis count to accelerate a pathologist's work and improve reproducibility.

Mitotic index of invasive breast carcinoma. Achieving clinically meaningful precision and evaluating tertial cutoffs.

CONTEXT: Mitotic figure counts are related to breast cancer behavior but have not been sufficiently reproducible to be accepted for clinical decision-making. OBJECTIVE: To improve reproducibility and accuracy of the mitotic count. DESIGN: Mitotic index (MI) was defined as the mitotic cell count per 10 high-power fields (HPFs), an area 0.183 mm(2). Two to 6 replicate sets of 10 HPFs were counted from 328 invasive breast carcinomas. Standard errors and coefficients of variation for mean MI were compared with expected results predicted by the binomial distribution. RESULTS: The boundaries for MI that separated the data into equal thirds (tertials) were 1.14 and 5.33. Standard errors and coefficients of variation for MI followed distributions predicted by binomial probability. Mean coefficient of variation was 147% for the low tertial, 72% for the midtertial, and 34.6% for the upper tertial. CONCLUSIONS: Standard errors for MI based on a single count of 10 HPFs are too broad and coefficients of variation too large to be acceptable for clinical use. This is explained as a binomial probability effect, possibly with a contribution from tumor heterogeneity. Errors can be reduced in proportion to the square root of the number of sets of 10 HPFs counted. Tertial cutoffs of MI of the Nottingham system currently used in breast carcinoma grading are too high to be applicable to the population we studied. We recommend validation of cutoffs before they are applied to a particular population of breast carcinomas. Counting 5 sets of 10 HPFs is necessary to accurately rank carcinomas with low MIs.