The NCI Imaging Data Commons as a platform for reproducible research in computational pathology.

BACKGROUND AND OBJECTIVES: Reproducibility is a major challenge in developing machine learning (ML)-based solutions in computational pathology (CompPath). The NCI Imaging Data Commons (IDC) provides >120 cancer image collections according to the FAIR principles and is designed to be used with cloud ML services. Here, we explore its potential to facilitate reproducibility in CompPath research. METHODS: Using the IDC, we implemented two experiments in which a representative ML-based method for classifying lung tumor tissue was trained and/or evaluated on different datasets. To assess reproducibility, the experiments were run multiple times with separate but identically configured instances of common ML services. RESULTS: The results of different runs of the same experiment were reproducible to a large extent. However, we observed occasional, small variations in AUC values, indicating a practical limit to reproducibility. CONCLUSIONS: We conclude that the IDC facilitates approaching the reproducibility limit of CompPath research (i) by enabling researchers to reuse exactly the same datasets and (ii) by integrating with cloud ML services so that experiments can be run in identically configured computing environments.

Artificial Intelligence in Pathology: From Prototype to Product.

Modern image analysis techniques based on artificial intelligence (AI) have great potential to improve the quality and efficiency of diagnostic procedures in pathology and to detect novel biomarkers. Despite thousands of published research papers on applications of AI in pathology, hardly any research implementations have matured into commercial products for routine use. Bringing an AI solution for pathology to market poses significant technological, business, and regulatory challenges. In this paper, we provide a comprehensive overview and advice on how to meet these challenges. We outline how research prototypes can be turned into a product-ready state and integrated into the IT infrastructure of clinical laboratories. We also discuss business models for profitable AI solutions and reimbursement options for computer assistance in pathology. Moreover, we explain how to obtain regulatory approval so that AI solutions can be launched as in vitro diagnostic medical devices. Thus, this paper offers computer scientists, software companies, and pathologists a road map for transforming prototypes of AI solutions into commercial products.

NCI Imaging Data Commons.

The National Cancer Institute (NCI) Cancer Research Data Commons (CRDC) aims to establish a national cloud-based data science infrastructure. Imaging Data Commons (IDC) is a new component of CRDC supported by the Cancer Moonshot. The goal of IDC is to enable a broad spectrum of cancer researchers, with and without imaging expertise, to easily access and explore the value of deidentified imaging data and to support integrated analyses with nonimaging data. We achieve this goal by colocating versatile imaging collections with cloud-based computing resources and data exploration, visualization, and analysis tools. The IDC pilot was released in October 2020 and is being continuously populated with radiology and histopathology collections. IDC provides access to curated imaging collections, accompanied by documentation, a user forum, and a growing number of analysis use cases that aim to demonstrate the value of a data commons framework applied to cancer imaging research. SIGNIFICANCE: This study introduces NCI Imaging Data Commons, a new repository of the NCI Cancer Research Data Commons, which will support cancer imaging research on the cloud.

Automated density-based counting of FISH amplification signals for HER2 status assessment.

BACKGROUND: Automated image analysis can make quantification of FISH signals in histological sections more efficient and reproducible. Current detection-based methods, however, often fail to accurately quantify densely clustered FISH signals. METHODS: We propose a novel density-based approach to quantifying FISH signals. Instead of detecting individual signals, this approach quantifies FISH signals in terms of the integral over a density map predicted by Deep Learning. We apply the density-based approach to the task of counting and determining ratios of ERBB2 and CEN17 signals and compare it to common detection-based and area-based approaches. RESULTS: The ratios determined by our approach were strongly correlated with results obtained by manual annotation of individual FISH signals (Pearson's r = 0.907). In addition, they were highly consistent with cutoff-scores determined by a pathologist (balanced concordance = 0.971). The density-based approach generally outperformed the other approaches. Its superiority was particularly evident in the presence of dense signal clusters. CONCLUSIONS: The presented approach enables accurate and efficient automated quantification of FISH signals. Since signals in clusters can hardly be detected individually even by human observers, the density-based quantification performs better than detection-based approaches.

Focused scores enable reliable discrimination of small differences in steatosis.

BACKGROUND: Automated image analysis enables quantitative measurement of steatosis in histological images. However, spatial heterogeneity of steatosis can make quantitative steatosis scores unreliable. To improve the reliability, we have developed novel scores that are "focused" on steatotic tissue areas. METHODS: Focused scores use concepts of tile-based hotspot analysis in order to compute statistics about steatotic tissue areas in an objective way. We evaluated focused scores on three data sets of images of rodent liver sections exhibiting different amounts of dietary-induced steatosis. The same evaluation was conducted with the standard steatosis score computed by most image analysis methods. RESULTS: The standard score reliably discriminated large differences in steatosis (intraclass correlation coefficient ICC = 0.86), but failed to discriminate small (ICC = 0.54) and very small (ICC = 0.14) differences. With an appropriate tile size, mean-based focused scores reliably discriminated large (ICC = 0.92), small (ICC = 0.86) and very small (ICC = 0.83) differences. Focused scores based on high percentiles showed promise in further improving the discrimination of very small differences (ICC = 0.93). CONCLUSIONS: Focused scores enable reliable discrimination of small differences in steatosis in histological images. They are conceptually simple and straightforward to use in research studies.

Deep learning nuclei detection: A simple approach can deliver state-of-the-art results.

BACKGROUND: Deep convolutional neural networks have become a widespread tool for the detection of nuclei in histopathology images. Many implementations share a basic approach that includes generation of an intermediate map indicating the presence of a nucleus center, which we refer to as PMap. Nevertheless, these implementations often still differ in several parameters, resulting in different detection qualities. METHODS: We identified several essential parameters and configured the basic PMap approach using combinations of them. We thoroughly evaluated and compared various configurations on multiple datasets with respect to detection quality, efficiency and training effort. RESULTS: Post-processing of the PMap was found to have the largest impact on detection quality. Also, two different network architectures were identified that improve either detection quality or runtime performance. The best-performing configuration yields f1-measures of 0.816 on H&E stained images of colorectal adenocarcinomas and 0.819 on Ki-67 stained images of breast tumor tissue. On average, it was fully trained in less than 15,000 iterations and processed 4.15 megapixels per second at prediction time. CONCLUSIONS: The basic PMap approach is greatly affected by certain parameters. Our evaluation provides guidance on their impact and best settings. When configured properly, this simple and efficient approach can yield equal detection quality as more complex and time-consuming state-of-the-art approaches.