Robust, credible, and interpretable AI-based histopathological prostate cancer grading.

Background: Prostate cancer (PCa) is among the most common cancers in men and its diagnosis requires the histopathological evaluation of biopsies by human experts. While several recent artificial intelligence-based (AI) approaches have reached human expert-level PCa grading, they often display significantly reduced performance on external datasets. This reduced performance can be caused by variations in sample preparation, for instance the staining protocol, section thickness, or scanner used. Another limiting factor of contemporary AI-based PCa grading is the prediction of ISUP grades, which leads to the perpetuation of human annotation errors. Methods: We developed the prostate cancer aggressiveness index (PCAI), an AI-based PCa detection and grading framework that is trained on objective patient outcome, rather than subjective ISUP grades. We designed PCAI as a clinical application, containing algorithmic modules that offer robustness to data variation, medical interpretability, and a measure of prediction confidence. To train and evaluate PCAI, we generated a multicentric, retrospective, observational trial consisting of six cohorts with 25,591 patients, 83,864 images, and 5 years of median follow-up from 5 different centers and 3 countries. This includes a high-variance dataset of 8,157 patients and 28,236 images with variations in sample thickness, staining protocol, and scanner, allowing for the systematic evaluation and optimization of model robustness to data variation. The performance of PCAI was assessed on three external test cohorts from two countries, comprising 2,255 patients and 9,437 images. Findings: Using our high-variance datasets, we show how differences in sample processing, particularly slide thickness and staining time, significantly reduce the performance of AI-based PCa grading by up to 6.2 percentage points in the concordance index (C-index). We show how a select set of algorithmic improvements, including domain adversarial training, conferred robustness to data variation, interpretability, and a measure of credibility to PCAI. These changes lead to significant prediction improvement across two biopsy cohorts and one TMA cohort, systematically exceeding expert ISUP grading in C-index and AUROC by up to 22 percentage points. Interpretation: Data variation poses serious risks for AI-based histopathological PCa grading, even when models are trained on large datasets. Algorithmic improvements for model robustness, interpretability, credibility, and training on high-variance data as well as outcome-based severity prediction gives rise to robust models with above ISUP-level PCa grading performance.

AI-based prostate analysis system trained without human supervision to predict patient outcome from tissue samples.

In order to plan the best treatment for prostate cancer patients, the aggressiveness of the tumor is graded based on visual assessment of tissue biopsies according to the Gleason scale. Recently, a number of AI models have been developed that can be trained to do this grading as well as human pathologists. But the accuracy of the AI grading will be limited by the accuracy of the subjective "ground truth" Gleason grades used for the training. We have trained an AI to predict patient outcome directly based on image analysis of a large biobank of tissue samples with known outcome without input of any human knowledge about cancer grading. The model has shown similar and in some cases better ability to predict patient outcome on an independent test-set than expert pathologists doing the conventional grading.

Improved centerline tree detection of diseased peripheral arteries with a cascading algorithm for vascular segmentation.

Vascular segmentation plays an important role in the assessment of peripheral arterial disease. The segmentation is very challenging especially for arteries with severe stenosis or complete occlusion. We present a cascading algorithm for vascular centerline tree detection specializing in detecting centerlines in diseased peripheral arteries. It takes a three-dimensional computed tomography angiography (CTA) volume and returns a vascular centerline tree, which can be used for accelerating and facilitating the vascular segmentation. The algorithm consists of four levels, two of which detect healthy arteries of varying sizes and two that specialize in different types of vascular pathology: severe calcification and occlusion. We perform four main steps at each level: appropriate parameters for each level are selected automatically, a set of centrally located voxels is detected, these voxels are connected together based on the connection criteria, and the resulting centerline tree is corrected from spurious branches. The proposed method was tested on 25 CTA scans of the lower limbs, achieving an average overlap rate of 89% and an average detection rate of 82%. The average execution time using four CPU cores was 70 s, and the technique was successful also in detecting very distal artery branches, e.g., in the foot.

Acrosome integrity assessment of boar spermatozoa images using an early fusion of texture and contour descriptors.

The assessment of the state of the acrosome is a priority in artificial insemination centres since it is one of the main causes of function loss. In this work, boar spermatozoa present in gray scale images acquired with a phase-contrast microscope have been classified as acrosome-intact or acrosome-damaged, after using fluorescent images for creating the ground truth. Based on shape prior criteria combined with Otsu's thresholding, regional minima and watershed transform, the spermatozoa heads were segmented and registered. One of the main novelties of this proposal is that, unlike what previous works stated, the obtained results show that the contour information of the spermatozoon head is important for improving description and classification. Other of this work novelties is that it confirms that combining different texture descriptors and contour descriptors yield the best classification rates for this problem up to date. The classification was performed with a Support Vector Machine backed by a Least Squares training algorithm and a linear kernel. Using the biggest acrosome intact-damaged dataset ever created, the early fusion approach followed provides a 0.9913 F-Score, outperforming all previous related works.

Automated classification of glandular tissue by statistical proximity sampling.

Due to the complexity of biological tissue and variations in staining procedures, features that are based on the explicit extraction of properties from subglandular structures in tissue images may have difficulty generalizing well over an unrestricted set of images and staining variations. We circumvent this problem by an implicit representation that is both robust and highly descriptive, especially when combined with a multiple instance learning approach to image classification. The new feature method is able to describe tissue architecture based on glandular structure. It is based on statistically representing the relative distribution of tissue components around lumen regions, while preserving spatial and quantitative information, as a basis for diagnosing and analyzing different areas within an image. We demonstrate the efficacy of the method in extracting discriminative features for obtaining high classification rates for tubular formation in both healthy and cancerous tissue, which is an important component in Gleason and tubule-based Elston grading. The proposed method may be used for glandular classification, also in other tissue types, in addition to general applicability as a region-based feature descriptor in image analysis where the image represents a bag with a certain label (or grade) and the region-based feature vectors represent instances.

Simulation of bright-field microscopy images depicting pap-smear specimen.

As digital imaging is becoming a fundamental part of medical and biomedical research, the demand for computer-based evaluation using advanced image analysis is becoming an integral part of many research projects. A common problem when developing new image analysis algorithms is the need of large datasets with ground truth on which the algorithms can be tested and optimized. Generating such datasets is often tedious and introduces subjectivity and interindividual and intraindividual variations. An alternative to manually created ground-truth data is to generate synthetic images where the ground truth is known. The challenge then is to make the images sufficiently similar to the real ones to be useful in algorithm development. One of the first and most widely studied medical image analysis tasks is to automate screening for cervical cancer through Pap-smear analysis. As part of an effort to develop a new generation cervical cancer screening system, we have developed a framework for the creation of realistic synthetic bright-field microscopy images that can be used for algorithm development and benchmarking. The resulting framework has been assessed through a visual evaluation by experts with extensive experience of Pap-smear images. The results show that images produced using our described methods are realistic enough to be mistaken for real microscopy images. The developed simulation framework is very flexible and can be modified to mimic many other types of bright-field microscopy images.

3D texture analysis in renal cell carcinoma tissue image grading.

One of the most significant processes in cancer cell and tissue image analysis is the efficient extraction of features for grading purposes. This research applied two types of three-dimensional texture analysis methods to the extraction of feature values from renal cell carcinoma tissue images, and then evaluated the validity of the methods statistically through grade classification. First, we used a confocal laser scanning microscope to obtain image slices of four grades of renal cell carcinoma, which were then reconstructed into 3D volumes. Next, we extracted quantitative values using a 3D gray level cooccurrence matrix (GLCM) and a 3D wavelet based on two types of basis functions. To evaluate their validity, we predefined 6 different statistical classifiers and applied these to the extracted feature sets. In the grade classification results, 3D Haar wavelet texture features combined with principal component analysis showed the best discrimination results. Classification using 3D wavelet texture features was significantly better than 3D GLCM, suggesting that the former has potential for use in a computer-based grading system.

Image segmentation and identification of paired antibodies in breast tissue.

Comparing staining patterns of paired antibodies designed towards a specific protein but toward different epitopes of the protein provides quality control over the binding and the antibodies' ability to identify the target protein correctly and exclusively. We present a method for automated quantification of immunostaining patterns for antibodies in breast tissue using the Human Protein Atlas database. In such tissue, dark brown dye 3,3'-diaminobenzidine is used as an antibody-specific stain whereas the blue dye hematoxylin is used as a counterstain. The proposed method is based on clustering and relative scaling of features following principal component analysis. Our method is able (1) to accurately segment and identify staining patterns and quantify the amount of staining and (2) to detect paired antibodies by correlating the segmentation results among different cases. Moreover, the method is simple, operating in a low-dimensional feature space, and computationally efficient which makes it suitable for high-throughput processing of tissue microarrays.

Screening for cervical cancer using automated analysis of PAP-smears.

Cervical cancer is one of the most deadly and common forms of cancer among women if no action is taken to prevent it, yet it is preventable through a simple screening test, the so-called PAP-smear. This is the most effective cancer prevention measure developed so far. But the visual examination of the smears is time consuming and expensive and there have been numerous attempts at automating the analysis ever since the test was introduced more than 60 years ago. The first commercial systems for automated analysis of the cell samples appeared around the turn of the millennium but they have had limited impact on the screening costs. In this paper we examine the key issues that need to be addressed when an automated analysis system is developed and discuss how these challenges have been met over the years. The lessons learned may be useful in the efforts to create a cost-effective screening system that could make affordable screening for cervical cancer available for all women globally, thus preventing most of the quarter million annual unnecessary deaths still caused by this disease.

Debris removal in Pap-smear images.

Since its introduction in the 1940s the Pap-smear test has helped reduce the incidence of cervical cancer dramatically in countries where regular screening is standard. The automation of this procedure is an open problem that has been ongoing for over fifty years without reaching satisfactory results. Existing systems are discouragingly expensive and yet they are only able to make a correct distinction between normal and abnormal samples in a fraction of cases. Therefore, they are limited to acting as support for the cytotechnicians as they perform their manual screening. The main reason for the current limitations is that the automated systems struggle to overcome the complexity of the cell structures. Samples are covered in artefacts such as blood cells, overlapping and folded cells, and bacteria, that hamper the segmentation processes and generate large number of suspicious objects. The classifiers designed to differentiate between normal cells and pre-cancerous cells produce unpredictable results when classifying artefacts. In this paper, we propose a sequential classification scheme focused on removing unwanted objects, debris, from an initial segmentation result, intended to be run before the actual normal/abnormal classifier. The method has been evaluated using three separate datasets obtained from cervical samples prepared using both the standard Pap-smear approach as well as the more recent liquid based cytology sample preparation technique. We show success in removing more than 99% of the debris without loosing more than around one percent of the epithelial cells detected by the segmentation process.

Blind color decomposition of histological images.

Cancer diagnosis is based on visual examination under a microscope of tissue sections from biopsies. But whereas pathologists rely on tissue stains to identify morphological features, automated tissue recognition using color is fraught with problems that stem from image intensity variations due to variations in tissue preparation, variations in spectral signatures of the stained tissue, spectral overlap and spatial aliasing in acquisition, and noise at image acquisition. We present a blind method for color decomposition of histological images. The method decouples intensity from color information and bases the decomposition only on the tissue absorption characteristics of each stain. By modeling the charge-coupled device sensor noise, we improve the method accuracy. We extend current linear decomposition methods to include stained tissues where one spectral signature cannot be separated from all combinations of the other tissues' spectral signatures. We demonstrate both qualitatively and quantitatively that our method results in more accurate decompositions than methods based on non-negative matrix factorization and independent component analysis. The result is one density map for each stained tissue type that classifies portions of pixels into the correct stained tissue allowing accurate identification of morphological features that may be linked to cancer.

Automated segmentation of free-lying cell nuclei in Pap smears for malignancy-associated change analysis.

This paper presents an automated algorithm for robustly detecting and segmenting free-lying cell nuclei in bright-field microscope images of Pap smears. This is an essential initial step in the development of an automated screening system for cervical cancer based on malignancy associated change (MAC) analysis. The proposed segmentation algorithm makes use of gray-scale annular closings to identify free-lying nuclei-like objects together with marker-based watershed segmentation to accurately delineate the nuclear boundaries. The algorithm also employs artifact rejection based on size, shape, and granularity to ensure only the nuclei of intermediate squamous epithelial cells are retained. An evaluation of the performance of the algorithm relative to expert manual segmentation of 33 fields-of-view from 11 Pap smear slides is also presented. The results show that the sensitivity and specificity of nucleus detection is 94.71% and 85.30% respectively, and that the accuracy of segmentation, measured using the Dice coefficient, of the detected nuclei is 97.30±1.3%.

Characterization and reduction of noise in dynamic PET data using masked volumewise principal component analysis.

UNLABELLED: Masked volumewise principal component (PC) analysis (PCA) is used in PET to distinguish structures that display different kinetic behaviors after administration of a tracer. When masked volumewise PCA was introduced, one article proposed noise prenormalization because of temporal and spatial variations of the noise between slices. However, the noise prenormalization proposed in that article was applicable only to datasets reconstructed using filtered backprojection (FBP). The study presented in this article aimed at developing a new noise prenormalization that is applicable to datasets regardless of whether they were reconstructed with FBP or an iterative reconstruction algorithm, such as ordered-subset expectation maximization (OSEM). METHODS: A phantom study was performed to investigate differences in the expectation values and SDs of datasets reconstructed with FBP and OSEM. A novel method, higher-order PC noise prenormalization, was suggested and evaluated against other prenormalization methods on clinical datasets. RESULTS: Masked volumewise PCA of data reconstructed with FBP was much more dependent on an appropriate prenormalization than was analysis of data reconstructed with OSEM. Higher-order PC noise prenormalization showed an overall good performance with both FBP and OSEM reconstructions, whereas the other prenormalization methods performed well with only 1 of the 2 methods. CONCLUSION: Higher-order PC noise prenormalization has potential for improving the results from masked volumewise PCA on dynamic PET datasets independent of the type of reconstruction algorithm.

Effects of aging and gender on the spatial organization of nuclei in single human skeletal muscle cells.

The skeletal muscle fibre is a syncitium where each myonucleus regulates the gene products in a finite volume of the cytoplasm, i.e., the myonuclear domain (MND). We analysed aging- and gender-related effects on myonuclei organization and the MND size in single muscle fibres from six young (21-31 years) and nine old men (72-96 years), and from six young (24-32 years) and nine old women (65-96 years), using a novel image analysis algorithm applied to confocal images. Muscle fibres were classified according to myosin heavy chain (MyHC) isoform expression. Our image analysis algorithm was effective in determining the spatial organization of myonuclei and the distribution of individual MNDs along the single fibre segments. Significant linear relations were observed between MND size and fibre size, irrespective age, gender and MyHC isoform expression. The spatial organization of individual myonuclei, calculated as the distribution of nearest neighbour distances in 3D, and MND size were affected in old age, but changes were dependent on MyHC isoform expression. In type I muscle fibres, average NN-values were lower and showed an increased variability in old age, reflecting an aggregation of myonuclei in old age. Average MND size did not change in old age, but there was an increased MND size variability. In type IIa fibres, average NN-values and MND sizes were lower in old age, reflecting the smaller size of these muscle fibres in old age. It is suggested that these changes have a significant impact on protein synthesis and degradation during the aging process.

Signal extraction and separation in in vivo animal PET studies with masked volumewise principal-component analysis.

UNLABELLED: The standardized uptake value is commonly used as a tool to supplement visual interpretation and to quantify the images acquired from static in vivo animal PET. The preferred approach for analyzing PET data is either to sum the images and calculate the standardized uptake value or to use kinetic modeling. The aim of this study was to investigate the performance of masked volumewise principal-component analysis (MVW-PCA) used in dynamic in vivo animal PET studies to extract and separate signals with different kinetic behaviors. METHODS: PET data were acquired with a small-animal PET scanner and a fluorine tracer in a study of rats and mice. After acquisition, the data were reconstructed by use of 4 time protocols with different frame lengths. Data were analyzed by use of MVW-PCA with applied noise prenormalization and a new masking technique developed in this study. RESULTS: The resulting principal-component images showed a clear separation of the activity in the spine into the first MVW-PCA component and the activity in the kidneys into the second MVW-PCA component. In addition, the different time protocols were shown to have little or no impact on the results obtained with MVW-PCA. CONCLUSION: MVW-PCA can efficiently separate different kinetic behaviors into different principal-component images. Moreover, MVW-PCA is a stable technique in the sense that the time protocol chosen has only a small impact on the resulting principal-component images.

Performance of principal component analysis and independent component analysis with respect to signal extraction from noisy positron emission tomography data - a study on computer simulated images.

Multivariate image analysis tools are used for analyzing dynamic or multidimensional Positron Emission Tomography, PET data with the aim of noise reduction, dimension reduction and signal separation. Principal Component Analysis is one of the most commonly used multivariate image analysis tools, applied on dynamic PET data. Independent Component Analysis is another multivariate image analysis tool used to extract and separate signals. Because of the presence of high and variable noise levels and correlation in the different PET images which may confound the multivariate analysis, it is essential to explore and investigate different types of pre-normalization (transformation) methods that need to be applied, prior to application of these tools. In this study, we explored the performance of Principal Component Analysis (PCA) and Independent Component Analysis (ICA) to extract signals and reduce noise, thereby increasing the Signal to Noise Ratio (SNR) in a dynamic sequence of PET images, where the features of the noise are different compared with some other medical imaging techniques. Applications on computer simulated PET images were explored and compared. Application of PCA generated relatively similar results, with some minor differences, on the images with different noise characteristics. However, clear differences were seen with respect to the type of pre-normalization. ICA on images normalized using two types of normalization methods also seemed to perform relatively well but did not reach the improvement in SNR as PCA. Furthermore ICA seems to have a tendency under some conditions to shift over information from IC1 to other independent components and to be more sensitive to the level of noise. PCA is a more stable technique than ICA and creates better results both qualitatively and quantitatively in the simulated PET images. PCA can extract the signals from the noise rather well and is not sensitive to type of noise, magnitude and correlation, when the input data are correctly handled by a proper pre-normalization. It is important to note that PCA as inherently a method to separate signal information into different components could still generate PC1 images with improved SNR as compared to mean images.

Myonuclear domain size and myosin isoform expression in muscle fibres from mammals representing a 100,000-fold difference in body size.

This comparative study of myonuclear domain (MND) size in mammalian species representing a 100,000-fold difference in body mass, ranging from 25 g to 2500 kg, was undertaken to improve our understanding of myonuclear organization in skeletal muscle fibres. Myonuclear domain size was calculated from three-dimensional reconstructions in a total of 235 single muscle fibre segments at a fixed sarcomere length. Irrespective of species, the largest MND size was observed in muscle fibres expressing fast myosin heavy chain (MyHC) isoforms, but in the two smallest mammalian species studied (mouse and rat), MND size was not larger in the fast-twitch fibres expressing the IIA MyHC isofom than in the slow-twitch type I fibres. In the larger mammals, the type I fibres always had the smallest average MND size, but contrary to mouse and rat muscles, type IIA fibres had lower mitochondrial enzyme activities than type I fibres. Myonuclear domain size was highly dependent on body mass in the two muscle fibre types expressed in all species, i.e. types I and IIA. Myonuclear domain size increased in muscle fibres expressing both the beta/slow (type I; r = 0.84, P < 0.001) and the fast IIA MyHC isoform (r = 0.90; P < 0.001). Thus, MND size scales with body size and is highly dependent on muscle fibre type, independent of species. However, myosin isoform expression is not the sole protein determining MND size, and other protein systems, such as mitochondrial proteins, may be equally or more important determinants of MND size.