Computational pathology: A survey review and the way forward.

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.

Wireless Body Area Nanonetworks via Vascular Molecular Communication.

Advancements in biotechnology and molecular communication have enabled the utilization of nanomachines in Wireless Body Area Networks (WBAN2) for applications such as drug delivery, cancer detection, and emergency rescue services. To study these networks effectively, it is essential to develop an ideal propagation model that includes the channel response between each pair of in-range nanomachines and accounts for the interference received at each receiver node. In this paper, we employ an advection-diffusion equation to obtain a deterministic channel matrix through a vascular WBAN2. Additionally, the closed forms of inter-symbol interference (ISI) and co-channel interference (CCI) are derived for both full duplex (FDX) and half duplex transmission (HDX) modes. By applying these deterministic formulations, we then present the stochastic equivalents of the ideal channel model and interference to provide an innovative communication model by simultaneously incorporating CCI, ISI, and background noise. Finally, we evaluate the results with numerous experiments and use signal-to-interference-plus-noise ratio (SINR) and capacity as metrics.

BoostCaps: A Boosted Capsule Network for Brain Tumor Classification.

Brain tumor is among the deadliest cancers, whose effective treatment is partially dependent on the accurate diagnosis of the tumor type. Convolutional neural networks (CNNs), which have been the state-of-the-art in brain tumor classification, fail to identify the spatial relations in the image. Capsule networks, proposed to overcome this drawback, are sensitive to miscellaneous backgrounds and cannot manage to focus on the main target. To address this shortcoming, we have recently proposed a capsule network-based architecture capable of taking both brain images and tumor rough boundary boxes as inputs, to have access to the surrounding tissue as well as the main target. Similar to other architectures, however, this network requires extensive search within the space of all possible configurations, to find the optimal architecture. To eliminate this need, in this study, we propose a boosted capsule network, referred to as BoostCaps, which takes advantage of the ability of boosting methods to handle weak learners, by gradually boosting the models. BoosCaps, to the best of our knowledge, is the first capsule network model that incorporates an internal boosting mechanism. Our results show that the proposed BoostCaps framework outperforms its single capsule network counterpart.

How much off-the-shelf knowledge is transferable from natural images to pathology images?

Deep learning has achieved a great success in natural image classification. To overcome data-scarcity in computational pathology, recent studies exploit transfer learning to reuse knowledge gained from natural images in pathology image analysis, aiming to build effective pathology image diagnosis models. Since transferability of knowledge heavily depends on the similarity of the original and target tasks, significant differences in image content and statistics between pathology images and natural images raise the questions: how much knowledge is transferable? Is the transferred information equally contributed by pre-trained layers? If not, is there a sweet spot in transfer learning that balances transferred model's complexity and performance? To answer these questions, this paper proposes a framework to quantify knowledge gain by a particular layer, conducts an empirical investigation in pathology image centered transfer learning, and reports some interesting observations. Particularly, compared to the performance baseline obtained by a random-weight model, though transferability of off-the-shelf representations from deep layers heavily depend on specific pathology image sets, the general representation generated by early layers does convey transferred knowledge in various image classification applications. The trade-off between transferable performance and transferred model's complexity observed in this study encourages further investigation of specific metric and tools to quantify effectiveness of transfer learning in future.

[Formula: see text]: deep learning-based radiomics for the time-to-event outcome prediction in lung cancer.

Hand-crafted radiomics has been used for developing models in order to predict time-to-event clinical outcomes in patients with lung cancer. Hand-crafted features, however, are pre-defined and extracted without taking the desired target into account. Furthermore, accurate segmentation of the tumor is required for development of a reliable predictive model, which may be objective and a time-consuming task. To address these drawbacks, we propose a deep learning-based radiomics model for the time-to-event outcome prediction, referred to as DRTOP that takes raw images as inputs, and calculates the image-based risk of death or recurrence, for each patient. Our experiments on an in-house dataset of 132 lung cancer patients show that the obtained image-based risks are significant predictors of the time-to-event outcomes. Computed Tomography (CT)-based features are predictors of the overall survival (OS), with the hazard ratio (HR) of 1.35, distant control (DC), with HR of 1.06, and local control (LC), with HR of 2.66. The Positron Emission Tomography (PET)-based features are predictors of OS and recurrence free survival (RFS), with hazard ratios of 1.67 and 1.18, respectively. The concordance indices of [Formula: see text], [Formula: see text], and [Formula: see text] for predicting the OS, DC, and RFS show that the deep learning-based radiomics model is as accurate or better in predicting predefined clinical outcomes compared to hand-crafted radiomics, with concordance indices of [Formula: see text], [Formula: see text], and [Formula: see text], for predicting the OS, DC, and RFS, respectively. Deep learning-based radiomics has the potential to offer complimentary predictive information in the personalized management of lung cancer patients.

3D-MCN: A 3D Multi-scale Capsule Network for Lung Nodule Malignancy Prediction.

Despite the advances in automatic lung cancer malignancy prediction, achieving high accuracy remains challenging. Existing solutions are mostly based on Convolutional Neural Networks (CNNs), which require a large amount of training data. Most of the developed CNN models are based only on the main nodule region, without considering the surrounding tissues. Obtaining high sensitivity is challenging with lung nodule malignancy prediction. Moreover, the interpretability of the proposed techniques should be a consideration when the end goal is to utilize the model in a clinical setting. Capsule networks (CapsNets) are new and revolutionary machine learning architectures proposed to overcome shortcomings of CNNs. Capitalizing on the success of CapsNet in biomedical domains, we propose a novel model for lung tumor malignancy prediction. The proposed framework, referred to as the 3D Multi-scale Capsule Network (3D-MCN), is uniquely designed to benefit from: (i) 3D inputs, providing information about the nodule in 3D; (ii) Multi-scale input, capturing the nodule's local features, as well as the characteristics of the surrounding tissues, and; (iii) CapsNet-based design, being capable of dealing with a small number of training samples. The proposed 3D-MCN architecture predicted lung nodule malignancy with a high accuracy of 93.12%, sensitivity of 94.94%, area under the curve (AUC) of 0.9641, and specificity of 90% when tested on the LIDC-IDRI dataset. When classifying patients as having a malignant condition (i.e., at least one malignant nodule is detected) or not, the proposed model achieved an accuracy of 83%, and a sensitivity and specificity of 84% and 81% respectively.

Size and Shape Filtering of Malignant Cell Clusters within Breast Tumors Identifies Scattered Individual Epithelial Cells as the Most Valuable Histomorphological Clue in the Prognosis of Distant Metastasis Risk.

Survival and life quality of breast cancer patients could be improved by more aggressive chemotherapy for those at high metastasis risk and less intense treatments for low-risk patients. Such personalized treatment cannot be currently achieved due to the insufficient reliability of metastasis risk prognosis. The purpose of this study was therefore, to identify novel histopathological prognostic markers of metastasis risk through exhaustive computational image analysis of 80 size and shape subsets of epithelial clusters in breast tumors. The group of 102 patients had a follow-up median of 12.3 years, without lymph node spread and systemic treatments. Epithelial cells were stained by the AE1/AE3 pan-cytokeratin antibody cocktail. The size and shape subsets of the stained epithelial cell clusters were defined in each image by use of the circularity and size filters and analyzed for prognostic performance. Epithelial areas with the optimal prognostic performance were uniformly small and round and could be recognized as individual epithelial cells scattered in tumor stroma. Their count achieved an area under the receiver operating characteristic curve (AUC) of 0.82, total area (AUC = 0.77), average size (AUC = 0.63), and circularity (AUC = 0.62). In conclusion, by use of computational image analysis as a hypothesis-free discovery tool, this study reveals the histomorphological marker with a high prognostic value that is simple and therefore easy to quantify by visual microscopy.

Novel chromaticity similarity based color texture descriptor for digital pathology image analysis.

Pathology images are color in nature due to the use of chemical staining in biopsy examination. Aware of the high color diagnosticity in pathology images, this work introduces a compact rotation-invariant texture descriptor, named quantized diagnostic counter-color pattern (QDCP), for digital pathology image understanding. On the basis of color similarity quantified by the inner product of unit-length color vectors, local counter-color textons are indexed first. Then the underlined distribution of QDCP indexes is estimated by an image-wise histogram. Since QDCP is computed based on color difference directly, it is robust to small color variation usually observed in pathology images. This study also discusses QDCP's extraction, parameter settings, and feature fusion techniques in a generic pathology image analysis pipeline, and introduces two more descriptors QDCP-LBP and QDCP/LBP. Experimentation on public pathology image sets suggests that the introduced color texture descriptors, especially QDCP-LBP, outperform prior color texture features in terms of strong descriptive power, low computational complexity, and high adaptability to different image sets.

The Pan-Cytokeratin Staining Intensity and Fractal Computational Analysis of Breast Tumor Malignant Growth Patterns Prognosticate the Occurrence of Distant Metastasis.

Improved prognosis of breast cancer outcome could prolong patient survival by reliable identification of patients at high risk of metastasis occurrence which could benefit from more aggressive treatments. Based on such clinical need, we prognostically evaluated the malignant cells in breast tumors, as the obvious potential source of unexploited prognostic information. The patient group was homogeneous, without any systemic treatments or lymph node spread, with smaller tumor size (pT1/2) and a long follow-up. Epithelial cells were labeled with AE1/AE3 pan-cytokeratin antibody cocktail and comprehensively analyzed. Monofractal and multifractal analyses were applied for quantification of distribution, shape, complexity and texture of malignant cell clusters, while mean pixel intensity and total area were measures of the pan-cytokeratin immunostaining intensity. The results surprisingly indicate that simple binary images and monofractal analysis provided better prognostic information then grayscale images and multifractal analysis. The key findings were that shapes and distribution of malignant cell clusters (by binary fractal dimension; AUC = 0.29), their contour shapes (by outline fractal dimension; AUC = 0.31) and intensity of the pan-cytokeratin immunostaining (by mean pixel intensity; AUC = 0.30) offered significant performance in metastasis risk prognostication. The results reveal an association between the lower pan-cytokeratin staining intensity and the high metastasis risk. Another interesting result was that multivariate analysis could confirm the prognostic independence only for fractal but not for immunostaining intensity features. The obtained results reveal several novel and unexpected findings highlighting the independent prognostic efficacy of malignant cell cluster distribution and contour shapes in breast tumors.

Circular Mixture Modeling of Color Distribution for Blind Stain Separation in Pathology Images.

In digital pathology, to address color variation and histological component colocalization in pathology images, stain decomposition is usually performed preceding spectral normalization and tissue component segmentation. This paper examines the problem of stain decomposition, which is a naturally nonnegative matrix factorization (NMF) problem in algebra, and introduces a systematical and analytical solution consisting of a circular color analysis module and an NMF-based computation module. Unlike the paradigm of existing stain decomposition algorithms where stain proportions are computed from estimated stain spectra using a matrix inverse operation directly, the introduced solution estimates stain spectra and stain depths via probabilistic reasoning individually. Since the proposed method pays extra attentions to achromatic pixels in color analysis and stain co-occurrence in pixel clustering, it achieves consistent and reliable stain decomposition with minimum decomposition residue. Particularly, aware of the periodic and angular nature of hue, we propose the use of a circular von Mises mixture model to analyze the hue distribution, and provide a complete color-based pixel soft-clustering solution to address color mixing introduced by stain overlap. This innovation combined with saturation-weighted computation makes our study effective for weak stains and broad-spectrum stains. Extensive experimentation on multiple public pathology datasets suggests that our approach outperforms state-of-the-art blind stain separation methods in terms of decomposition effectiveness.

A Complete Color Normalization Approach to Histopathology Images Using Color Cues Computed From Saturation-Weighted Statistics.

GOAL: In digital histopathology, tasks of segmentation and disease diagnosis are achieved by quantitative analysis of image content. However, color variation in image samples makes it challenging to produce reliable results. This paper introduces a complete normalization scheme to address the problem of color variation in histopathology images jointly caused by inconsistent biopsy staining and nonstandard imaging condition. Method : Different from existing normalization methods that either address partial cause of color variation or lump them together, our method identifies causes of color variation based on a microscopic imaging model and addresses inconsistency in biopsy imaging and staining by an illuminant normalization module and a spectral normalization module, respectively. In evaluation, we use two public datasets that are representative of histopathology images commonly received in clinics to examine the proposed method from the aspects of robustness to system settings, performance consistency against achromatic pixels, and normalization effectiveness in terms of histological information preservation. RESULTS: As the saturation-weighted statistics proposed in this study generates stable and reliable color cues for stain normalization, our scheme is robust to system parameters and insensitive to image content and achromatic colors. CONCLUSION: Extensive experimentation suggests that our approach outperforms state-of-the-art normalization methods as the proposed method is the only approach that succeeds to preserve histological information after normalization. SIGNIFICANCE: The proposed color normalization solution would be useful to mitigate effects of color variation in pathology images on subsequent quantitative analysis.

Computer-aided detection (CAD) of breast masses in mammography: combined detection and ensemble classification.

We propose a novel computer-aided detection (CAD) framework of breast masses in mammography. To increase detection sensitivity for various types of mammographic masses, we propose the combined use of different detection algorithms. In particular, we develop a region-of-interest combination mechanism that integrates detection information gained from unsupervised and supervised detection algorithms. Also, to significantly reduce the number of false-positive (FP) detections, the new ensemble classification algorithm is developed. Extensive experiments have been conducted on a benchmark mammogram database. Results show that our combined detection approach can considerably improve the detection sensitivity with a small loss of FP rate, compared to representative detection algorithms previously developed for mammographic CAD systems. The proposed ensemble classification solution also has a dramatic impact on the reduction of FP detections; as much as 70% (from 15 to 4.5 per image) at only cost of 4.6% sensitivity loss (from 90.0% to 85.4%). Moreover, our proposed CAD method performs as well or better (70.7% and 80.0% per 1.5 and 3.5 FPs per image respectively) than the results of mammography CAD algorithms previously reported in the literature.

Denoising 3D ultrasound volumes using sparse representation.

In this paper, a new 3D ultrasound (US) denoising technique that adopts the sparse representation has been proposed for an effective noise reduction in 3D US volumes. The purpose of the proposed method is to reduce image noise while preserving 3D objects edges, hence improving the human interpretation for clinical diagnosis and the 3D segmentation accuracy for further automatic malignancy detection. For denoising 3D US volumes, sparse representation was employed, which has showed an excellent performance in reducing Gaussian noise. It has been well known that US images contain severe multiplicative speckle noise, which has different characteristics compared to the additive Gaussian noise. In this paper, we propose a denoising framework for effectively reducing both Gaussian noise and speckle noise on 3D US volumes. The proposed method removes Gaussian noise using sparse representation. Then, a logarithmic transform is performed to transform the speckle noise into Gaussian noise for applying the sparse representation. To demonstrate the effectiveness of the proposed denoising method, comparative and quantitative experiments had been conducted on a synthesized 3D US phantom data. Experimental results showed that the proposed denoising could improve image quality in terms of denoising measurements.

Combining multiple feature representations and AdaBoost ensemble learning for reducing false-positive detections in computer-aided detection of masses on mammograms.

One of the drawbacks of current Computer-aided Detection (CADe) systems is a high number of false-positive (FP) detections, especially for detecting mass abnormalities. In a typical CADe system, classifier design is one of the key steps for determining FP detection rates. This paper presents the effective classifier ensemble system for tackling FP reduction problem in CADe. To construct ensemble consisting of correct classifiers while disagreeing with each other as much as possible, we develop a new ensemble construction solution that combines data resampling underpinning AdaBoost learning with the use of different feature representations. In addition, to cope with the limitation of weak classifiers in conventional AdaBoost, our method has an effective mechanism for tuning the level of weakness of base classifiers. Further, for combining multiple decision outputs of ensemble members, a weighted sum fusion strategy is used to maximize a complementary effect for correct classification. Comparative experiments have been conducted on benchmark mammogram dataset. Results show that the proposed classifier ensemble outperforms the best single classifier in terms of reducing the FP detections of masses.