Nondestructive 3D pathology with analysis of nuclear features for prostate cancer risk assessment.

Prostate cancer treatment decisions rely heavily on subjective visual interpretation [assigning Gleason patterns or International Society of Urological Pathology (ISUP) grade groups] of limited numbers of two-dimensional (2D) histology sections. Under this paradigm, interobserver variance is high, with ISUP grades not correlating well with outcome for individual patients, and this contributes to the over- and undertreatment of patients. Recent studies have demonstrated improved prognostication of prostate cancer outcomes based on computational analyses of glands and nuclei within 2D whole slide images. Our group has also shown that the computational analysis of three-dimensional (3D) glandular features, extracted from 3D pathology datasets of whole intact biopsies, can allow for improved recurrence prediction compared to corresponding 2D features. Here we seek to expand on these prior studies by exploring the prognostic value of 3D shape-based nuclear features in prostate cancer (e.g. nuclear size, sphericity). 3D pathology datasets were generated using open-top light-sheet (OTLS) microscopy of 102 cancer-containing biopsies extracted ex vivo from the prostatectomy specimens of 46 patients. A deep learning-based workflow was developed for 3D nuclear segmentation within the glandular epithelium versus stromal regions of the biopsies. 3D shape-based nuclear features were extracted, and a nested cross-validation scheme was used to train a supervised machine classifier based on 5-year biochemical recurrence (BCR) outcomes. Nuclear features of the glandular epithelium were found to be more prognostic than stromal cell nuclear features (area under the ROC curve [AUC] = 0.72 versus 0.63). 3D shape-based nuclear features of the glandular epithelium were also more strongly associated with the risk of BCR than analogous 2D features (AUC = 0.72 versus 0.62). The results of this preliminary investigation suggest that 3D shape-based nuclear features are associated with prostate cancer aggressiveness and could be of value for the development of decision-support tools. © 2023 The Pathological Society of Great Britain and Ireland.

Visual Assessment of 2-Dimensional Levels Within 3-Dimensional Pathology Data Sets of Prostate Needle Biopsies Reveals Substantial Spatial Heterogeneity.

Prostate cancer prognostication largely relies on visual assessment of a few thinly sectioned biopsy specimens under a microscope to assign a Gleason grade group (GG). Unfortunately, the assigned GG is not always associated with a patient's outcome in part because of the limited sampling of spatially heterogeneous tumors achieved by 2-dimensional histopathology. In this study, open-top light-sheet microscopy was used to obtain 3-dimensional pathology data sets that were assessed by 4 human readers. Intrabiopsy variability was assessed by asking readers to perform Gleason grading of 5 different levels per biopsy for a total of 20 core needle biopsies (ie, 100 total images). Intrabiopsy variability (Cohen κ) was calculated as the worst pairwise agreement in GG between individual levels within each biopsy and found to be 0.34, 0.34, 0.38, and 0.43 for the 4 pathologists. These preliminary results reveal that even within a 1-mm-diameter needle core, GG based on 2-dimensional images can vary dramatically depending on the location within a biopsy being analyzed. We believe that morphologic assessment of whole biopsies in 3 dimension has the potential to enable more reliable and consistent tumor grading.

Intra-nucleus mosaic pattern (InMop) and whole-cell Haralick combined-descriptor for identifying and characterizing acute leukemia blasts on single cell peripheral blood images.

Acute leukemia is usually diagnosed when a test of peripheral blood shows at least 20% of abnormal immature cells (blasts), a figure even lower in case of recurrent cytogenetic abnormalities. Blast identification is crucial for white blood cell (WBC) counting, which depends on both identifying the cell type and characterizing the cellular morphology, processes susceptible of inter- and intraobserver variability. The present work introduces an image combined-descriptor to detect blasts and determine their probable lineage. This strategy uses an intra-nucleus mosaic pattern (InMop) descriptor that captures subtle nuclei differences within WBCs, and Haralick's statistics which quantify the local structure of both nucleus and cytoplasm. The InMop captures WBC inner-nucleus structure by applying a multiscale Shearlet decomposition over a repetitive pattern (mosaic) of automatically-segmented nuclei. As a complement, Haralick's statistics characterize the local structure of the whole cell from an intensity co-occurrence matrix representation. Both InMoP and Haralick-based descriptors are calculated using the b-channel from Lab color-space. The combined-descriptor is assessed by differentiating blasts from nonleukemic cells with support vector machine (SVM) classifiers and different transformation kernels, in two public and independent databases. The first database-D1 (n = 260) is composed of healthy and acute lymphoid leukemia (ALL) single cell images, and second database-D2 contains acute myeloid leukemia (AML) blasts (n = 3294) and nonblast (n = 15,071) cell images. In a first experiment, blasts versus nonblast differentiation is performed by training with a subset of D2 (n = 6588) and testing in D1 (n = 260), obtaining a training AUC of 0.991 ± 0.002 and AUC = 0.782 for the independent validation. A second experiment automatically differentiates AML blasts (260 images from D2) from ALL blasts (260 images from D1), with an AUC of 0.93. In a third experiment, state-of-the-art strategies, VGG16 and RESNEXT convolutional neural networks (CNN), separate blast from nonblast cells in both databases. The VGG16 showed an AUC of 0.673 and the RESNEXT of 0.75. Reported metrics for all the experiments are area under the ROC curve (AUC), accuracy and F1-score.

The state of the art for artificial intelligence in lung digital pathology.

Lung diseases carry a significant burden of morbidity and mortality worldwide. The advent of digital pathology (DP) and an increase in computational power have led to the development of artificial intelligence (AI)-based tools that can assist pathologists and pulmonologists in improving clinical workflow and patient management. While previous works have explored the advances in computational approaches for breast, prostate, and head and neck cancers, there has been a growing interest in applying these technologies to lung diseases as well. The application of AI tools on radiology images for better characterization of indeterminate lung nodules, fibrotic lung disease, and lung cancer risk stratification has been well documented. In this article, we discuss methodologies used to build AI tools in lung DP, describing the various hand-crafted and deep learning-based unsupervised feature approaches. Next, we review AI tools across a wide spectrum of lung diseases including cancer, tuberculosis, idiopathic pulmonary fibrosis, and COVID-19. We discuss the utility of novel imaging biomarkers for different types of clinical problems including quantification of biomarkers like PD-L1, lung disease diagnosis, risk stratification, and prediction of response to treatments such as immune checkpoint inhibitors. We also look briefly at some emerging applications of AI tools in lung DP such as multimodal data analysis, 3D pathology, and transplant rejection. Lastly, we discuss the future of DP-based AI tools, describing the challenges with regulatory approval, developing reimbursement models, planning clinical deployment, and addressing AI biases. © 2022 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Computer-extracted features of nuclear morphology in hematoxylin and eosin images distinguish stage II and IV colon tumors.

We assessed the utility of quantitative features of colon cancer nuclei, extracted from digitized hematoxylin and eosin-stained whole slide images (WSIs), to distinguish between stage II and stage IV colon cancers. Our discovery cohort comprised 100 stage II and stage IV colon cancer cases sourced from the University Hospitals Cleveland Medical Center (UHCMC). We performed initial (independent) model validation on 51 (143) stage II and 79 (54) stage IV colon cancer cases from UHCMC (The Cancer Genome Atlas's Colon Adenocarcinoma, TCGA-COAD, cohort). Our approach comprised the following steps: (1) a fully convolutional deep neural network with VGG-18 architecture was trained to locate cancer on WSIs; (2) another deep-learning model based on Mask-RCNN with Resnet-50 architecture was used to segment all nuclei from within the identified cancer region; (3) a total of 26 641 quantitative morphometric features pertaining to nuclear shape, size, and texture were extracted from within and outside tumor nuclei; (4) a random forest classifier was trained to distinguish between stage II and stage IV colon cancers using the five most discriminatory features selected by the Wilcoxon rank-sum test. Our trained classifier using these top five features yielded an AUC of 0.81 and 0.78, respectively, on the held-out cases in the UHCMC and TCGA validation sets. For 197 TCGA-COAD cases, the Cox proportional hazards model yielded a hazard ratio of 2.20 (95% CI 1.24-3.88) with a concordance index of 0.71, using only the top five features for risk stratification of overall survival. The Kaplan-Meier estimate also showed statistically significant separation between the low-risk and high-risk patients, with a log-rank P value of 0.0097. Finally, unsupervised clustering of the top five features revealed that stage IV colon cancers with peritoneal spread were morphologically more similar to stage II colon cancers with no long-term metastases than to stage IV colon cancers with hematogenous spread. © 2022 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Image analysis reveals differences in tumor multinucleations in Black and White patients with human papillomavirus-associated oropharyngeal squamous cell carcinoma.

BACKGROUND: Understanding biological differences between different racial groups of human papillomavirus (HPV)-associated oropharyngeal squamous cell carcinoma (OPSCC) patients, who have differences in terms of incidence, survival, and tumor morphology, can facilitate accurate prognostic biomarkers, which can help develop personalized treatment strategies. METHODS: This study evaluated whether there were morphologic differences between HPV-associated tumors from Black and White patients in terms of multinucleation index (MuNI), an image analysis-derived metric that measures density of multinucleated tumor cells within epithelial regions on hematoxylin-eosin images and previously has been prognostic in HPV-associated OPSCC patients. In this study, the authors specifically evaluated whether the same MuNI cutoff that was prognostic of overall survival (OS) and disease-free survival in their previous study, TTR , is valid for Black and White patients, separately. We also evaluated population-specific cutoffs, TB for Blacks and TW for Whites, for risk stratification. RESULTS: MuNI was statistically significantly different between Black (mean, 3.88e-4; median, 3.67e-04) and White patients (mean, 3.36e-04; median, 2.99e-04), with p = .0078. Using TTR , MuNI was prognostic of OS in the entire population with hazard ratio (HR) of 1.71 (p = .002; 95% confidence interval [CI], 1.21-2.43) and in White patients with HR of 1.72 (p = .005; 95% CI, 1.18-2.51). Population-specific cutoff, TW , yielded improved HR of 1.77 (p = .003; 95% CI, 1.21-2.58) for White patients, whereas TB did not improve risk-stratification in Black patients with HR of 0.6 (p = .3; HR, 0.6; 95% CI, 0.2-1.80). CONCLUSIONS: Histological difference between White and Black patient tumors in terms of multinucleated tumor cells suggests the need for considering population-specific prognostic biomarkers for personalized risk stratification strategies for HPV-associated OPSCC patients.

Oropharyngeal cancer outcomes correlate with p16 status, multinucleation and immune infiltration.

Oropharyngeal squamous cell carcinoma (OPSCC), largely fueled by the human papillomavirus (HPV), has a complex biological and immunologic phenotype. Although HPV/p16 status can be used to stratify OPSCC patients as a function of survival, it remains unclear what drives an improved treatment response in HPV-associated OPSCC and whether targetable biomarkers exist that can inform a precision oncology approach. We analyzed OPSCC patients treated between 2000 and 2016 and correlated locoregional control (LRC), disease-free survival (DFS) and overall survival (OS) with conventional clinical parameters, risk parameters generated using deep-learning algorithms trained to quantify tumor-infiltrating lymphocytes (TILs) (OP-TIL) and multinucleated tumor cells (MuNI) and targeted transcriptomics. P16 was a dominant determinant of LRC, DFS and OS, but tobacco exposure, OP-TIL and MuNI risk features correlated with clinical outcomes independent of p16 status and the combination of p16, OP-TIL and MuNI generated a better stratification of OPSCC risk compared to individual parameters. Differential gene expression (DEG) analysis demonstrated overlap between MuNI and OP-TIL and identified genes involved in DNA repair, oxidative stress response and tumor immunity as the most prominent correlates with survival. Alteration of inflammatory/immune pathways correlated strongly with all risk features and oncologic outcomes. This suggests that development of OPSCC consists of an intersection between multiple required and permissive oncogenic and immunologic events which may be mechanistically linked. The strong relationship between tumor immunity and oncologic outcomes in OPSCC regardless of HPV status may provide opportunities for further biomarker development and precision oncology approaches incorporating immune checkpoint inhibitors for maximal anti-tumor efficacy.

Assessment of a computerized quantitative quality control tool for whole slide images of kidney biopsies.

Inconsistencies in the preparation of histology slides and whole-slide images (WSIs) may lead to challenges with subsequent image analysis and machine learning approaches for interrogating the WSI. These variabilities are especially pronounced in multicenter cohorts, where batch effects (i.e. systematic technical artifacts unrelated to biological variability) may introduce biases to machine learning algorithms. To date, manual quality control (QC) has been the de facto standard for dataset curation, but remains highly subjective and is too laborious in light of the increasing scale of tissue slide digitization efforts. This study aimed to evaluate a computer-aided QC pipeline for facilitating a reproducible QC process of WSI datasets. An open source tool, HistoQC, was employed to identify image artifacts and compute quantitative metrics describing visual attributes of WSIs to the Nephrotic Syndrome Study Network (NEPTUNE) digital pathology repository. A comparison in inter-reader concordance between HistoQC aided and unaided curation was performed to quantify improvements in curation reproducibility. HistoQC metrics were additionally employed to quantify the presence of batch effects within NEPTUNE WSIs. Of the 1814 WSIs (458 H&E, 470 PAS, 438 silver, 448 trichrome) from n = 512 cases considered in this study, approximately 9% (163) were identified as unsuitable for subsequent computational analysis. The concordance in the identification of these WSIs among computational pathologists rose from moderate (Gwet's AC1 range 0.43 to 0.59 across stains) to excellent (Gwet's AC1 range 0.79 to 0.93 across stains) agreement when aided by HistoQC. Furthermore, statistically significant batch effects (p < 0.001) in the NEPTUNE WSI dataset were discovered. Taken together, our findings strongly suggest that quantitative QC is a necessary step in the curation of digital pathology cohorts. © 2020 The Pathological Society of Great Britain and Ireland. Published by John Wiley & Sons, Ltd.

An automated computational image analysis pipeline for histological grading of cardiac allograft rejection.

AIM: Allograft rejection is a serious concern in heart transplant medicine. Though endomyocardial biopsy with histological grading is the diagnostic standard for rejection, poor inter-pathologist agreement creates significant clinical uncertainty. The aim of this investigation is to demonstrate that cellular rejection grades generated via computational histological analysis are on-par with those provided by expert pathologists. METHODS AND RESULTS: The study cohort consisted of 2472 endomyocardial biopsy slides originating from three major US transplant centres. The 'Computer-Assisted Cardiac Histologic Evaluation (CACHE)-Grader' pipeline was trained using an interpretable, biologically inspired, 'hand-crafted' feature extraction approach. From a menu of 154 quantitative histological features relating the density and orientation of lymphocytes, myocytes, and stroma, a model was developed to reproduce the 4-grade clinical standard for cellular rejection diagnosis. CACHE-grader interpretations were compared with independent pathologists and the 'grade of record', testing for non-inferiority (δ = 6%). Study pathologists achieved a 60.7% agreement [95% confidence interval (CI): 55.2-66.0%] with the grade of record, and pair-wise agreement among all human graders was 61.5% (95% CI: 57.0-65.8%). The CACHE-Grader met the threshold for non-inferiority, achieving a 65.9% agreement (95% CI: 63.4-68.3%) with the grade of record and a 62.6% agreement (95% CI: 60.3-64.8%) with all human graders. The CACHE-Grader demonstrated nearly identical performance in internal and external validation sets (66.1% vs. 65.8%), resilience to inter-centre variations in tissue processing/digitization, and superior sensitivity for high-grade rejection (74.4% vs. 39.5%, P < 0.001). CONCLUSION: These results show that the CACHE-grader pipeline, derived using intuitive morphological features, can provide expert-quality rejection grading, performing within the range of inter-grader variability seen among human pathologists.

Development and evaluation of deep learning-based segmentation of histologic structures in the kidney cortex with multiple histologic stains.

The application of deep learning for automated segmentation (delineation of boundaries) of histologic primitives (structures) from whole slide images can facilitate the establishment of novel protocols for kidney biopsy assessment. Here, we developed and validated deep learning networks for the segmentation of histologic structures on kidney biopsies and nephrectomies. For development, we examined 125 biopsies for Minimal Change Disease collected across 29 NEPTUNE enrolling centers along with 459 whole slide images stained with Hematoxylin & Eosin (125), Periodic Acid Schiff (125), Silver (102), and Trichrome (107) divided into training, validation and testing sets (ratio 6:1:3). Histologic structures were manually segmented (30048 total annotations) by five nephropathologists. Twenty deep learning models were trained with optimal digital magnification across the structures and stains. Periodic Acid Schiff-stained whole slide images yielded the best concordance between pathologists and deep learning segmentation across all structures (F-scores: 0.93 for glomerular tufts, 0.94 for glomerular tuft plus Bowman's capsule, 0.91 for proximal tubules, 0.93 for distal tubular segments, 0.81 for peritubular capillaries, and 0.85 for arteries and afferent arterioles). Optimal digital magnifications were 5X for glomerular tuft/tuft plus Bowman's capsule, 10X for proximal/distal tubule, arteries and afferent arterioles, and 40X for peritubular capillaries. Silver stained whole slide images yielded the worst deep learning performance. Thus, this largest study to date adapted deep learning for the segmentation of kidney histologic structures across multiple stains and pathology laboratories. All data used for training and testing and a detailed online tutorial will be publicly available.

Quality control stress test for deep learning-based diagnostic model in digital pathology.

Digital pathology provides a possibility for computational analysis of histological slides and automatization of routine pathological tasks. Histological slides are very heterogeneous concerning staining, sections' thickness, and artifacts arising during tissue processing, cutting, staining, and digitization. In this study, we digitally reproduce major types of artifacts. Using six datasets from four different institutions digitized by different scanner systems, we systematically explore artifacts' influence on the accuracy of the pre-trained, validated, deep learning-based model for prostate cancer detection in histological slides. We provide evidence that any histological artifact dependent on severity can lead to a substantial loss in model performance. Strategies for the prevention of diagnostic model accuracy losses in the context of artifacts are warranted. Stress-testing of diagnostic models using synthetically generated artifacts might be an essential step during clinical validation of deep learning-based algorithms.

Prospective Evaluation of Repeatability and Robustness of Radiomic Descriptors in Healthy Brain Tissue Regions In Vivo Across Systematic Variations in T2-Weighted Magnetic Resonance Imaging Acquisition Parameters.

BACKGROUND: Radiomic descriptors from magnetic resonance imaging (MRI) are promising for disease diagnosis and characterization but may be sensitive to differences in imaging parameters. OBJECTIVE: To evaluate the repeatability and robustness of radiomic descriptors within healthy brain tissue regions on prospectively acquired MRI scans; in a test-retest setting, under controlled systematic variations of MRI acquisition parameters, and after postprocessing. STUDY TYPE: Prospective. SUBJECTS: Fifteen healthy participants. FIELD STRENGTH/SEQUENCE: A 3.0 T, axial T2 -weighted 2D turbo spin-echo pulse sequence, 181 scans acquired (2 test/retest reference scans and 12 with systematic variations in contrast weighting, resolution, and acceleration per participant; removing scans with artifacts). ASSESSMENT: One hundred and forty-six radiomic descriptors were extracted from a contiguous 2D region of white matter in each scan, before and after postprocessing. STATISTICAL TESTS: Repeatability was assessed in a test/retest setting and between manual and automated annotations for the reference scan. Robustness was evaluated between the reference scan and each group of variant scans (contrast weighting, resolution, and acceleration). Both repeatability and robustness were quantified as the proportion of radiomic descriptors that fell into distinct ranges of the concordance correlation coefficient (CCC): excellent (CCC > 0.85), good (0.7 ≤ CCC ≤ 0.85), moderate (0.5 ≤ CCC < 0.7), and poor (CCC < 0.5); for unprocessed and postprocessed scans separately. RESULTS: Good to excellent repeatability was observed for 52% of radiomic descriptors between test/retest scans and 48% of descriptors between automated vs. manual annotations, respectively. Contrast weighting (TR/TE) changes were associated with the largest proportion of highly robust radiomic descriptors (21%, after processing). Image resolution changes resulted in the largest proportion of poorly robust radiomic descriptors (97%, before postprocessing). Postprocessing of images with only resolution/acceleration differences resulted in 73% of radiomic descriptors showing poor robustness. DATA CONCLUSIONS: Many radiomic descriptors appear to be nonrobust across variations in MR contrast weighting, resolution, and acceleration, as well in test-retest settings, depending on feature formulation and postprocessing. EVIDENCE LEVEL: 2 TECHNICAL EFFICACY: Stage 2.

Test-retest repeatability of a deep learning architecture in detecting and segmenting clinically significant prostate cancer on apparent diffusion coefficient (ADC) maps.

OBJECTIVES: To evaluate short-term test-retest repeatability of a deep learning architecture (U-Net) in slice- and lesion-level detection and segmentation of clinically significant prostate cancer (csPCa: Gleason grade group > 1) using diffusion-weighted imaging fitted with monoexponential function, ADCm. METHODS: One hundred twelve patients with prostate cancer (PCa) underwent 2 prostate MRI examinations on the same day. PCa areas were annotated using whole mount prostatectomy sections. Two U-Net-based convolutional neural networks were trained on three different ADCm b value settings for (a) slice- and (b) lesion-level detection and (c) segmentation of csPCa. Short-term test-retest repeatability was estimated using intra-class correlation coefficient (ICC(3,1)), proportionate agreement, and dice similarity coefficient (DSC). A 3-fold cross-validation was performed on training set (N = 78 patients) and evaluated for performance and repeatability on testing data (N = 34 patients). RESULTS: For the three ADCm b value settings, repeatability of mean ADCm of csPCa lesions was ICC(3,1) = 0.86-0.98. Two CNNs with U-Net-based architecture demonstrated ICC(3,1) in the range of 0.80-0.83, agreement of 66-72%, and DSC of 0.68-0.72 for slice- and lesion-level detection and segmentation of csPCa. Bland-Altman plots suggest that there is no systematic bias in agreement between inter-scan ground truth segmentation repeatability and segmentation repeatability of the networks. CONCLUSIONS: For the three ADCm b value settings, two CNNs with U-Net-based architecture were repeatable for the problem of detection of csPCa at the slice-level. The network repeatability in segmenting csPCa lesions is affected by inter-scan variability and ground truth segmentation repeatability and may thus improve with better inter-scan reproducibility. KEY POINTS: • For the three ADCm b value settings, two CNNs with U-Net-based architecture were repeatable for the problem of detection of csPCa at the slice-level. • The network repeatability in segmenting csPCa lesions is affected by inter-scan variability and ground truth segmentation repeatability and may thus improve with better inter-scan reproducibility.

T1 and T2 MR fingerprinting measurements of prostate cancer and prostatitis correlate with deep learning-derived estimates of epithelium, lumen, and stromal composition on corresponding whole mount histopathology.

OBJECTIVES: To explore the associations between T1 and T2 magnetic resonance fingerprinting (MRF) measurements and corresponding tissue compartment ratios (TCRs) on whole mount histopathology of prostate cancer (PCa) and prostatitis. MATERIALS AND METHODS: A retrospective, IRB-approved, HIPAA-compliant cohort consisting of 14 PCa patients who underwent 3 T multiparametric MRI along with T1 and T2 MRF maps prior to radical prostatectomy was used. Correspondences between whole mount specimens and MRI and MRF were manually established. Prostatitis, PCa, and normal peripheral zone (PZ) regions of interest (ROIs) on pathology were segmented for TCRs of epithelium, lumen, and stroma using two U-net deep learning models. Corresponding ROIs were mapped to T2-weighted MRI (T2w), apparent diffusion coefficient (ADC), and T1 and T2 MRF maps. Their correlations with TCRs were computed using Pearson's correlation coefficient (R). Statistically significant differences in means were assessed using one-way ANOVA. RESULTS: Statistically significant differences (p < 0.01) in means of TCRs and T1 and T2 MRF were observed between PCa, prostatitis, and normal PZ. A negative correlation was observed between T1 and T2 MRF and epithelium (R = - 0.38, - 0.44, p < 0.05) of PCa. T1 MRF was correlated in opposite directions with stroma of PCa and prostatitis (R = 0.35, - 0.44, p < 0.05). T2 MRF was positively correlated with lumen of PCa and prostatitis (R = 0.57, 0.46, p < 0.01). Mean T2 MRF showed significant differences (p < 0.01) between PCa and prostatitis across both transition zone (TZ) and PZ, while mean T1 MRF was significant (p = 0.02) in TZ. CONCLUSION: Significant associations between MRF (T1 in the TZ and T2 in the PZ) and tissue compartments on corresponding histopathology were observed. KEY POINTS: • Mean T2 MRF measurements and ADC within cancerous regions of interest dropped with increasing ISUP prognostic groups (IPG). • Mean T1 and T2 MRF measurements were significantly different (p < 0.001) across IPGs, prostatitis, and normal peripheral zone (NPZ). • T2 MRF showed stronger correlations in the peripheral zone, while T1 MRF showed stronger correlations in the transition zone with histopathology for prostate cancer.

A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises.

Since its renaissance, deep learning has been widely used in various medical imaging tasks and has achieved remarkable success in many medical imaging applications, thereby propelling us into the so-called artificial intelligence (AI) era. It is known that the success of AI is mostly attributed to the availability of big data with annotations for a single task and the advances in high performance computing. However, medical imaging presents unique challenges that confront deep learning approaches. In this survey paper, we first present traits of medical imaging, highlight both clinical needs and technical challenges in medical imaging, and describe how emerging trends in deep learning are addressing these issues. We cover the topics of network architecture, sparse and noisy labels, federating learning, interpretability, uncertainty quantification, etc. Then, we present several case studies that are commonly found in clinical practice, including digital pathology and chest, brain, cardiovascular, and abdominal imaging. Rather than presenting an exhaustive literature survey, we instead describe some prominent research highlights related to these case study applications. We conclude with a discussion and presentation of promising future directions.

A new machine learning approach for predicting likelihood of recurrence following ablation for atrial fibrillation from CT.

OBJECTIVE: To investigate left atrial shape differences on CT scans of atrial fibrillation (AF) patients with (AF+) versus without (AF-) post-ablation recurrence and whether these shape differences predict AF recurrence. METHODS: This retrospective study included 68 AF patients who had pre-catheter ablation cardiac CT scans with contrast. AF recurrence was defined at 1 year, excluding a 3-month post-ablation blanking period. After creating atlases of atrial models from segmented AF+ and AF- CT images, an atlas-based implicit shape differentiation method was used to identify surface of interest (SOI). After registering the SOI to each patient model, statistics of the deformation on the SOI were used to create shape descriptors. The performance in predicting AF recurrence using shape features at and outside the SOI and eight clinical factors (age, sex, left atrial volume, left ventricular ejection fraction, body mass index, sinus rhythm, and AF type [persistent vs paroxysmal], catheter-ablation type [Cryoablation vs Irrigated RF]) were compared using 100 runs of fivefold cross validation. RESULTS: Differences in atrial shape were found surrounding the pulmonary vein ostia and the base of the left atrial appendage. In the prediction of AF recurrence, the area under the receiver-operating characteristics curve (AUC) was 0.67 for shape features from the SOI, 0.58 for shape features outside the SOI, 0.71 for the clinical parameters, and 0.78 combining shape and clinical features. CONCLUSION: Differences in left atrial shape were identified between AF recurrent and non-recurrent patients using pre-procedure CT scans. New radiomic features corresponding to the differences in shape were found to predict post-ablation AF recurrence.

Repeatability of radiomics and machine learning for DWI: Short-term repeatability study of 112 patients with prostate cancer.

PURPOSE: To evaluate repeatability of prostate DWI-derived radiomics and machine learning methods for prostate cancer (PCa) characterization. METHODS: A total of 112 patients with diagnosed PCa underwent 2 prostate MRI examinations (Scan1 and Scan2) performed on the same day. DWI was performed using 12 b-values (0-2000 s/mm2 ), post-processed using kurtosis function, and PCa areas were annotated using whole mount prostatectomy sections. A total of 1694 radiomic features including Sobel, Kirch, Gradient, Zernike Moments, Gabor, Haralick, CoLIAGe, Haar wavelet coefficients, 3D analogue to Laws features, 2D contours, and corner detectors were calculated. Radiomics and 4 feature pruning methods (area under the receiver operator characteristic curve, maximum relevance minimum redundancy, Spearman's ρ, Wilcoxon rank-sum) were evaluated in terms of Scan1-Scan2 repeatability using intraclass correlation coefficient (ICC)(3,1). Classification performance for clinically significant and insignificant PCa with Gleason grade groups 1 versus >1 was evaluated by area under the receiver operator characteristic curve in unseen random 30% data split. RESULTS: The ICC(3,1) values for conventional radiomics and feature pruning methods were in the range of 0.28-0.90. The machine learning classifications varied between Scan1 and Scan2 with % of same class labels between Scan1 and Scan2 in the range of 61-81%. Surface-to-volume ratio and corner detector-based features were among the most represented features with high repeatability, ICC(3,1) >0.75, consistently high ranking using all 4 feature pruning methods, and classification performance with area under the receiver operator characteristic curve >0.70. CONCLUSION: Surface-to-volume ratio and corner detectors for prostate DWI led to good classification of unseen data and performed similarly in Scan1 and Scan2 in contrast to multiple conventional radiomic features.

Radiomic Features of Primary Rectal Cancers on Baseline T2 -Weighted MRI Are Associated With Pathologic Complete Response to Neoadjuvant Chemoradiation: A Multisite Study.

BACKGROUND: Twenty-five percent of rectal adenocarcinoma patients achieve pathologic complete response (pCR) to neoadjuvant chemoradiation and could avoid proctectomy. However, pretreatment clinical or imaging markers are lacking in predicting response to chemoradiation. Radiomic texture features from MRI have recently been associated with therapeutic response in other cancers. PURPOSE: To construct a radiomics texture model based on pretreatment MRI for identifying patients who will achieve pCR to neoadjuvant chemoradiation in rectal cancer, including validation across multiple scanners and sites. STUDY TYPE: Retrospective. SUBJECTS: In all, 104 rectal cancer patients staged with MRI prior to long-course chemoradiation followed by proctectomy; curated from three institutions. FIELD STRENGTH/SEQUENCE: 1.5T-3.0T, axial higher resolution T2 -weighted turbo spin echo sequence. ASSESSMENT: Pathologic response was graded on postsurgical specimens. In total, 764 radiomic features were extracted from single-slice sections of rectal tumors on processed pretreatment T2 -weighted MRI. STATISTICAL TESTS: Three feature selection schemes were compared for identifying radiomic texture descriptors associated with pCR via a discovery cohort (one site, N = 60, cross-validation). The top-selected radiomic texture features were used to train and validate a random forest classifier model for pretreatment identification of pCR (two external sites, N = 44). Model performance was evaluated via area under the curve (AUC), accuracy, sensitivity, and specificity. RESULTS: Laws kernel responses and gradient organization features were most associated with pCR (P ≤ 0.01); as well as being commonly identified across all feature selection schemes. The radiomics model yielded a discovery AUC of 0.699 ± 0.076 and a hold-out validation AUC of 0.712 with 70.5% accuracy (70.0% sensitivity, 70.6% specificity) in identifying pCR. Radiomic texture features were resilient to variations in magnetic field strength as well as being consistent between two different expert annotations. Univariate analysis revealed no significant associations of baseline clinicopathologic or MRI findings with pCR (P = 0.07-0.96). DATA CONCLUSION: Radiomic texture features from pretreatment MRIs may enable early identification of potential pCR to neoadjuvant chemoradiation, as well as generalize across sites. LEVEL OF EVIDENCE: 3 TECHNICAL EFFICACY STAGE: 2.

Quantitative nuclear histomorphometric features are predictive of Oncotype DX risk categories in ductal carcinoma in situ: preliminary findings.

BACKGROUND: Oncotype DX (ODx) is a 12-gene assay assessing the recurrence risk (high, intermediate, and low) of ductal carcinoma in situ (pre-invasive breast cancer), which guides clinicians regarding prescription of radiotherapy. However, ODx is expensive, time-consuming, and tissue-destructive. In addition, the actual prognostic meaning for the intermediate ODx risk category remains unclear. METHODS: In this work, we evaluated the ability of quantitative nuclear histomorphometric features extracted from hematoxylin and eosin-stained slide images of 62 ductal carcinoma in situ (DCIS) patients to distinguish between the corresponding ODx risk categories. The prognostic value of the identified image signature was further evaluated on an independent validation set of 30 DCIS patients in its ability to distinguish those DCIS patients who progressed to invasive carcinoma versus those who did not. Following nuclear segmentation and feature extraction, feature ranking strategies were employed to identify the most discriminating features between individual ODx risk categories. The selected features were then combined with machine learning classifiers to establish models to predict ODx risk categories. The model performance was evaluated using the average area under the receiver operating characteristic curve (AUC) using cross validation. In addition, an unsupervised clustering approach was also implemented to evaluate the ability of nuclear histomorphometric features to discriminate between the ODx risk categories. RESULTS: Features relating to spatial distribution, orientation disorder, and texture of nuclei were identified as most discriminating between the high ODx and the intermediate, low ODx risk categories. Additionally, the AUC of the most discriminating set of features for the different classification tasks was as follows: (1) high vs low ODx (0.68), (2) high vs. intermediate ODx (0.67), (3) intermediate vs. low ODx (0.57), (4) high and intermediate vs. low ODx (0.63), (5) high vs. low and intermediate ODx (0.66). Additionally, the unsupervised clustering resulted in intermediate ODx risk category patients being co-clustered with low ODx patients compared to high ODx. CONCLUSION: Our results appear to suggest that nuclear histomorphometric features can distinguish high from low and intermediate ODx risk category patients. Additionally, our findings suggest that histomorphometric features for intermediate ODx were more similar to low ODx compared to high ODx risk category.

Perinodular and Intranodular Radiomic Features on Lung CT Images Distinguish Adenocarcinomas from Granulomas.

Purpose To evaluate ability of radiomic (computer-extracted imaging) features to distinguish non-small cell lung cancer adenocarcinomas from granulomas at noncontrast CT. Materials and Methods For this retrospective study, screening or standard diagnostic noncontrast CT images were collected for 290 patients (mean age, 68 years; range, 18-92 years; 125 men [mean age, 67 years; range, 18-90 years] and 165 women [mean age, 68 years; range, 33-92 years]) from two institutions between 2007 and 2013. Histopathologic analysis was available for one nodule per patient. Corresponding nodule of interest was identified on axial CT images by a radiologist with manual annotation. Nodule shape, wavelet (Gabor), and texture-based (Haralick and Laws energy) features were extracted from intra- and perinodular regions. Features were pruned to train machine learning classifiers with 145 patients. In a test set of 145 patients, classifier results were compared against a convolutional neural network (CNN) and diagnostic readings of two radiologists. Results Support vector machine classifier with intranodular radiomic features achieved an area under the receiver operating characteristic curve (AUC) of 0.75 on the test set. Combining radiomics of intranodular with perinodular regions improved the AUC to 0.80. On the same test set, CNN resulted in an AUC of 0.76. Radiologist readers achieved AUCs of 0.61 and 0.60, respectively. Conclusion Radiomic features from intranodular and perinodular regions of nodules can distinguish non-small cell lung cancer adenocarcinomas from benign granulomas at noncontrast CT. © RSNA, 2018 Online supplemental material is available for this article. See also the editorial by Nishino in this issue.