RAPHIA: A deep learning pipeline for the registration of MRI and whole-mount histopathology images of the prostate.

Image registration can map the ground truth extent of prostate cancer from histopathology images onto MRI, facilitating the development of machine learning methods for early prostate cancer detection. Here, we present RAdiology PatHology Image Alignment (RAPHIA), an end-to-end pipeline for efficient and accurate registration of MRI and histopathology images. RAPHIA automates several time-consuming manual steps in existing approaches including prostate segmentation, estimation of the rotation angle and horizontal flipping in histopathology images, and estimation of MRI-histopathology slice correspondences. By utilizing deep learning registration networks, RAPHIA substantially reduces computational time. Furthermore, RAPHIA obviates the need for a multimodal image similarity metric by transferring histopathology image representations to MRI image representations and vice versa. With the assistance of RAPHIA, novice users achieved expert-level performance, and their mean error in estimating histopathology rotation angle was reduced by 51% (12 degrees vs 8 degrees), their mean accuracy of estimating histopathology flipping was increased by 5% (95.3% vs 100%), and their mean error in estimating MRI-histopathology slice correspondences was reduced by 45% (1.12 slices vs 0.62 slices). When compared to a recent conventional registration approach and a deep learning registration approach, RAPHIA achieved better mapping of histopathology cancer labels, with an improved mean Dice coefficient of cancer regions outlined on MRI and the deformed histopathology (0.44 vs 0.48 vs 0.50), and a reduced mean per-case processing time (51 vs 11 vs 4.5 min). The improved performance by RAPHIA allows efficient processing of large datasets for the development of machine learning models for prostate cancer detection on MRI. Our code is publicly available at: https://github.com/pimed/RAPHIA.

External Validation of a Digital Pathology-based Multimodal Artificial Intelligence Architecture in the NRG/RTOG 9902 Phase 3 Trial.

BACKGROUND: Accurate risk stratification is critical to guide management decisions in localized prostate cancer (PCa). Previously, we had developed and validated a multimodal artificial intelligence (MMAI) model generated from digital histopathology and clinical features. Here, we externally validate this model on men with high-risk or locally advanced PCa treated and followed as part of a phase 3 randomized control trial. OBJECTIVE: To externally validate the MMAI model on men with high-risk or locally advanced PCa treated and followed as part of a phase 3 randomized control trial. DESIGN, SETTING, AND PARTICIPANTS: Our validation cohort included 318 localized high-risk PCa patients from NRG/RTOG 9902 with available histopathology (337 [85%] of the 397 patients enrolled into the trial had available slides, of which 19 [5.6%] failed due to poor image quality). OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: Two previously locked prognostic MMAI models were validated for their intended endpoint: distant metastasis (DM) and PCa-specific mortality (PCSM). Individual clinical factors and the number of National Comprehensive Cancer Network (NCCN) high-risk features served as comparators. Subdistribution hazard ratio (sHR) was reported per standard deviation increase of the score with corresponding 95% confidence interval (CI) using Fine-Gray or Cox proportional hazards models. RESULTS AND LIMITATIONS: The DM and PCSM MMAI algorithms were significantly and independently associated with the risk of DM (sHR [95% CI] = 2.33 [1.60-3.38], p < 0.001) and PCSM, respectively (sHR [95% CI] = 3.54 [2.38-5.28], p < 0.001) when compared against other prognostic clinical factors and NCCN high-risk features. The lower 75% of patients by DM MMAI had estimated 5- and 10-yr DM rates of 4% and 7%, and the highest quartile had average 5- and 10-yr DM rates of 19% and 32%, respectively (p < 0.001). Similar results were observed for the PCSM MMAI algorithm. CONCLUSIONS: We externally validated the prognostic ability of MMAI models previously developed among men with localized high-risk disease. MMAI prognostic models further risk stratify beyond the clinical and pathological variables for DM and PCSM in a population of men already at a high risk for disease progression. This study provides evidence for consistent validation of our deep learning MMAI models to improve prognostication and enable more informed decision-making for patient care. PATIENT SUMMARY: This paper presents a novel approach using images from pathology slides along with clinical variables to validate artificial intelligence (computer-generated) prognostic models. When implemented, clinicians can offer a more personalized and tailored prognostic discussion for men with localized prostate cancer.

Artificial Intelligence Predictive Model for Hormone Therapy Use in Prostate Cancer.

Background: Androgen deprivation therapy (ADT) with radiotherapy can benefit patients with localized prostate cancer. However, ADT can negatively impact quality of life and there remain no validated predictive models to guide its use. Methods: Digital pathology image and clinical data from pre-treatment prostate tissue from 5,727 patients enrolled on five phase III randomized trials treated with radiotherapy +/- ADT were used to develop and validate an artificial intelligence (AI)-derived predictive model to assess ADT benefit with the primary endpoint of distant metastasis. After the model was locked, validation was performed on NRG/RTOG 9408 (n = 1,594) that randomized men to radiotherapy +/- 4 months of ADT. Fine-Gray regression and restricted mean survival times were used to assess the interaction between treatment and predictive model and within predictive model positive and negative subgroup treatment effects. Results: In the NRG/RTOG 9408 validation cohort (14.9 years of median follow-up), ADT significantly improved time to distant metastasis (subdistribution hazard ratio [sHR] = 0.64, 95%CI [0.45-0.90], p = 0.01). The predictive model-treatment interaction was significant (p-interaction = 0.01). In predictive model positive patients (n = 543, 34%), ADT significantly reduced the risk of distant metastasis compared to radiotherapy alone (sHR = 0.34, 95%CI [0.19-0.63], p < 0.001). There were no significant differences between treatment arms in the predictive model negative subgroup (n = 1,051, 66%; sHR = 0.92, 95%CI [0.59-1.43], p = 0.71). Conclusions: Our data, derived and validated from completed randomized phase III trials, show that an AI-based predictive model was able to identify prostate cancer patients, with predominately intermediate-risk disease, who are likely to benefit from short-term ADT.

Contrastive learning-based pretraining improves representation and transferability of diabetic retinopathy classification models.

Diabetic retinopathy (DR) is a major cause of vision impairment in diabetic patients worldwide. Due to its prevalence, early clinical diagnosis is essential to improve treatment management of DR patients. Despite recent demonstration of successful machine learning (ML) models for automated DR detection, there is a significant clinical need for robust models that can be trained with smaller cohorts of dataset and still perform with high diagnostic accuracy in independent clinical datasets (i.e., high model generalizability). Towards this need, we have developed a self-supervised contrastive learning (CL) based pipeline for classification of referable vs non-referable DR. Self-supervised CL based pretraining allows enhanced data representation, therefore, the development of robust and generalized deep learning (DL) models, even with small, labeled datasets. We have integrated a neural style transfer (NST) augmentation in the CL pipeline to produce models with better representations and initializations for the detection of DR in color fundus images. We compare our CL pretrained model performance with two state of the art baseline models pretrained with Imagenet weights. We further investigate the model performance with reduced labeled training data (down to 10 percent) to test the robustness of the model when trained with small, labeled datasets. The model is trained and validated on the EyePACS dataset and tested independently on clinical datasets from the University of Illinois, Chicago (UIC). Compared to baseline models, our CL pretrained FundusNet model had higher area under the receiver operating characteristics (ROC) curve (AUC) (CI) values (0.91 (0.898 to 0.930) vs 0.80 (0.783 to 0.820) and 0.83 (0.801 to 0.853) on UIC data). At 10 percent labeled training data, the FundusNet AUC was 0.81 (0.78 to 0.84) vs 0.58 (0.56 to 0.64) and 0.63 (0.60 to 0.66) in baseline models, when tested on the UIC dataset. CL based pretraining with NST significantly improves DL classification performance, helps the model generalize well (transferable from EyePACS to UIC data), and allows training with small, annotated datasets, therefore reducing ground truth annotation burden of the clinicians.

Differentiation of benign from malignant solid renal lesions using CT-based radiomics and machine learning: comparison with radiologist interpretation.

PURPOSE: To assess the performance of a machine learning model trained with contrast-enhanced CT-based radiomics features in distinguishing benign from malignant solid renal masses and to compare model performance with three abdominal radiologists. METHODS: Patients who underwent intra-operative ultrasound during a partial nephrectomy were identified within our institutional database, and those who had pre-operative contrast-enhanced CT examinations were selected. The renal masses were segmented from the CT images and radiomics features were derived from the segmentations. The pathology of each mass was identified; masses were labeled as either benign [oncocytoma or angiomyolipoma (AML)] or malignant [clear cell, papillary, or chromophobe renal cell carcinoma (RCC)] depending on the pathology. The data were parsed into a 70/30 train/test split and a random forest machine learning model was developed to distinguish benign from malignant lesions. Three radiologists assessed the cohort of masses and labeled cases as benign or malignant. RESULTS: 148 masses were identified from the cohort, including 50 benign lesions (23 AMLs, 27 oncocytomas) and 98 malignant lesions (23 clear cell RCC, 44 papillary RCC, and 31 chromophobe RCCs). The machine learning algorithm yielded an overall accuracy of 0.82 for distinguishing benign from malignant lesions, with an area under the receiver operating curve of 0.80. In comparison, the three radiologists had significantly lower accuracies (p = 0.02) ranging from 0.67 to 0.75. CONCLUSION: A machine learning model trained with CT-based radiomics features can provide superior accuracy for distinguishing benign from malignant solid renal masses compared to abdominal radiologists.

Artificial Intelligence Predictive Model for Hormone Therapy Use in Prostate Cancer.

BACKGROUND: Androgen deprivation therapy (ADT) with radiotherapy can benefit patients with localized prostate cancer. However, ADT can negatively impact quality of life, and there remain no validated predictive models to guide its use. METHODS: We used digital pathology images from pretreatment prostate tissue and clinical data from 5727 patients enrolled in five phase 3 randomized trials, in which treatment was radiotherapy with or without ADT, as our data source to develop and validate an artificial intelligence (AI)­derived predictive patient-specific model that would determine which patients would develop the primary end point of distant metastasis. The model used baseline data to provide a binary output that a given patient will likely benefit from ADT or not. After the model was locked, validation was performed using data from NRG Oncology/Radiation Therapy Oncology Group (RTOG) 9408 (n=1594), a trial that randomly assigned men to radiotherapy plus or minus 4 months of ADT. Fine­Gray regression and restricted mean survival times were used to assess the interaction between treatment and the predictive model and within predictive model­positive, i.e., benefited from ADT, and ­negative subgroup treatment effects. RESULTS: Overall, in the NRG/RTOG 9408 validation cohort (14.9 years of median follow-up), ADT significantly improved time to distant metastasis. Of these enrolled patients, 543 (34%) were model positive, and ADT significantly reduced the risk of distant metastasis compared with radiotherapy alone. Of 1051 patients who were model negative, ADT did not provide benefit. CONCLUSIONS: Our AI-based predictive model was able to identify patients with a predominantly intermediate risk for prostate cancer likely to benefit from short-term ADT. (Supported by a grant [U10CA180822] from NRG Oncology Statistical and Data Management Center, a grant [UG1CA189867] from NCI Community Oncology Research Program, a grant [U10CA180868] from NRG Oncology Operations, and a grant [U24CA196067] from NRG Specimen Bank from the National Cancer Institute and by Artera, Inc. ClinicalTrials.gov numbers NCT00767286, NCT00002597, NCT00769548, NCT00005044, and NCT00033631.)

Toward Reduction in False-Positive Thyroid Nodule Biopsies with a Deep Learning-based Risk Stratification System Using US Cine-Clip Images.

Purpose: To develop a deep learning-based risk stratification system for thyroid nodules using US cine images. Materials and Methods: In this retrospective study, 192 biopsy-confirmed thyroid nodules (175 benign, 17 malignant) in 167 unique patients (mean age, 56 years ± 16 [SD], 137 women) undergoing cine US between April 2017 and May 2018 with American College of Radiology (ACR) Thyroid Imaging Reporting and Data System (TI-RADS)-structured radiology reports were evaluated. A deep learning-based system that exploits the cine images obtained during three-dimensional volumetric thyroid scans and outputs malignancy risk was developed and compared, using fivefold cross-validation, against a two-dimensional (2D) deep learning-based model (Static-2DCNN), a radiomics-based model using cine images (Cine-Radiomics), and the ACR TI-RADS level, with histopathologic diagnosis as ground truth. The system was used to revise the ACR TI-RADS recommendation, and its diagnostic performance was compared against the original ACR TI-RADS. Results: The system achieved higher average area under the receiver operating characteristic curve (AUC, 0.88) than Static-2DCNN (0.72, P = .03) and tended toward higher average AUC than Cine-Radiomics (0.78, P = .16) and ACR TI-RADS level (0.80, P = .21). The system downgraded recommendations for 92 benign and two malignant nodules and upgraded none. The revised recommendation achieved higher specificity (139 of 175, 79.4%) than the original ACR TI-RADS (47 of 175, 26.9%; P < .001), with no difference in sensitivity (12 of 17, 71% and 14 of 17, 82%, respectively; P = .63). Conclusion: The risk stratification system using US cine images had higher diagnostic performance than prior models and improved specificity of ACR TI-RADS when used to revise ACR TI-RADS recommendation.Keywords: Neural Networks, US, Abdomen/GI, Head/Neck, Thyroid, Computer Applications-3D, Oncology, Diagnosis, Supervised Learning, Transfer Learning, Convolutional Neural Network (CNN) Supplemental material is available for this article. © RSNA, 2022.

Automated Identification and Measurement Extraction of Pancreatic Cystic Lesions from Free-Text Radiology Reports Using Natural Language Processing.

Purpose: To automatically identify a cohort of patients with pancreatic cystic lesions (PCLs) and extract PCL measurements from historical CT and MRI reports using natural language processing (NLP) and a question answering system. Materials and Methods: Institutional review board approval was obtained for this retrospective Health Insurance Portability and Accountability Act-compliant study, and the requirement to obtain informed consent was waived. A cohort of free-text CT and MRI reports generated between January 1991 and July 2019 that covered the pancreatic region were identified. A PCL identification model was developed by modifying a rule-based information extraction model; measurement extraction was performed using a state-of-the-art question answering system. The system's performance was evaluated against radiologists' annotations. Results: For this study, 430 426 free-text radiology reports from 199 783 unique patients were identified. The NLP model for identifying PCL was applied to 1000 test samples. The interobserver agreement between the model and two radiologists was almost perfect (Fleiss κ = 0.951), and the false-positive rate and true-positive rate were 3.0% and 98.2%, respectively, against consensus of radiologists' annotations as ground truths. The overall accuracy and Lin concordance correlation coefficient for measurement extraction were 0.958 and 0.874, respectively, against radiologists' annotations as ground truths. Conclusion: An NLP-based system was developed that identifies patients with PCLs and extracts measurements from a large single-institution archive of free-text radiology reports. This approach may prove valuable to study the natural history and potential risks of PCLs and can be applied to many other use cases.Keywords: Informatics, Abdomen/GI, Pancreas, Cysts, Computer Applications-General (Informatics), Named Entity Recognition Supplemental material is available for this article. © RSNA, 2022See also commentary by Horii in this issue.

Deep Learning: An Update for Radiologists.

Deep learning is a class of machine learning methods that has been successful in computer vision. Unlike traditional machine learning methods that require hand-engineered feature extraction from input images, deep learning methods learn the image features by which to classify data. Convolutional neural networks (CNNs), the core of deep learning methods for imaging, are multilayered artificial neural networks with weighted connections between neurons that are iteratively adjusted through repeated exposure to training data. These networks have numerous applications in radiology, particularly in image classification, object detection, semantic segmentation, and instance segmentation. The authors provide an update on a recent primer on deep learning for radiologists, and they review terminology, data requirements, and recent trends in the design of CNNs; illustrate building blocks and architectures adapted to computer vision tasks, including generative architectures; and discuss training and validation, performance metrics, visualization, and future directions. Familiarity with the key concepts described will help radiologists understand advances of deep learning in medical imaging and facilitate clinical adoption of these techniques. Online supplemental material is available for this article. ©RSNA, 2021.

Learning Domain-Agnostic Visual Representation for Computational Pathology Using Medically-Irrelevant Style Transfer Augmentation.

Suboptimal generalization of machine learning models on unseen data is a key challenge which hampers the clinical applicability of such models to medical imaging. Although various methods such as domain adaptation and domain generalization have evolved to combat this challenge, learning robust and generalizable representations is core to medical image understanding, and continues to be a problem. Here, we propose STRAP (Style TRansfer Augmentation for histoPathology), a form of data augmentation based on random style transfer from non-medical style sources such as artistic paintings, for learning domain-agnostic visual representations in computational pathology. Style transfer replaces the low-level texture content of an image with the uninformative style of randomly selected style source image, while preserving the original high-level semantic content. This improves robustness to domain shift and can be used as a simple yet powerful tool for learning domain-agnostic representations. We demonstrate that STRAP leads to state-of-the-art performance, particularly in the presence of domain shifts, on two particular classification tasks in computational pathology. Our code is available at https://github.com/rikiyay/style-transfer-for-digital-pathology.

Multiplexed imaging analysis of the tumor-immune microenvironment reveals predictors of outcome in triple-negative breast cancer.

Triple-negative breast cancer, the poorest-prognosis breast cancer subtype, lacks clinically approved biomarkers for patient risk stratification and treatment management. Prior literature has shown that interrogation of the tumor-immune microenvironment may be a promising approach to fill these gaps. Recently developed high-dimensional tissue imaging technology, such as multiplexed ion beam imaging, provide spatial context to protein expression in the microenvironment, allowing in-depth characterization of cellular processes. We demonstrate that profiling the functional proteins involved in cell-to-cell interactions in the microenvironment can predict recurrence and overall survival. We highlight the immunological relevance of the immunoregulatory proteins PD-1, PD-L1, IDO, and Lag3 by tying interactions involving them to recurrence and survival. Multivariate analysis reveals that our methods provide additional prognostic information compared to clinical variables. In this work, we present a computational pipeline for the examination of the tumor-immune microenvironment using multiplexed ion beam imaging that produces interpretable results, and is generalizable to other cancer types.

Quantitative Computed Tomography Image Analysis to Predict Pancreatic Neuroendocrine Tumor Grade.

PURPOSE: The therapeutic management of pancreatic neuroendocrine tumors (PanNETs) is based on pathological tumor grade assessment. A noninvasive imaging method to grade tumors would facilitate treatment selection. This study evaluated the ability of quantitative image analysis derived from computed tomography (CT) images to predict PanNET grade. METHODS: Institutional database was queried for resected PanNET (2000-2017) with a preoperative arterial phase CT scan. Radiomic features were extracted from the primary tumor on the CT scan using quantitative image analysis, and qualitative radiographic descriptors were assessed by two radiologists. Significant features were identified by univariable analysis and used to build multivariable models to predict PanNET grade. RESULTS: Overall, 150 patients were included. The performance of models based on qualitative radiographic descriptors varied between the two radiologists (reader 1: sensitivity, 33%; specificity, 66%; negative predictive value [NPV], 63%; and positive predictive value [PPV], 37%; reader 2: sensitivity, 45%; specificity, 70%; NPV, 72%; and PPV, 47%). The model based on radiomics had a better performance predicting the tumor grade with a sensitivity of 54%, a specificity of 80%, an NPV of 81%, and a PPV of 54%. The inclusion of radiomics in the radiographic descriptor models improved both the radiologists' performance. CONCLUSION: CT quantitative image analysis of PanNETs helps predict tumor grade from routinely acquired scans and should be investigated in future prospective studies.

Data valuation for medical imaging using Shapley value and application to a large-scale chest X-ray dataset.

The reliability of machine learning models can be compromised when trained on low quality data. Many large-scale medical imaging datasets contain low quality labels extracted from sources such as medical reports. Moreover, images within a dataset may have heterogeneous quality due to artifacts and biases arising from equipment or measurement errors. Therefore, algorithms that can automatically identify low quality data are highly desired. In this study, we used data Shapley, a data valuation metric, to quantify the value of training data to the performance of a pneumonia detection algorithm in a large chest X-ray dataset. We characterized the effectiveness of data Shapley in identifying low quality versus valuable data for pneumonia detection. We found that removing training data with high Shapley values decreased the pneumonia detection performance, whereas removing data with low Shapley values improved the model performance. Furthermore, there were more mislabeled examples in low Shapley value data and more true pneumonia cases in high Shapley value data. Our results suggest that low Shapley value indicates mislabeled or poor quality images, whereas high Shapley value indicates data that are valuable for pneumonia detection. Our method can serve as a framework for using data Shapley to denoise large-scale medical imaging datasets.

Development and Use of Natural Language Processing for Identification of Distant Cancer Recurrence and Sites of Distant Recurrence Using Unstructured Electronic Health Record Data.

PURPOSE: Large-scale analysis of real-world evidence is often limited to structured data fields that do not contain reliable information on recurrence status and disease sites. In this report, we describe a natural language processing (NLP) framework that uses data from free-text, unstructured reports to classify recurrence status and sites of recurrence for patients with breast and hepatocellular carcinomas (HCC). METHODS: Using two cohorts of breast cancer and HCC cases, we validated the ability of a previously developed NLP model to distinguish between no recurrence, local recurrence, and distant recurrence, based on clinician notes, radiology reports, and pathology reports compared with manual curation. A second NLP model was trained and validated to identify sites of recurrence. We compared the ability of each NLP model to identify the presence, timing, and site of recurrence, when compared against manual chart review and International Classification of Diseases coding. RESULTS: A total of 1,273 patients were included in the development and validation of the two models. The NLP model for recurrence detects distant recurrence with an area under the curve of 0.98 (95% CI, 0.96 to 0.99) and 0.95 (95% CI, 0.88 to 0.98) in breast and HCC cohorts, respectively. The mean accuracy of the NLP model for detecting any site of distant recurrence was 0.9 for breast cancer and 0.83 for HCC. The NLP model for recurrence identified a larger proportion of patients with distant recurrence in a breast cancer database (11.1%) compared with International Classification of Diseases coding (2.31%). CONCLUSION: We developed two NLP models to identify distant cancer recurrence, timing of recurrence, and sites of recurrence based on unstructured electronic health record data. These models can be used to perform large-scale retrospective studies in oncology.

Convolutional neural network for classifying primary liver cancer based on triple-phase CT and tumor marker information: a pilot study.

PURPOSE: To develop convolutional neural network (CNN) models for differentiating intrahepatic cholangiocarcinoma (ICC) from hepatocellular carcinoma (HCC) and predicting histopathological grade of HCC. MATERIALS AND METHODS: Preoperative computed tomography and tumor marker information of 617 primary liver cancer patients were retrospectively collected to develop CNN models categorizing tumors into three categories: moderately differentiated HCC (mHCC), poorly differentiated HCC (pHCC), and ICC, where the histopathological diagnoses were considered as ground truths. The models processed manually cropped tumor with and without tumor marker information (two-input and one-input models, respectively). Overall accuracy was assessed using a held-out dataset (10%). Area under the curve, sensitivity, and specificity for differentiating ICC from HCCs (mHCC + pHCC), and pHCC from mHCC were also evaluated. We assessed two radiologists' performance without tumor marker information as references (overall accuracy, sensitivity, and specificity). The two-input model was compared with the one-input model and radiologists using permutation tests. RESULTS: The overall accuracy was 0.61, 0.60, 0.55, 0.53 for the two-input model, one-input model, radiologist 1, and radiologist 2, respectively. For differentiating pHCC from mHCC, the two-input model showed significantly higher specificity than radiologist 1 (0.68 [95% confidence interval: 0.50-0.83] vs 0.45 [95% confidence interval: 0.27-0.63]; p = 0.04). CONCLUSION: Our CNN model with tumor marker information showed feasibility and potential for three-class classification within primary liver cancer.

Deep learning-based intravascular ultrasound segmentation for the assessment of coronary artery disease.

BACKGROUND: Accurate segmentation of the coronary arteries with intravascular ultrasound (IVUS) is important to optimize coronary stent implantation. Recently, deep learning (DL) methods have been proposed to develop automatic IVUS segmentation. However, most of those have been limited to segmenting the lumen and vessel (i.e. lumen-intima and media-adventitia borders), not applied to segmenting stent dimension. Hence, this study aimed to develop a DL method for automatic IVUS segmentation of stent area in addition to lumen and vessel area. METHODS: This study included a total of 45,449 images from 1576 IVUS pullback runs. The datasets were randomly split into training, validation, and test datasets (0.7:0.15:0.15). After developing the DL-based system to segment IVUS images using the training and validation datasets, we evaluated the performance through the independent test dataset. RESULTS: The DL-based segmentation correlated well with the expert-analyzed segmentation with a mean intersection over union (± standard deviation) of 0.80 ± 0.20, correlation coefficient of 0.98 (95% confidence intervals: 0.98 to 0.98), 0.96 (0.95 to 0.96), and 0.96 (0.96 to 0.96) for lumen, vessel, and stent area, and the mean difference (± standard deviation) of 0.02 ± 0.57, -0.44 ± 1.56 and - 0.17 ± 0.74 mm2 for lumen, vessel and stent area, respectively. CONCLUSION: This automated DL-based IVUS segmentation of lumen, vessel and stent area showed an excellent agreement with manual segmentation by experts, supporting the feasibility of artificial intelligence-assisted IVUS assessment in patients undergoing coronary stent implantation.

Deep learning predicts postsurgical recurrence of hepatocellular carcinoma from digital histopathologic images.

Recurrence risk stratification of patients undergoing primary surgical resection for hepatocellular carcinoma (HCC) is an area of active investigation, and several staging systems have been proposed to optimize treatment strategies. However, as many as 70% of patients still experience tumor recurrence at 5 years post-surgery. We developed and validated a deep learning-based system (HCC-SurvNet) that provides risk scores for disease recurrence after primary resection, directly from hematoxylin and eosin-stained digital whole-slide images of formalin-fixed, paraffin embedded liver resections. Our model achieved concordance indices of 0.724 and 0.683 on the internal and external test cohorts, respectively, exceeding the performance of the standard Tumor-Node-Metastasis classification system. The model's risk score stratified patients into low- and high-risk subgroups with statistically significant differences in their survival distributions, and was an independent risk factor for post-surgical recurrence in both test cohorts. Our results suggest that deep learning-based models can provide recurrence risk scores which may augment current patient stratification methods and help refine the clinical management of patients undergoing primary surgical resection for HCC.

Deep learning model for the prediction of microsatellite instability in colorectal cancer: a diagnostic study.

BACKGROUND: Detecting microsatellite instability (MSI) in colorectal cancer is crucial for clinical decision making, as it identifies patients with differential treatment response and prognosis. Universal MSI testing is recommended, but many patients remain untested. A critical need exists for broadly accessible, cost-efficient tools to aid patient selection for testing. Here, we investigate the potential of a deep learning-based system for automated MSI prediction directly from haematoxylin and eosin (H&E)-stained whole-slide images (WSIs). METHODS: Our deep learning model (MSINet) was developed using 100 H&E-stained WSIs (50 with microsatellite stability [MSS] and 50 with MSI) scanned at 40× magnification, each from a patient randomly selected in a class-balanced manner from the pool of 343 patients who underwent primary colorectal cancer resection at Stanford University Medical Center (Stanford, CA, USA; internal dataset) between Jan 1, 2015, and Dec 31, 2017. We internally validated the model on a holdout test set (15 H&E-stained WSIs from 15 patients; seven cases with MSS and eight with MSI) and externally validated the model on 484 H&E-stained WSIs (402 cases with MSS and 77 with MSI; 479 patients) from The Cancer Genome Atlas, containing WSIs scanned at 40× and 20× magnification. Performance was primarily evaluated using the sensitivity, specificity, negative predictive value (NPV), and area under the receiver operating characteristic curve (AUROC). We compared the model's performance with that of five gastrointestinal pathologists on a class-balanced, randomly selected subset of 40× magnification WSIs from the external dataset (20 with MSS and 20 with MSI). FINDINGS: The MSINet model achieved an AUROC of 0·931 (95% CI 0·771-1·000) on the holdout test set from the internal dataset and 0·779 (0·720-0·838) on the external dataset. On the external dataset, using a sensitivity-weighted operating point, the model achieved an NPV of 93·7% (95% CI 90·3-96·2), sensitivity of 76·0% (64·8-85·1), and specificity of 66·6% (61·8-71·2). On the reader experiment (40 cases), the model achieved an AUROC of 0·865 (95% CI 0·735-0·995). The mean AUROC performance of the five pathologists was 0·605 (95% CI 0·453-0·757). INTERPRETATION: Our deep learning model exceeded the performance of experienced gastrointestinal pathologists at predicting MSI on H&E-stained WSIs. Within the current universal MSI testing paradigm, such a model might contribute value as an automated screening tool to triage patients for confirmatory testing, potentially reducing the number of tested patients, thereby resulting in substantial test-related labour and cost savings. FUNDING: Stanford Cancer Institute and Stanford Departments of Pathology and Biomedical Data Science.

Quantitative and Qualitative Evaluation of Convolutional Neural Networks with a Deeper U-Net for Sparse-View Computed Tomography Reconstruction.

RATIONALE AND OBJECTIVES: To evaluate the utility of a convolutional neural network (CNN) with an increased number of contracting and expanding paths of U-net for sparse-view CT reconstruction. MATERIALS AND METHODS: This study used 60 anonymized chest CT cases from a public database called "The Cancer Imaging Archive". Eight thousand images from 40 cases were used for training. Eight hundred and 80 images from another 20 cases were used for quantitative and qualitative evaluation, respectively. Sparse-view CT images subsampled by a factor of 20 were simulated, and two CNNs were trained to create denoised images from the sparse-view CT. A CNN based on U-net with residual learning with four contracting and expanding paths (the preceding CNN) was compared with another CNN with eight contracting and expanding paths (the proposed CNN) both quantitatively (peak signal to noise ratio, structural similarity index), and qualitatively (the scores given by two radiologists for anatomical visibility, artifact and noise, and overall image quality) using the Wilcoxon signed-rank test. Nodule and emphysema appearance were also evaluated qualitatively. RESULTS: The proposed CNN was significantly better than the preceding CNN both quantitatively and qualitatively (overall image quality interquartile range, 3.0-3.5 versus 1.0-1.0 reported from the preceding CNN; p < 0.001). However, only 2 of 22 cases used for emphysematous evaluation (2 CNNs for every 11 cases with emphysema) had an average score of ≥ 2 (on a 3 point scale). CONCLUSION: Increasing contracting and expanding paths may be useful for sparse-view CT reconstruction with CNN. However, poor reproducibility of emphysema appearance should also be noted.

Radiomic feature reproducibility in contrast-enhanced CT of the pancreas is affected by variabilities in scan parameters and manual segmentation.

OBJECTIVES: This study aims to measure the reproducibility of radiomic features in pancreatic parenchyma and ductal adenocarcinomas (PDAC) in patients who underwent consecutive contrast-enhanced computed tomography (CECT) scans. METHODS: In this IRB-approved and HIPAA-compliant retrospective study, 37 pairs of scans from 37 unique patients who underwent CECTs within a 2-week interval were included in the analysis of the reproducibility of features derived from pancreatic parenchyma, and a subset of 18 pairs of scans were further analyzed for the reproducibility of features derived from PDAC. In each patient, pancreatic parenchyma and pancreatic tumor (when present) were manually segmented by two radiologists independently. A total of 266 radiomic features were extracted from the pancreatic parenchyma and tumor region and also the volume and diameter of the tumor. The concordance correlation coefficient (CCC) was calculated to assess feature reproducibility for each patient in three scenarios: (1) different radiologists, same CECT; (2) same radiologist, different CECTs; and (3) different radiologists, different CECTs. RESULTS: Among pancreatic parenchyma-derived features, using a threshold of CCC > 0.90, 58/266 (21.8%) and 48/266 (18.1%) features met the threshold for scenario 1, 14/266 (5.3%) and 15/266 (5.6%) for scenario 2, and 14/266 (5.3%) and 10/266 (3.8%) for scenario 3. Among pancreatic tumor-derived features, 11/268 (4.1%) and 17/268 (6.3%) features met the threshold for scenario 1, 1/268 (0.4%) and 5/268 (1.9%) features met the threshold for scenario 2, and no features for scenario 3 met the threshold, respectively. CONCLUSIONS: Variations between CECT scans affected radiomic feature reproducibility to a greater extent than variation in segmentation. A smaller number of pancreatic tumor-derived radiomic features were reproducible compared with pancreatic parenchyma-derived radiomic features under the same conditions. KEY POINTS: • For pancreatic-derived radiomic features from contrast-enhanced CT (CECT), fewer than 25% are reproducible (with a threshold of CCC < 0.9) in a clinical heterogeneous dataset. • Variations between CECT scans affected the number of reproducible radiomic features to a greater extent than variations in radiologist segmentation. • A smaller number of pancreatic tumor-derived radiomic features were reproducible compared with pancreatic parenchyma-derived radiomic features under the same conditions.