Role of sureness in evaluating AI/CADx: Lesion-based repeatability of machine learning classification performance on breast MRI.

BACKGROUND: Artificial intelligence/computer-aided diagnosis (AI/CADx) and its use of radiomics have shown potential in diagnosis and prognosis of breast cancer. Performance metrics such as the area under the receiver operating characteristic (ROC) curve (AUC) are frequently used as figures of merit for the evaluation of CADx. Methods for evaluating lesion-based measures of performance may enhance the assessment of AI/CADx pipelines, particularly in the situation of comparing performances by classifier. PURPOSE: The purpose of this study was to investigate the use case of two standard classifiers to (1) compare overall classification performance of the classifiers in the task of distinguishing between benign and malignant breast lesions using radiomic features extracted from dynamic contrast-enhanced magnetic resonance (DCE-MR) images, (2) define a new repeatability metric (termed sureness), and (3) use sureness to examine if one classifier provides an advantage in AI diagnostic performance by lesion when using radiomic features. METHODS: Images of 1052 breast lesions (201 benign, 851 cancers) had been retrospectively collected under HIPAA/IRB compliance. The lesions had been segmented automatically using a fuzzy c-means method and thirty-two radiomic features had been extracted. Classification was investigated for the task of malignant lesions (81% of the dataset) versus benign lesions (19%). Two classifiers (linear discriminant analysis, LDA and support vector machines, SVM) were trained and tested within 0.632 bootstrap analyses (2000 iterations). Whole-set classification performance was evaluated at two levels: (1) the 0.632+ bias-corrected area under the ROC curve (AUC) and (2) performance metric curves which give variability in operating sensitivity and specificity at a target operating point (95% target sensitivity). Sureness was defined as 1-95% confidence interval of the classifier output for each lesion for each classifier. Lesion-based repeatability was evaluated at two levels: (1) repeatability profiles, which represent the distribution of sureness across the decision threshold and (2) sureness of each lesion. The latter was used to identify lesions with better sureness with one classifier over another while maintaining lesion-based performance across the bootstrap iterations. RESULTS: In classification performance assessment, the median and 95% CI of difference in AUC between the two classifiers did not show evidence of difference (ΔAUC = -0.003 [-0.031, 0.018]). Both classifiers achieved the target sensitivity. Sureness was more consistent across the classifier output range for the SVM classifier than the LDA classifier. The SVM resulted in a net gain of 33 benign lesions and 307 cancers with higher sureness and maintained lesion-based performance. However, with the LDA there was a notable percentage of benign lesions (42%) with better sureness but lower lesion-based performance. CONCLUSIONS: When there is no evidence for difference in performance between classifiers using AUC or other performance summary measures, a lesion-based sureness metric may provide additional insight into AI pipeline design. These findings present and emphasize the utility of lesion-based repeatability via sureness in AI/CADx as a complementary enhancement to other evaluation measures.

A Competition, Benchmark, Code, and Data for Using Artificial Intelligence to Detect Lesions in Digital Breast Tomosynthesis.

Importance: An accurate and robust artificial intelligence (AI) algorithm for detecting cancer in digital breast tomosynthesis (DBT) could significantly improve detection accuracy and reduce health care costs worldwide. Objectives: To make training and evaluation data for the development of AI algorithms for DBT analysis available, to develop well-defined benchmarks, and to create publicly available code for existing methods. Design, Setting, and Participants: This diagnostic study is based on a multi-institutional international grand challenge in which research teams developed algorithms to detect lesions in DBT. A data set of 22 032 reconstructed DBT volumes was made available to research teams. Phase 1, in which teams were provided 700 scans from the training set, 120 from the validation set, and 180 from the test set, took place from December 2020 to January 2021, and phase 2, in which teams were given the full data set, took place from May to July 2021. Main Outcomes and Measures: The overall performance was evaluated by mean sensitivity for biopsied lesions using only DBT volumes with biopsied lesions; ties were broken by including all DBT volumes. Results: A total of 8 teams participated in the challenge. The team with the highest mean sensitivity for biopsied lesions was the NYU B-Team, with 0.957 (95% CI, 0.924-0.984), and the second-place team, ZeDuS, had a mean sensitivity of 0.926 (95% CI, 0.881-0.964). When the results were aggregated, the mean sensitivity for all submitted algorithms was 0.879; for only those who participated in phase 2, it was 0.926. Conclusions and Relevance: In this diagnostic study, an international competition produced algorithms with high sensitivity for using AI to detect lesions on DBT images. A standardized performance benchmark for the detection task using publicly available clinical imaging data was released, with detailed descriptions and analyses of submitted algorithms accompanied by a public release of their predictions and code for selected methods. These resources will serve as a foundation for future research on computer-assisted diagnosis methods for DBT, significantly lowering the barrier of entry for new researchers.

Performance metric curve analysis framework to assess impact of the decision variable threshold, disease prevalence, and dataset variability in two-class classification.

Purpose: The aim of this study is to (1) demonstrate a graphical method and interpretation framework to extend performance evaluation beyond receiver operating characteristic curve analysis and (2) assess the impact of disease prevalence and variability in training and testing sets, particularly when a specific operating point is used. Approach: The proposed performance metric curves (PMCs) simultaneously assess sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), and the 95% confidence intervals thereof, as a function of the threshold for the decision variable. We investigated the utility of PMCs using six example operating points associated with commonly used methods to select operating points (including the Youden index and maximum mutual information). As an example, we applied PMCs to the task of distinguishing between malignant and benign breast lesions using human-engineered radiomic features extracted from dynamic contrast-enhanced magnetic resonance images. The dataset had 1885 lesions, with the images acquired in 2015 and 2016 serving as the training set (1450 lesions) and those acquired in 2017 as the test set (435 lesions). Our study used this dataset in two ways: (1) the clinical dataset itself and (2) simulated datasets with features based on the clinical set but with five different disease prevalences. The median and 95% CI of the number of type I (false positive) and type II (false negative) errors were determined for each operating point of interest. Results: PMCs from both the clinical and simulated datasets demonstrated that PMCs could support interpretation of the impact of decision threshold choice on type I and type II errors of classification, particularly relevant to prevalence. Conclusion: PMCs allow simultaneous evaluation of the four performance metrics of sensitivity, specificity, PPV, and NPV as a function of the decision threshold. This may create a better understanding of two-class classifier performance in machine learning.

Robustness of radiomic features of benign breast lesions and hormone receptor positive/HER2-negative cancers across DCE-MR magnet strengths.

Radiomic features extracted from breast lesion images have shown potential in diagnosis and prognosis of breast cancer. As medical centers transition from 1.5 T to 3.0 T magnetic resonance (MR) imaging, it is beneficial to identify potentially robust radiomic features across field strengths because images acquired at different field strengths could be used in machine learning models. Dynamic contrast-enhanced MR images of benign breast lesions and hormone receptor positive/HER2-negative (HR+/HER2-) breast cancers were acquired retrospectively, yielding 612 unique cases: 150 and 99 benign lesions imaged at 1.5 T and 3.0 T, and 223 and 140 HR+/HER2- cancerous lesions imaged at 1.5 T and 3.0 T, respectively. In addition, an independent set of seven lesions imaged at both field strengths, three benign lesions and four HR+/HER2- cancers, was analyzed separately. Lesions were automatically segmented using a 4D fuzzy c-means method; thirty-eight radiomic features were extracted. Feature value distributions were compared by cancer status and imaging field strength using the Kolmogorov-Smirnov test. Features that did not demonstrate a statistically significant difference were considered to be potentially robust. The area under the receiver operating characteristic curve (AUC), for the task of classifying lesions as benign or HR+/HER2- cancer, was determined for each feature at each field strength. Three features were found to be both potentially robust across field strength and of high classification performance, i.e., AUCs statistically greater than 0.5 in the classification task: one shape feature (irregularity), one texture feature (sum average) and one enhancement variance kinetics features (enhancement variance increasing rate). In the demonstration set of lesions imaged at both field strengths, two of the three potentially robust features showed qualitative agreement across field strength. These findings may contribute to the development of computer-aided diagnosis models that are robust across field strength for this classification task.

SPIE-AAPM-NCI BreastPathQ challenge: an image analysis challenge for quantitative tumor cellularity assessment in breast cancer histology images following neoadjuvant treatment.

Purpose: The breast pathology quantitative biomarkers (BreastPathQ) challenge was a grand challenge organized jointly by the International Society for Optics and Photonics (SPIE), the American Association of Physicists in Medicine (AAPM), the U.S. National Cancer Institute (NCI), and the U.S. Food and Drug Administration (FDA). The task of the BreastPathQ challenge was computerized estimation of tumor cellularity (TC) in breast cancer histology images following neoadjuvant treatment. Approach: A total of 39 teams developed, validated, and tested their TC estimation algorithms during the challenge. The training, validation, and testing sets consisted of 2394, 185, and 1119 image patches originating from 63, 6, and 27 scanned pathology slides from 33, 4, and 18 patients, respectively. The summary performance metric used for comparing and ranking algorithms was the average prediction probability concordance (PK) using scores from two pathologists as the TC reference standard. Results: Test PK performance ranged from 0.497 to 0.941 across the 100 submitted algorithms. The submitted algorithms generally performed well in estimating TC, with high-performing algorithms obtaining comparable results to the average interrater PK of 0.927 from the two pathologists providing the reference TC scores. Conclusions: The SPIE-AAPM-NCI BreastPathQ challenge was a success, indicating that artificial intelligence/machine learning algorithms may be able to approach human performance for cellularity assessment and may have some utility in clinical practice for improving efficiency and reducing reader variability. The BreastPathQ challenge can be accessed on the Grand Challenge website.

Dual-energy three-compartment breast imaging for compositional biomarkers to improve detection of malignant lesions.

Background: While breast imaging such as full-field digital mammography and digital breast tomosynthesis have helped to reduced breast cancer mortality, issues with low specificity exist resulting in unnecessary biopsies. The fundamental information used in diagnostic decisions are primarily based in lesion morphology. We explore a dual-energy compositional breast imaging technique known as three-compartment breast (3CB) to show how the addition of compositional information improves malignancy detection. Methods: Women who presented with Breast Imaging-Reporting and Data System (BI-RADS) diagnostic categories 4 or 5 and who were scheduled for breast biopsies were consecutively recruited for both standard mammography and 3CB imaging. Computer-aided detection (CAD) software was used to assign a morphology-based prediction of malignancy for all biopsied lesions. Compositional signatures for all lesions were calculated using 3CB imaging and a neural network evaluated CAD predictions with composition to predict a new probability of malignancy. CAD and neural network predictions were compared to the biopsy pathology. Results: The addition of 3CB compositional information to CAD improves malignancy predictions resulting in an area under the receiver operating characteristic curve (AUC) of 0.81 (confidence interval (CI) of 0.74-0.88) on a held-out test set, while CAD software alone achieves an AUC of 0.69 (CI 0.60-0.78). We also identify that invasive breast cancers have a unique compositional signature characterized by reduced lipid content and increased water and protein content when compared to surrounding tissues. Conclusion: Clinically, 3CB may potentially provide increased accuracy in predicting malignancy and a feasible avenue to explore compositional breast imaging biomarkers.

Breast MRI radiomics for the pretreatment prediction of response to neoadjuvant chemotherapy in node-positive breast cancer patients.

The purpose of this study was to evaluate breast MRI radiomics in predicting, prior to any treatment, the response to neoadjuvant chemotherapy (NAC) in patients with invasive lymph node (LN)-positive breast cancer for two tasks: (1) prediction of pathologic complete response and (2) prediction of post-NAC LN status. Our study included 158 patients, with 19 showing post-NAC complete pathologic response (pathologic TNM stage T0,N0,MX) and 139 showing incomplete response. Forty-two patients were post-NAC LN-negative, and 116 were post-NAC LN-positive. We further analyzed prediction of response by hormone receptor subtype of the primary cancer (77 hormone receptor-positive, 39 HER2-enriched, 38 triple negative, and 4 cancers with unknown receptor status). Only pre-NAC MRIs underwent computer analysis, initialized by an expert breast radiologist indicating index cancers and metastatic axillary sentinel LNs on DCE-MRI images. Forty-nine computer-extracted radiomics features were obtained, both for the primary cancers and for the metastatic sentinel LNs. Since the dataset contained MRIs acquired at 1.5 T and at 3.0 T, we eliminated features affected by magnet strength using the Mann-Whitney U-test with the null-hypothesis that 1.5 T and 3.0 T samples were selected from populations having the same distribution. Bootstrapping and ROC analysis were used to assess performance of individual features in the two classification tasks. Eighteen features appeared unaffected by magnet strength. Pre-NAC tumor features generally appeared uninformative in predicting response to therapy. In contrast, some pre-NAC LN features were able to predict response: two pre-NAC LN features were able to predict pathologic complete response (area under the ROC curve (AUC) up to 0.82 [0.70; 0.88]), and another two were able to predict post-NAC LN-status (AUC up to 0.72 [0.62; 0.77]), respectively. In the analysis by a hormone receptor subtype, several potentially useful features were identified for predicting response to therapy in the hormone receptor-positive and HER2-enriched cancers.

Radiogenomics of breast cancer using dynamic contrast enhanced MRI and gene expression profiling.

BACKGROUND: Imaging techniques can provide information about the tumor non-invasively and have been shown to provide information about the underlying genetic makeup. Correlating image-based phenotypes (radiomics) with genomic analyses is an emerging area of research commonly referred to as "radiogenomics" or "imaging-genomics". The purpose of this study was to assess the potential for using an automated, quantitative radiomics platform on magnetic resonance (MR) breast imaging for inferring underlying activity of clinically relevant gene pathways derived from RNA sequencing of invasive breast cancers prior to therapy. METHODS: We performed quantitative radiomic analysis on 47 invasive breast cancers based on dynamic contrast enhanced 3 Tesla MR images acquired before surgery and obtained gene expression data by performing total RNA sequencing on corresponding fresh frozen tissue samples. We used gene set enrichment analysis to identify significant associations between the 186 gene pathways and the 38 image-based features that have previously been validated. RESULTS: All radiomic size features were positively associated with multiple replication and proliferation pathways and were negatively associated with the apoptosis pathway. Gene pathways related to immune system regulation and extracellular signaling had the highest number of significant radiomic feature associations, with an average of 18.9 and 16 features per pathway, respectively. Tumors with upregulation of immune signaling pathways such as T-cell receptor signaling and chemokine signaling as well as extracellular signaling pathways such as cell adhesion molecule and cytokine-cytokine interactions were smaller, more spherical, and had a more heterogeneous texture upon contrast enhancement. Tumors with higher expression levels of JAK/STAT and VEGF pathways had more intratumor heterogeneity in image enhancement texture. Other pathways with robust associations to image-based features include metabolic and catabolic pathways. CONCLUSIONS: We provide further evidence that MR imaging of breast tumors can infer underlying gene expression by using RNA sequencing. Size and shape features were appropriately correlated with proliferative and apoptotic pathways. Given the high number of radiomic feature associations with immune pathways, our results raise the possibility of using MR imaging to distinguish tumors that are more immunologically active, although further studies are necessary to confirm this observation.

Additive Benefit of Radiomics Over Size Alone in the Distinction Between Benign Lesions and Luminal A Cancers on a Large Clinical Breast MRI Dataset.

RATIONALE AND OBJECTIVES: The objective of this study was to demonstrate improvement in distinguishing between benign lesions and luminal A breast cancers in a large clinical breast magnetic resonance imaging database by using quantitative radiomics over maximum linear size alone. MATERIALS AND METHODS: In this retrospective study, 264 benign lesions and 390 luminal A breast cancers were automatically segmented from dynamic contrast-enhanced breast magnetic resonance images. Thirty-eight radiomic features were extracted. Tenfold cross validation was performed to assess the ability to distinguish between lesions and cancers using maximum linear size alone and lesion signatures obtained with stepwise feature selection and a linear discriminant analysis classifier including and excluding size features. Area under the receiver operating characteristic curve (AUC) was used as the figure of merit. RESULTS: For maximum linear size alone, AUC and 95% confidence interval was 0.684 (0.642, 0.724) compared to 0.728 (0.687, 0.766) (P = 0.005) and 0.729 (0.689, 0.767) (P = 0.005) for lesion signature feature selection protocols including and excluding size features, respectively. The features of irregularity and entropy were chosen in all folds when size features were included and excluded. AUC for the radiomic signature using feature selection from all features was statistically equivalent to using feature selection from all features excluding size features, within an equivalence margin of 2%. CONCLUSIONS: Inclusion of multiple radiomic features, automatically extracted from magnetic resonance images, in a lesion signature significantly improved the ability to distinguish between benign lesions and luminal A breast cancers, compared to using maximum linear size alone. The radiomic features of irregularity and entropy appear to play an important but not a solitary role within the context of feature selection and computer-aided diagnosis.

Deep learning in medical imaging and radiation therapy.

The goals of this review paper on deep learning (DL) in medical imaging and radiation therapy are to (a) summarize what has been achieved to date; (b) identify common and unique challenges, and strategies that researchers have taken to address these challenges; and (c) identify some of the promising avenues for the future both in terms of applications as well as technical innovations. We introduce the general principles of DL and convolutional neural networks, survey five major areas of application of DL in medical imaging and radiation therapy, identify common themes, discuss methods for dataset expansion, and conclude by summarizing lessons learned, remaining challenges, and future directions.

Effect of biopsy on the MRI radiomics classification of benign lesions and luminal A cancers.

Radiomic features extracted from magnetic resonance (MR) images have potential for diagnosis and prognosis of breast cancer. However, presentation of lesions on images may be affected by biopsy. Thirty-four nonsize features were extracted from 338 dynamic contrast-enhanced MR images of benign lesions and luminal A cancers (80 benign/34 luminal A prebiopsy; 46 benign/178 luminal A postbiopsy). Feature value distributions were compared by biopsy condition using the Kolmogorov-Smirnov test. Classification performance was assessed by biopsy condition in the task of distinguishing between lesion types using the area under the receiver operating characteristic curve (AUCROC) as performance metric. Superiority and equivalence testing of differences in AUCROC between biopsy conditions were conducted using Bonferroni-Holm-adjusted significance levels. Distributions for most nonsize features for each lesion type failed to show a statistically significant difference between biopsy conditions. Fourteen features outperformed random guessing in classification. Their differences in AUCROC by biopsy condition failed to reach statistical significance, but we were unable to prove equivalence using a margin of Δ AUCROC = ± 0.10 . However, classification performance for lesions imaged either prebiopsy or postbiopsy appears to be similar when taking into account biopsy condition.

Combined Benefit of Quantitative Three-Compartment Breast Image Analysis and Mammography Radiomics in the Classification of Breast Masses in a Clinical Data Set.

Purpose To investigate the combination of mammography radiomics and quantitative three-compartment breast (3CB) image analysis of dual-energy mammography to limit unnecessary benign breast biopsies. Materials and Methods For this prospective study, dual-energy craniocaudal and mediolateral oblique mammograms were obtained immediately before biopsy in 109 women (mean age, 51 years; range, 31-85 years) with Breast Imaging Reporting and Data System category 4 or 5 breast masses (35 invasive cancers, 74 benign) from 2013 through 2017. The three quantitative compartments of water, lipid, and protein thickness at each pixel were calculated from the attenuation at high and low energy by using a within-image phantom. Masses were automatically segmented and features were extracted from the low-energy mammograms and the quantitative compartment images. Tenfold cross-validations using a linear discriminant classifier with predefined feature signatures helped differentiate between malignant and benign masses by means of (a) water-lipid-protein composition images alone, (b) mammography radiomics alone, and (c) a combined image analysis of both. Positive predictive value of biopsy performed (PPV3) at maximum sensitivity was the primary performance metric, and results were compared with those for conventional diagnostic digital mammography. Results The PPV3 for conventional diagnostic digital mammography in our data set was 32.1% (35 of 109; 95% confidence interval [CI]: 23.9%, 41.3%), with a sensitivity of 100%. In comparison, combined mammography radiomics plus quantitative 3CB image analysis had PPV3 of 49% (34 of 70; 95% CI: 36.5%, 58.9%; P < .001), with a sensitivity of 97% (34 of 35; 95% CI: 90.3%, 100%; P < .001) and 35.8% (39 of 109) fewer total biopsies (P < .001). Conclusion Quantitative three-compartment breast image analysis of breast masses combined with mammography radiomics has the potential to reduce unnecessary breast biopsies. © RSNA, 2018 Online supplemental material is available for this article.

Most-enhancing tumor volume by MRI radiomics predicts recurrence-free survival "early on" in neoadjuvant treatment of breast cancer.

BACKGROUND: The hypothesis of this study was that MRI-based radiomics has the ability to predict recurrence-free survival "early on" in breast cancer neoadjuvant chemotherapy. METHODS: A subset, based on availability, of the ACRIN 6657 dynamic contrast-enhanced MR images was used in which we analyzed images of all women imaged at pre-treatment baseline (141 women: 40 with a recurrence, 101 without) and all those imaged after completion of the first cycle of chemotherapy, i.e., at early treatment (143 women: 37 with a recurrence vs. 105 without). Our method was completely automated apart from manual localization of the approximate tumor center. The most enhancing tumor volume (METV) was automatically calculated for the pre-treatment and early treatment exams. Performance of METV in the task of predicting a recurrence was evaluated using ROC analysis. The association of recurrence-free survival with METV was assessed using a Cox regression model controlling for patient age, race, and hormone receptor status and evaluated by C-statistics. Kaplan-Meier analysis was used to estimate survival functions. RESULTS: The C-statistics for the association of METV with recurrence-free survival were 0.69 with 95% confidence interval of [0.58; 0.80] at pre-treatment and 0.72 [0.60; 0.84] at early treatment. The hazard ratios calculated from Kaplan-Meier curves were 2.28 [1.08; 4.61], 3.43 [1.83; 6.75], and 4.81 [2.16; 10.72] for the lowest quartile, median quartile, and upper quartile cut-points for METV at early treatment, respectively. CONCLUSION: The performance of the automatically-calculated METV rivaled that of a semi-manual model described for the ACRIN 6657 study (published C-statistic 0.72 [0.60; 0.84]), which involved the same dataset but required semi-manual delineation of the functional tumor volume (FTV) and knowledge of the pre-surgical residual cancer burden.

PROSTATEx Challenges for computerized classification of prostate lesions from multiparametric magnetic resonance images.

Grand challenges stimulate advances within the medical imaging research community; within a competitive yet friendly environment, they allow for a direct comparison of algorithms through a well-defined, centralized infrastructure. The tasks of the two-part PROSTATEx Challenges (the PROSTATEx Challenge and the PROSTATEx-2 Challenge) are (1) the computerized classification of clinically significant prostate lesions and (2) the computerized determination of Gleason Grade Group in prostate cancer, both based on multiparametric magnetic resonance images. The challenges incorporate well-vetted cases for training and testing, a centralized performance assessment process to evaluate results, and an established infrastructure for case dissemination, communication, and result submission. In the PROSTATEx Challenge, 32 groups apply their computerized methods (71 methods total) to 208 prostate lesions in the test set. The area under the receiver operating characteristic curve for these methods in the task of differentiating between lesions that are and are not clinically significant ranged from 0.45 to 0.87; statistically significant differences in performance among the top-performing methods, however, are not observed. In the PROSTATEx-2 Challenge, 21 groups apply their computerized methods (43 methods total) to 70 prostate lesions in the test set. When compared with the reference standard, the quadratic-weighted kappa values for these methods in the task of assigning a five-point Gleason Grade Group to each lesion range from - 0.24 to 0.27; superiority to random guessing can be established for only two methods. When approached with a sense of commitment and scientific rigor, challenges foster interest in the designated task and encourage innovation in the field.

Breast MRI radiomics: comparison of computer- and human-extracted imaging phenotypes.

BACKGROUND: In this study, we sought to investigate if computer-extracted magnetic resonance imaging (MRI) phenotypes of breast cancer could replicate human-extracted size and Breast Imaging-Reporting and Data System (BI-RADS) imaging phenotypes using MRI data from The Cancer Genome Atlas (TCGA) project of the National Cancer Institute. METHODS: Our retrospective interpretation study involved analysis of Health Insurance Portability and Accountability Act-compliant breast MRI data from The Cancer Imaging Archive, an open-source database from the TCGA project. This study was exempt from institutional review board approval at Memorial Sloan Kettering Cancer Center and the need for informed consent was waived. Ninety-one pre-operative breast MRIs with verified invasive breast cancers were analysed. Three fellowship-trained breast radiologists evaluated the index cancer in each case according to size and the BI-RADS lexicon for shape, margin, and enhancement (human-extracted image phenotypes [HEIP]). Human inter-observer agreement was analysed by the intra-class correlation coefficient (ICC) for size and Krippendorff's α for other measurements. Quantitative MRI radiomics of computerised three-dimensional segmentations of each cancer generated computer-extracted image phenotypes (CEIP). Spearman's rank correlation coefficients were used to compare HEIP and CEIP. RESULTS: Inter-observer agreement for HEIP varied, with the highest agreement seen for size (ICC 0.679) and shape (ICC 0.527). The computer-extracted maximum linear size replicated the human measurement with p < 10-12. CEIP of shape, specifically sphericity and irregularity, replicated HEIP with both p values < 0.001. CEIP did not demonstrate agreement with HEIP of tumour margin or internal enhancement. CONCLUSIONS: Quantitative radiomics of breast cancer may replicate human-extracted tumour size and BI-RADS imaging phenotypes, thus enabling precision medicine.

LUNGx Challenge for computerized lung nodule classification.

The purpose of this work is to describe the LUNGx Challenge for the computerized classification of lung nodules on diagnostic computed tomography (CT) scans as benign or malignant and report the performance of participants' computerized methods along with that of six radiologists who participated in an observer study performing the same Challenge task on the same dataset. The Challenge provided sets of calibration and testing scans, established a performance assessment process, and created an infrastructure for case dissemination and result submission. Ten groups applied their own methods to 73 lung nodules (37 benign and 36 malignant) that were selected to achieve approximate size matching between the two cohorts. Area under the receiver operating characteristic curve (AUC) values for these methods ranged from 0.50 to 0.68; only three methods performed statistically better than random guessing. The radiologists' AUC values ranged from 0.70 to 0.85; three radiologists performed statistically better than the best-performing computer method. The LUNGx Challenge compared the performance of computerized methods in the task of differentiating benign from malignant lung nodules on CT scans, placed in the context of the performance of radiologists on the same task. The continued public availability of the Challenge cases will provide a valuable resource for the medical imaging research community.

Quantitative MRI radiomics in the prediction of molecular classifications of breast cancer subtypes in the TCGA/TCIA data set.

Using quantitative radiomics, we demonstrate that computer-extracted magnetic resonance (MR) image-based tumor phenotypes can be predictive of the molecular classification of invasive breast cancers. Radiomics analysis was performed on 91 MRIs of biopsy-proven invasive breast cancers from National Cancer Institute's multi-institutional TCGA/TCIA. Immunohistochemistry molecular classification was performed including estrogen receptor, progesterone receptor, human epidermal growth factor receptor 2, and for 84 cases, the molecular subtype (normal-like, luminal A, luminal B, HER2-enriched, and basal-like). Computerized quantitative image analysis included: three-dimensional lesion segmentation, phenotype extraction, and leave-one-case-out cross validation involving stepwise feature selection and linear discriminant analysis. The performance of the classifier model for molecular subtyping was evaluated using receiver operating characteristic analysis. The computer-extracted tumor phenotypes were able to distinguish between molecular prognostic indicators; area under the ROC curve values of 0.89, 0.69, 0.65, and 0.67 in the tasks of distinguishing between ER+ versus ER-, PR+ versus PR-, HER2+ versus HER2-, and triple-negative versus others, respectively. Statistically significant associations between tumor phenotypes and receptor status were observed. More aggressive cancers are likely to be larger in size with more heterogeneity in their contrast enhancement. Even after controlling for tumor size, a statistically significant trend was observed within each size group (P = 0.04 for lesions ≤ 2 cm; P = 0.02 for lesions >2 to ≤5 cm) as with the entire data set (P-value = 0.006) for the relationship between enhancement texture (entropy) and molecular subtypes (normal-like, luminal A, luminal B, HER2-enriched, basal-like). In conclusion, computer-extracted image phenotypes show promise for high-throughput discrimination of breast cancer subtypes and may yield a quantitative predictive signature for advancing precision medicine.

MR Imaging Radiomics Signatures for Predicting the Risk of Breast Cancer Recurrence as Given by Research Versions of MammaPrint, Oncotype DX, and PAM50 Gene Assays.

Purpose To investigate relationships between computer-extracted breast magnetic resonance (MR) imaging phenotypes with multigene assays of MammaPrint, Oncotype DX, and PAM50 to assess the role of radiomics in evaluating the risk of breast cancer recurrence. Materials and Methods Analysis was conducted on an institutional review board-approved retrospective data set of 84 deidentified, multi-institutional breast MR examinations from the National Cancer Institute Cancer Imaging Archive, along with clinical, histopathologic, and genomic data from The Cancer Genome Atlas. The data set of biopsy-proven invasive breast cancers included 74 (88%) ductal, eight (10%) lobular, and two (2%) mixed cancers. Of these, 73 (87%) were estrogen receptor positive, 67 (80%) were progesterone receptor positive, and 19 (23%) were human epidermal growth factor receptor 2 positive. For each case, computerized radiomics of the MR images yielded computer-extracted tumor phenotypes of size, shape, margin morphology, enhancement texture, and kinetic assessment. Regression and receiver operating characteristic analysis were conducted to assess the predictive ability of the MR radiomics features relative to the multigene assay classifications. Results Multiple linear regression analyses demonstrated significant associations (R2 = 0.25-0.32, r = 0.5-0.56, P < .0001) between radiomics signatures and multigene assay recurrence scores. Important radiomics features included tumor size and enhancement texture, which indicated tumor heterogeneity. Use of radiomics in the task of distinguishing between good and poor prognosis yielded area under the receiver operating characteristic curve values of 0.88 (standard error, 0.05), 0.76 (standard error, 0.06), 0.68 (standard error, 0.08), and 0.55 (standard error, 0.09) for MammaPrint, Oncotype DX, PAM50 risk of relapse based on subtype, and PAM50 risk of relapse based on subtype and proliferation, respectively, with all but the latter showing statistical difference from chance. Conclusion Quantitative breast MR imaging radiomics shows promise for image-based phenotyping in assessing the risk of breast cancer recurrence. © RSNA, 2016 Online supplemental material is available for this article.

Automated Breast Ultrasound in Breast Cancer Screening of Women With Dense Breasts: Reader Study of Mammography-Negative and Mammography-Positive Cancers.

OBJECTIVE: The objective of our study was to assess and compare, in a reader study, radiologists' performance in the detection of breast cancer using full-field digital mammography (FFDM) alone and using FFDM with 3D automated breast ultrasound (ABUS). MATERIALS AND METHODS: In this multireader, multicase, sequential-design reader study, 17 Mammography Quality Standards Act-qualified radiologists interpreted a cancer-enriched set of FFDM and ABUS examinations. All imaging studies were of asymptomatic women with BI-RADS C or D breast density. Readers first interpreted FFDM alone and subsequently interpreted FFDM combined with ABUS. The analysis included 185 cases: 133 noncancers and 52 biopsy-proven cancers. Of the 52 cancer cases, the screening FFDM images were interpreted as showing BI-RADS 1 or 2 findings in 31 cases and BI-RADS 0 findings in 21 cases. For the cases interpreted as BI-RADS 0, a forced BI-RADS score was also given. Reader performance was compared in terms of AUC under the ROC curve, sensitivity, and specificity. RESULTS: The AUC was 0.72 for FFDM alone and 0.82 for FFDM combined with ABUS, yielding a statistically significant 14% relative improvement in AUC (i.e., change in AUC = 0.10 [95% CI, 0.07-0.14]; p < 0.001). When a cutpoint of BI-RADS 3 was used, the sensitivity across all readers was 57.5% for FFDM alone and 74.1% for FFDM with ABUS, yielding a statistically significant increase in sensitivity (p < 0.001) (relative increase = 29%). Overall specificity was 78.1% for FFDM alone and 76.1% for FFDM with ABUS (p = 0.496). For only the mammography-negative cancers, the average AUC was 0.60 for FFDM alone and 0.75 for FFDM with ABUS, yielding a statistically significant 25% relative improvement in AUC with the addition of ABUS (p < 0.001). CONCLUSION: Combining mammography with ABUS, compared with mammography alone, significantly improved readers' detection of breast cancers in women with dense breast tissue without substantially affecting specificity.