Examination of fully automated mammographic density measures using LIBRA and breast cancer risk in a cohort of 21,000 non-Hispanic white women.

BACKGROUND: Breast density is strongly associated with breast cancer risk. Fully automated quantitative density assessment methods have recently been developed that could facilitate large-scale studies, although data on associations with long-term breast cancer risk are limited. We examined LIBRA assessments and breast cancer risk and compared results to prior assessments using Cumulus, an established computer-assisted method requiring manual thresholding. METHODS: We conducted a cohort study among 21,150 non-Hispanic white female participants of the Research Program in Genes, Environment and Health of Kaiser Permanente Northern California who were 40-74 years at enrollment, followed for up to 10 years, and had archived processed screening mammograms acquired on Hologic or General Electric full-field digital mammography (FFDM) machines and prior Cumulus density assessments available for analysis. Dense area (DA), non-dense area (NDA), and percent density (PD) were assessed using LIBRA software. Cox regression was used to estimate hazard ratios (HRs) for breast cancer associated with DA, NDA and PD modeled continuously in standard deviation (SD) increments, adjusting for age, mammogram year, body mass index, parity, first-degree family history of breast cancer, and menopausal hormone use. We also examined differences by machine type and breast view. RESULTS: The adjusted HRs for breast cancer associated with each SD increment of DA, NDA and PD were 1.36 (95% confidence interval, 1.18-1.57), 0.85 (0.77-0.93) and 1.44 (1.26-1.66) for LIBRA and 1.44 (1.33-1.55), 0.81 (0.74-0.89) and 1.54 (1.34-1.77) for Cumulus, respectively. LIBRA results were generally similar by machine type and breast view, although associations were strongest for Hologic machines and mediolateral oblique views. Results were also similar during the first 2 years, 2-5 years and 5-10 years after the baseline mammogram. CONCLUSION: Associations with breast cancer risk were generally similar for LIBRA and Cumulus density measures and were sustained for up to 10 years. These findings support the suitability of fully automated LIBRA assessments on processed FFDM images for large-scale research on breast density and cancer risk.

Atrial fibrillation ablation outcome prediction with a machine learning fusion framework incorporating cardiac computed tomography.

BACKGROUND: Structural changes in the left atrium (LA) modestly predict outcomes in patients undergoing catheter ablation for atrial fibrillation (AF). Machine learning (ML) is a promising approach to personalize AF management strategies and improve predictive risk models after catheter ablation by integrating atrial geometry from cardiac computed tomography (CT) scans and patient-specific clinical data. We hypothesized that ML approaches based on a patient's specific data can identify responders to AF ablation. METHODS: Consecutive patients undergoing AF ablation, who had preprocedural CT scans, demographics, and 1-year follow-up data, were included in the study for a retrospective analysis. The inputs of models were CT-derived morphological features from left atrial segmentation (including the shape, volume of the LA, LA appendage, and pulmonary vein ostia) along with deep features learned directly from raw CT images, and clinical data. These were merged intelligently in a framework to learn their individual importance and produce the optimal classification. RESULTS: Three hundred twenty-one patients (64.2 ± 10.6 years, 69% male, 40% paroxysmal AF) were analyzed. Post 10-fold nested cross-validation, the model trained to intelligently merge and learn appropriate weights for clinical, morphological, and imaging data (AUC 0.821) outperformed those trained solely on clinical data (AUC 0.626), morphological (AUC 0.659), or imaging data (AUC 0.764). CONCLUSION: Our ML approach provides an end-to-end automated technique to predict AF ablation outcomes using deep learning from CT images, derived structural properties of LA, augmented by incorporation of clinical data in a merged ML framework. This can help develop personalized strategies for patient selection in invasive management of AF.

Clinical outcome prediction using observational supervision with electronic health records and audit logs.

OBJECTIVE: Audit logs in electronic health record (EHR) systems capture interactions of providers with clinical data. We determine if machine learning (ML) models trained using audit logs in conjunction with clinical data ("observational supervision") outperform ML models trained using clinical data alone in clinical outcome prediction tasks, and whether they are more robust to temporal distribution shifts in the data. MATERIALS AND METHODS: Using clinical and audit log data from Stanford Healthcare, we trained and evaluated various ML models including logistic regression, support vector machine (SVM) classifiers, neural networks, random forests, and gradient boosted machines (GBMs) on clinical EHR data, with and without audit logs for two clinical outcome prediction tasks: major adverse kidney events within 120 days of ICU admission (MAKE-120) in acute kidney injury (AKI) patients and 30-day readmission in acute stroke patients. We further tested the best performing models using patient data acquired during different time-intervals to evaluate the impact of temporal distribution shifts on model performance. RESULTS: Performance generally improved for all models when trained with clinical EHR data and audit log data compared with those trained with only clinical EHR data, with GBMs tending to have the overall best performance. GBMs trained with clinical EHR data and audit logs outperformed GBMs trained without audit logs in both clinical outcome prediction tasks: AUROC 0.88 (95% CI: 0.85-0.91) vs. 0.79 (95% CI: 0.77-0.81), respectively, for MAKE-120 prediction in AKI patients, and AUROC 0.74 (95% CI: 0.71-0.77) vs. 0.63 (95% CI: 0.62-0.64), respectively, for 30-day readmission prediction in acute stroke patients. The performance of GBM models trained using audit log and clinical data degraded less in later time-intervals than models trained using only clinical data. CONCLUSION: Observational supervision with audit logs improved the performance of ML models trained to predict important clinical outcomes in patients with AKI and acute stroke, and improved robustness to temporal distribution shifts.

Natural language processing of head CT reports to identify intracranial mass effect: CTIME algorithm.

BACKGROUND: The Mortality Probability Model (MPM) is used in research and quality improvement to adjust for severity of illness and can also inform triage decisions. However, a limitation for its automated use or application is that it includes the variable "intracranial mass effect" (IME), which requires human engagement with the electronic health record (EHR). We developed and tested a natural language processing (NLP) algorithm to identify IME from CT head reports. METHODS: We obtained initial CT head reports from adult patients who were admitted to the ICU from our ED between 10/2013 and 9/2016. Each head CT head report was labeled yes/no IME by at least two of five independent labelers. The reports were then randomly divided 80/20 into training and test sets. All reports were preprocessed to remove linguistic and style variability, and a dictionary was created to map similar common terms. We tested three vectorization strategies: Term Frequency-Inverse Document frequency (TF-IDF), Word2Vec, and Universal Sentence Encoder to convert the report text to a numerical vector. This vector served as the input to a classification-tree-based ensemble machine learning algorithm (XGBoost). After training, model performance was assessed in the test set using the area under the receiver operating characteristic curve (AUROC). We also divided the continuous range of scores into positive/inconclusive/negative categories for IME. RESULTS: Of the 1202 CT reports in the training set, 308 (25.6%) reports were manually labeled as "yes" for IME. Of the 355 reports in the test set, 108 (30.4%) were labeled as "yes" for IME. The TF-IDF vectorization strategy as an input for the XGBoost model had the best AUROC:-- 0.9625 (95% CI 0.9443-0.9807). TF-IDF score categories were defined and had the following likelihood ratios: "positive" (TF-IDF score > 0.5) LR = 24.59; "inconclusive" (TF-IDF 0.05-0.5) LR = 0.99; and "negative" (TF-IDF < 0.05) LR = 0.05. 82% of reports were classified as either "positive" or "negative". In the test set, only 4 of 199 (2.0%) reports with a "negative" classification were false negatives and only 8 of 93 (8.6%) reports classified as "positive" were false positives. CONCLUSION: NLP can accurately identify IME from free-text reports of head CTs in approximately 80% of records, adequate to allow automatic calculation of MPM based on EHR data for many applications.

Automating Scoliosis Measurements in Radiographic Studies with Machine Learning: Comparing Artificial Intelligence and Clinical Reports.

Scoliosis is a condition of abnormal lateral spinal curvature affecting an estimated 2 to 3% of the US population, or seven million people. The Cobb angle is the standard measurement of spinal curvature in scoliosis but is known to have high interobserver and intraobserver variability. Thus, the objective of this study was to build and validate a system for automatic quantitative evaluation of the Cobb angle and to compare AI generated and human reports in the clinical setting. After IRB was obtained, we retrospectively collected 2150 frontal view scoliosis radiographs at a tertiary referral center (January 1, 2019, to January 1, 2021, ≥ 16 years old, no hardware). The dataset was partitioned into 1505 train (70%), 215 validation (10%), and 430 test images (20%). All thoracic and lumbar vertebral bodies were segmented with bounding boxes, generating approximately 36,550 object annotations that were used to train a Faster R-CNN Resnet-101 object detection model. A controller algorithm was written to localize vertebral centroid coordinates and derive the Cobb properties (angle and endplate) of dominant and secondary curves. AI-derived Cobb angle measurements were compared to the clinical report measurements, and the Spearman rank-order demonstrated significant correlation (0.89, p < 0.001). Mean difference between AI and clinical report angle measurements was 7.34° (95% CI: 5.90-8.78°), which is similar to published literature (up to 10°). We demonstrate the feasibility of an AI system to automate measurement of level-by-level spinal angulation with performance comparable to radiologists.

Deep learning model for the prediction of microsatellite instability in colorectal cancer: a diagnostic study.

BACKGROUND: Detecting microsatellite instability (MSI) in colorectal cancer is crucial for clinical decision making, as it identifies patients with differential treatment response and prognosis. Universal MSI testing is recommended, but many patients remain untested. A critical need exists for broadly accessible, cost-efficient tools to aid patient selection for testing. Here, we investigate the potential of a deep learning-based system for automated MSI prediction directly from haematoxylin and eosin (H&E)-stained whole-slide images (WSIs). METHODS: Our deep learning model (MSINet) was developed using 100 H&E-stained WSIs (50 with microsatellite stability [MSS] and 50 with MSI) scanned at 40× magnification, each from a patient randomly selected in a class-balanced manner from the pool of 343 patients who underwent primary colorectal cancer resection at Stanford University Medical Center (Stanford, CA, USA; internal dataset) between Jan 1, 2015, and Dec 31, 2017. We internally validated the model on a holdout test set (15 H&E-stained WSIs from 15 patients; seven cases with MSS and eight with MSI) and externally validated the model on 484 H&E-stained WSIs (402 cases with MSS and 77 with MSI; 479 patients) from The Cancer Genome Atlas, containing WSIs scanned at 40× and 20× magnification. Performance was primarily evaluated using the sensitivity, specificity, negative predictive value (NPV), and area under the receiver operating characteristic curve (AUROC). We compared the model's performance with that of five gastrointestinal pathologists on a class-balanced, randomly selected subset of 40× magnification WSIs from the external dataset (20 with MSS and 20 with MSI). FINDINGS: The MSINet model achieved an AUROC of 0·931 (95% CI 0·771-1·000) on the holdout test set from the internal dataset and 0·779 (0·720-0·838) on the external dataset. On the external dataset, using a sensitivity-weighted operating point, the model achieved an NPV of 93·7% (95% CI 90·3-96·2), sensitivity of 76·0% (64·8-85·1), and specificity of 66·6% (61·8-71·2). On the reader experiment (40 cases), the model achieved an AUROC of 0·865 (95% CI 0·735-0·995). The mean AUROC performance of the five pathologists was 0·605 (95% CI 0·453-0·757). INTERPRETATION: Our deep learning model exceeded the performance of experienced gastrointestinal pathologists at predicting MSI on H&E-stained WSIs. Within the current universal MSI testing paradigm, such a model might contribute value as an automated screening tool to triage patients for confirmatory testing, potentially reducing the number of tested patients, thereby resulting in substantial test-related labour and cost savings. FUNDING: Stanford Cancer Institute and Stanford Departments of Pathology and Biomedical Data Science.

Comparison of segmentation-free and segmentation-dependent computer-aided diagnosis of breast masses on a public mammography dataset.

PURPOSE: To compare machine learning methods for classifying mass lesions on mammography images that use predefined image features computed over lesion segmentations to those that leverage segmentation-free representation learning on a standard, public evaluation dataset. METHODS: We apply several classification algorithms to the public Curated Breast Imaging Subset of the Digital Database for Screening Mammography (CBIS-DDSM), in which each image contains a mass lesion. Segmentation-free representation learning techniques for classifying lesions as benign or malignant include both a Bag-of-Visual-Words (BoVW) method and a Convolutional Neural Network (CNN). We compare classification performance of these techniques to that obtained using two different segmentation-dependent approaches from the literature that rely on specific combinations of end classifiers (e.g. linear discriminant analysis, neural networks) and predefined features computed over the lesion segmentation (e.g. spiculation measure, morphological characteristics, intensity metrics). RESULTS: We report area under the receiver operating characteristic curve (AZ) values for malignancy classification on CBIS-DDSM for each technique. We find average AZ values of 0.73 for a segmentation-free BoVW method, 0.86 for a segmentation-free CNN method, 0.75 for a segmentation-dependent linear discriminant analysis of Rubber-Band Straightening Transform features, and 0.58 for a hybrid rule-based neural network classification using a small number of hand-designed features. CONCLUSIONS: We find that malignancy classification performance on the CBIS-DDSM dataset using segmentation-free BoVW features is comparable to that of the best segmentation-dependent methods we study, but also observe that a common segmentation-free CNN model substantially and significantly outperforms each of these (p < 0.05). These results reinforce recent findings suggesting that representation learning techniques such as BoVW and CNNs are advantageous for mammogram analysis because they do not require lesion segmentation, the quality and specific characteristics of which can vary substantially across datasets. We further observe that segmentation-dependent methods achieve performance levels on CBIS-DDSM inferior to those achieved on the original evaluation datasets reported in the literature. Each of these findings reinforces the need for standardization of datasets, segmentation techniques, and model implementations in performance assessments of automated classifiers for medical imaging.

Preparing Medical Imaging Data for Machine Learning.

Artificial intelligence (AI) continues to garner substantial interest in medical imaging. The potential applications are vast and include the entirety of the medical imaging life cycle from image creation to diagnosis to outcome prediction. The chief obstacles to development and clinical implementation of AI algorithms include availability of sufficiently large, curated, and representative training data that includes expert labeling (eg, annotations). Current supervised AI methods require a curation process for data to optimally train, validate, and test algorithms. Currently, most research groups and industry have limited data access based on small sample sizes from small geographic areas. In addition, the preparation of data is a costly and time-intensive process, the results of which are algorithms with limited utility and poor generalization. In this article, the authors describe fundamental steps for preparing medical imaging data in AI algorithm development, explain current limitations to data curation, and explore new approaches to address the problem of data availability.

Automated Quantitative Imaging Measurements of Disease Severity in Patients with Nonthrombotic Iliac Vein Compression.

PURPOSE: An automated segmentation technique (AST) for computed tomography (CT) venography was developed to quantify measures of disease severity before and after stent placement in patients with left-sided nonthrombotic iliac vein compression. MATERIALS AND METHODS: Twenty-one patients with left-sided nonthrombotic iliac vein compression who underwent venous stent placement were retrospectively identified. Pre- and poststent CT venography studies were quantitatively analyzed using an AST to determine leg volume, skin thickness, and water content of fat. These measures were compared between diseased and nondiseased limbs and between pre- and poststent images, using patients as their own controls. Additionally, patients with and without postthrombotic lesions were compared. RESULTS: The AST detected significantly increased leg volume (12,437 cm3 vs 10,748 cm3, P < .0001), skin thickness (0.531 cm vs 0.508 cm, P < .0001), and water content of fat (8.2% vs 5.0%, P < .0001) in diseased left limbs compared with the contralateral nondiseased limbs, on prestent imaging. After stent placement in the left leg, there was a significant decrease in the water content of fat in the right (4.9% vs 2.7%, P < .0001) and left (8.2% vs 3.2%, P < .0001) legs. There were no significant changes in leg volume or skin thickness in either leg after stent placement. There were no significant differences between patients with or without postthrombotic lesions in their poststent improvement across the 3 measures of disease severity. CONCLUSIONS: ASTs can be used to quantify measures of disease severity and postintervention changes on CT venography for patients with lower extremity venous disease. Further investigation may clarify the clinical benefit of such technologies.

Lower Extremity Venous Stent Placement: A Large Retrospective Single-Center Analysis.

PURPOSE: To study short-term and long-term outcomes of lower extremity venous stents placed at a single center and to characterize changes in vein diameter achieved by stent placement. MATERIALS AND METHODS: A database of all patients who received lower extremity venous stents between 1996 and 2018 revealed 1,094 stents were placed in 406 patients (172 men, 234 women; median age, 49 y) in 513 limbs, including patients with iliocaval stents (9.4% acute thrombosis, 65.3% chronic thrombosis, 25.3% nonthrombotic lesions). Primary, primary assisted, and secondary patency rates were assessed for lower extremity venous stents at 1, 3, and 5 years using Kaplan-Meier analyses and summary statistics. Subset analyses and Cox regression were performed to identify risk factors for patency loss. Vein diameters and Villalta scores before and up to 12 months after stent placement were compared. Complication and mortality rates were calculated. RESULTS: Primary, primary assisted, and secondary patency rates at 5 years were 57.3%, 77.2%, and 80.9% by Kaplan-Meier methods and 78.6%, 90.3%, and 92.8% by summary statistics. Median follow-up was 199 days (interquartile range, 35.2-712.0 d). Patency rates for the subset of patients (n = 46) with ≥ 5 years of follow-up (mean ± SD 9.1 y ± 3.4) were nearly identical to cohort patency rates at 5 years. Patients with inferior vena cava stent placement (hazard ratio 2.11, P < .0001) or acute thrombosis (hazard ratio 3.65, P < .0001) during the index procedure had significantly increased risk of losing primary patency status. Vein diameters were significantly greater after stent placement. There were no instances of stent fracture, migration, or structural deformities. In patients with chronic deep vein thrombosis, Villalta scores significantly decreased after stent placement (from 15.7 to 7.4, P < .0001). Perioperative mortality was < 1%, and major perioperative complication rate was 3.7%. CONCLUSIONS: Cavo-ilio-femoral stent placement for venous occlusive disease achieves improvement of vein disease severity scores, increase in treated vein diameters, and satisfactory long-term patency rates.

Quantitative Framework for Risk Stratification of Thyroid Nodules With Ultrasound: A Step Toward Automated Triage of Thyroid Cancer.

OBJECTIVE. The purpose of this study was to explore whether a quantitative framework can be used to sonographically differentiate benign and malignant thyroid nodules at a level comparable to that of experts. MATERIALS AND METHODS. A dataset of ultrasound images of 92 biopsy-confirmed nodules was collected retrospectively. The nodules were delineated and annotated by two expert radiologists using the standardized Thyroid Imaging Reporting and Data System lexicon of the American College of Radiology. In the framework studied, quantitative features of echogenicity, texture, edge sharpness, and margin curvature properties of thyroid nodules were analyzed in a regularized logistic regression model to predict malignancy of a nodule. The framework was validated by leave-one-out cross-validation technique, and ROC AUC, sensitivity, and specificity were used to compare with those obtained with six expert annotation-based classifiers. RESULTS. The AUC of the proposed method was 0.828 (95% CI, 0.715-0.942), which was greater than or comparable to that of the expert classifiers, for which the AUC values ranged from 0.299 to 0.829 (p = 0.99). Use of the proposed framework could have avoided biopsy of 20 of 46 benign nodules in a curative strategy (at sensitivity of 1, statistically significantly higher than three expert classifiers) or helped identify 10 of 46 malignancies in a conservative strategy (at specificity of 1, statistically significantly higher than five expert classifiers). CONCLUSION. When the proposed quantitative framework was used, thyroid nodule malignancy was predicted at the level of expert classifiers. Such a framework may ultimately prove useful as the basis for a fully automated system of thyroid nodule triage.

Advancing Semantic Interoperability of Image Annotations: Automated Conversion of Non-standard Image Annotations in a Commercial PACS to the Annotation and Image Markup.

Sharing radiologic image annotations among multiple institutions is important in many clinical scenarios; however, interoperability is prevented because different vendors' PACS store annotations in non-standardized formats that lack semantic interoperability. Our goal was to develop software to automate the conversion of image annotations in a commercial PACS to the Annotation and Image Markup (AIM) standardized format and demonstrate the utility of this conversion for automated matching of lesion measurements across time points for cancer lesion tracking. We created a software module in Java to parse the DICOM presentation state (DICOM-PS) objects (that contain the image annotations) for imaging studies exported from a commercial PACS (GE Centricity v3.x). Our software identifies line annotations encoded within the DICOM-PS objects and exports the annotations in the AIM format. A separate Python script processes the AIM annotation files to match line measurements (on lesions) across time points by tracking the 3D coordinates of annotated lesions. To validate the interoperability of our approach, we exported annotations from Centricity PACS into ePAD (http://epad.stanford.edu) (Rubin et al., Transl Oncol 7(1):23-35, 2014), a freely available AIM-compliant workstation, and the lesion measurement annotations were correctly linked by ePAD across sequential imaging studies. As quantitative imaging becomes more prevalent in radiology, interoperability of image annotations gains increasing importance. Our work demonstrates that image annotations in a vendor system lacking standard semantics can be automatically converted to a standardized metadata format such as AIM, enabling interoperability and potentially facilitating large-scale analysis of image annotations and the generation of high-quality labels for deep learning initiatives. This effort could be extended for use with other vendors' PACS.

Toward Data-Driven Learning Healthcare Systems in Interventional Radiology: Implementation to Evaluate Venous Stent Patency.

We developed a code and data-driven system (learning healthcare system) for gleaning actionable clinical insight from interventional radiology (IR) data. To this end, we constructed a workflow for the collection, processing and analysis of electronic health record (EHR), imaging, and cancer registry data for a cohort of interventional radiology patients seen in the IR Clinic at our institution over a more than 20-year period. As part of this pipeline, we created a database in REDCap (VITAL) to store raw data, as collected by a team of clinical investigators and the Data Coordinating Center at our university. We developed a single, universal pre-processing codebank for our VITAL data in R; in addition, we also wrote widely extendable and easily modifiable analysis code in R that presents results from summary statistics, statistical tests, visualizations, Kaplan-Meier analyses, and Cox proportional hazard modeling, among other analysis techniques. We present our findings for a test case of supra versus infra-inguinal ligament stenting. The developed pre-processing and analysis pipelines were memory and speed-efficient, with both pipelines running in less than 2 min. Three different supra-inguinal ligament veins had a statistically significant improvement in vein diameters post-stenting versus pre-stenting, while no infra-inguinal ligament veins had a statistically significant improvement (due either to an insufficient sample size or a non-significant p value). However, infra-inguinal ligament stenting was not associated with worse restenosis or patency outcomes in either a univariate (summary-statistics and Kaplan-Meier based) or multivariate (Cox proportional hazard model based) analysis.

Reproductive Factors and Mammographic Density: Associations Among 24,840 Women and Comparison of Studies Using Digitized Film-Screen Mammography and Full-Field Digital Mammography.

Breast density is a modifiable factor that is strongly associated with breast cancer risk. We sought to understand the influence of newer technologies of full-field digital mammography (FFDM) on breast density research and to determine whether results are comparable across studies using FFDM and previous studies using traditional film-screen mammography. We studied 24,840 screening-age (40-74 years) non-Hispanic white women who were participants in the Research Program on Genes, Environment and Health of Kaiser Permanente Northern California and underwent screening mammography with either Hologic (Hologic, Inc., Marlborough, Massachusetts) or General Electric (General Electric Company, Boston, Massachusetts) FFDM machines between 2003 and 2013. We estimated the associations of parity, age at first birth, age at menarche, and menopausal status with percent density and dense area as measured by a single radiological technologist using Cumulus software (Canto Software, Inc., San Francisco, California). We found that associations between reproductive factors and mammographic density measured using processed FFDM images were generally similar in magnitude and direction to those from prior studies using film mammography. Estimated associations for both types of FFDM machines were in the same direction. There was some evidence of heterogeneity in the magnitude of the effect sizes by machine type, which we accounted for using random-effects meta-analysis when combining results. Our findings demonstrate the robustness of quantitative mammographic density measurements across FFDM and film mammography platforms.

Assessment of Convolutional Neural Networks for Automated Classification of Chest Radiographs.

Purpose To assess the ability of convolutional neural networks (CNNs) to enable high-performance automated binary classification of chest radiographs. Materials and Methods In a retrospective study, 216 431 frontal chest radiographs obtained between 1998 and 2012 were procured, along with associated text reports and a prospective label from the attending radiologist. This data set was used to train CNNs to classify chest radiographs as normal or abnormal before evaluation on a held-out set of 533 images hand-labeled by expert radiologists. The effects of development set size, training set size, initialization strategy, and network architecture on end performance were assessed by using standard binary classification metrics; detailed error analysis, including visualization of CNN activations, was also performed. Results Average area under the receiver operating characteristic curve (AUC) was 0.96 for a CNN trained with 200 000 images. This AUC value was greater than that observed when the same model was trained with 2000 images (AUC = 0.84, P < .005) but was not significantly different from that observed when the model was trained with 20 000 images (AUC = 0.95, P > .05). Averaging the CNN output score with the binary prospective label yielded the best-performing classifier, with an AUC of 0.98 (P < .005). Analysis of specific radiographs revealed that the model was heavily influenced by clinically relevant spatial regions but did not reliably generalize beyond thoracic disease. Conclusion CNNs trained with a modestly sized collection of prospectively labeled chest radiographs achieved high diagnostic performance in the classification of chest radiographs as normal or abnormal; this function may be useful for automated prioritization of abnormal chest radiographs. © RSNA, 2018 Online supplemental material is available for this article. See also the editorial by van Ginneken in this issue.

Automatic inference of BI-RADS final assessment categories from narrative mammography report findings.

We propose an efficient natural language processing approach for inferring the BI-RADS final assessment categories by analyzing only the mammogram findings reported by the mammographer in narrative form. The proposed hybrid method integrates semantic term embedding with distributional semantics, producing a context-aware vector representation of unstructured mammography reports. A large corpus of unannotated mammography reports (300,000) was used to learn the context of the key-terms using a distributional semantics approach, and the trained model was applied to generate context-aware vector representations of the reports annotated with BI-RADS category (22,091). The vectorized reports were utilized to train a supervised classifier to derive the BI-RADS assessment class. Even though the majority of the proposed embedding pipeline is unsupervised, the classifier was able to recognize substantial semantic information for deriving the BI-RADS categorization not only on a holdout internal testset and also on an external validation set (1900 reports). Our proposed method outperforms a recently published domain-specific rule-based system and could be relevant for evaluating concordance between radiologists. With minimal requirement for task specific customization, the proposed method can be easily transferable to a different domain to support large scale text mining or derivation of patient phenotype.

Automated Detection of Measurements and Their Descriptors in Radiology Reports Using a Hybrid Natural Language Processing Algorithm.

Radiological measurements are reported in free text reports, and it is challenging to extract such measures for treatment planning such as lesion summarization and cancer response assessment. The purpose of this work is to develop and evaluate a natural language processing (NLP) pipeline that can extract measurements and their core descriptors, such as temporality, anatomical entity, imaging observation, RadLex descriptors, series number, image number, and segment from a wide variety of radiology reports (MR, CT, and mammogram). We created a hybrid NLP pipeline that integrates rule-based feature extraction modules and conditional random field (CRF) model for extraction of the measurements from the radiology reports and links them with clinically relevant features such as anatomical entities or imaging observations. The pipeline was trained on 1117 CT/MR reports, and performance of the system was evaluated on an independent set of 100 expert-annotated CT/MR reports and also tested on 25 mammography reports. The system detected 813 out of 806 measurements in the CT/MR reports; 784 were true positives, 29 were false positives, and 0 were false negatives. Similarly, from the mammography reports, 96% of the measurements with their modifiers were extracted correctly. Our approach could enable the development of computerized applications that can utilize summarized lesion measurements from radiology report of varying modalities and improve practice by tracking the same lesions along multiple radiologic encounters.

Magnetic resonance imaging and molecular features associated with tumor-infiltrating lymphocytes in breast cancer.

BACKGROUND: We sought to investigate associations between dynamic contrast-enhanced (DCE) magnetic resonance imaging (MRI) features and tumor-infiltrating lymphocytes (TILs) in breast cancer, as well as to study if MRI features are complementary to molecular markers of TILs. METHODS: In this retrospective study, we extracted 17 computational DCE-MRI features to characterize tumor and parenchyma in The Cancer Genome Atlas cohort (n = 126). The percentage of stromal TILs was evaluated on H&E-stained histological whole-tumor sections. We first evaluated associations between individual imaging features and TILs. Multiple-hypothesis testing was corrected by the Benjamini-Hochberg method using false discovery rate (FDR). Second, we implemented LASSO (least absolute shrinkage and selection operator) and linear regression nested with tenfold cross-validation to develop an imaging signature for TILs. Next, we built a composite prediction model for TILs by combining imaging signature with molecular features. Finally, we tested the prognostic significance of the TIL model in an independent cohort (I-SPY 1; n = 106). RESULTS: Four imaging features were significantly associated with TILs (P < 0.05 and FDR < 0.2), including tumor volume, cluster shade of signal enhancement ratio (SER), mean SER of tumor-surrounding background parenchymal enhancement (BPE), and proportion of BPE. Among molecular and clinicopathological factors, only cytolytic score was correlated with TILs (ρ = 0.51; 95% CI, 0.36-0.63; P = 1.6E-9). An imaging signature that linearly combines five features showed correlation with TILs (ρ = 0.40; 95% CI, 0.24-0.54; P = 4.2E-6). A composite model combining the imaging signature and cytolytic score improved correlation with TILs (ρ = 0.62; 95% CI, 0.50-0.72; P = 9.7E-15). The composite model successfully distinguished low vs high, intermediate vs high, and low vs intermediate TIL groups, with AUCs of 0.94, 0.76, and 0.79, respectively. During validation (I-SPY 1), the predicted TILs from the imaging signature separated patients into two groups with distinct recurrence-free survival (RFS), with log-rank P = 0.042 among triple-negative breast cancer (TNBC). The composite model further improved stratification of patients with distinct RFS (log-rank P = 0.0008), where TNBC with no/minimal TILs had a worse prognosis. CONCLUSIONS: Specific MRI features of tumor and parenchyma are associated with TILs in breast cancer, and imaging may play an important role in the evaluation of TILs by providing key complementary information in equivocal cases or situations that are prone to sampling bias.

Non-Small Cell Lung Cancer Radiogenomics Map Identifies Relationships between Molecular and Imaging Phenotypes with Prognostic Implications.

Purpose To create a radiogenomic map linking computed tomographic (CT) image features and gene expression profiles generated by RNA sequencing for patients with non-small cell lung cancer (NSCLC). Materials and Methods A cohort of 113 patients with NSCLC diagnosed between April 2008 and September 2014 who had preoperative CT data and tumor tissue available was studied. For each tumor, a thoracic radiologist recorded 87 semantic image features, selected to reflect radiologic characteristics of nodule shape, margin, texture, tumor environment, and overall lung characteristics. Next, total RNA was extracted from the tissue and analyzed with RNA sequencing technology. Ten highly coexpressed gene clusters, termed metagenes, were identified, validated in publicly available gene-expression cohorts, and correlated with prognosis. Next, a radiogenomics map was built that linked semantic image features to metagenes by using the t statistic and the Spearman correlation metric with multiple testing correction. Results RNA sequencing analysis resulted in 10 metagenes that capture a variety of molecular pathways, including the epidermal growth factor (EGF) pathway. A radiogenomic map was created with 32 statistically significant correlations between semantic image features and metagenes. For example, nodule attenuation and margins are associated with the late cell-cycle genes, and a metagene that represents the EGF pathway was significantly correlated with the presence of ground-glass opacity and irregular nodules or nodules with poorly defined margins. Conclusion Radiogenomic analysis of NSCLC showed multiple associations between semantic image features and metagenes that represented canonical molecular pathways, and it can result in noninvasive identification of molecular properties of NSCLC. Online supplemental material is available for this article.

Intratumoral Spatial Heterogeneity at Perfusion MR Imaging Predicts Recurrence-free Survival in Locally Advanced Breast Cancer Treated with Neoadjuvant Chemotherapy.

Purpose To characterize intratumoral spatial heterogeneity at perfusion magnetic resonance (MR) imaging and investigate intratumoral heterogeneity as a predictor of recurrence-free survival (RFS) in breast cancer. Materials and Methods In this retrospective study, a discovery cohort (n = 60) and a multicenter validation cohort (n = 186) were analyzed. Each tumor was divided into multiple spatially segregated, phenotypically consistent subregions on the basis of perfusion MR imaging parameters. The authors first defined a multiregional spatial interaction (MSI) matrix and then, based on this matrix, calculated 22 image features. A network strategy was used to integrate all image features and classify patients into different risk groups. The prognostic value of imaging-based stratification was evaluated in relation to clinical-pathologic factors with multivariable Cox regression. Results Three intratumoral subregions with high, intermediate, and low MR perfusion were identified and showed high consistency between the two cohorts. Patients in both cohorts were stratified according to network analysis of multiregional image features regarding RFS (log-rank test, P = .002 for both). Aggressive tumors were associated with a larger volume of the poorly perfused subregion as well as interaction between poorly and moderately perfused subregions and surrounding parenchyma. At multivariable analysis, the proposed MSI-based marker was independently associated with RFS (hazard ratio: 3.42; 95% confidence interval: 1.55, 7.57; P = .002) adjusting for age, estrogen receptor (ER) status, progesterone receptor status, human epidermal growth factor receptor type 2 (HER2) status, tumor volume, and pathologic complete response (pCR). Furthermore, imaging helped stratify patients for RFS within the ER-positive and HER2-positive subgroups (log-rank test, P = .007 and .004) and among patients without pCR after neoadjuvant chemotherapy (log-rank test, P = .003). Conclusion Breast cancer consists of multiple spatially distinct subregions. Imaging heterogeneity is an independent prognostic factor beyond traditional risk predictors.