Recommendations for Additional Imaging on Head and Neck Imaging Examinations: Interradiologist Variation and Associated Factors.

BACKGROUND. A paucity of relevant guidelines may lead to pronounced variation among radiologists in issuing recommendations for additional imaging (RAI) for head and neck imaging. OBJECTIVE. The purpose of this article was to explore associations of RAI for head and neck imaging examinations with examination, patient, and radiologist factors and to assess the role of individual radiologist-specific behavior in issuing such RAI. METHODS. This retrospective study included 39,200 patients (median age, 58 years; 21,855 women, 17,315 men, 30 with missing sex information) who underwent 39,200 head and neck CT or MRI examinations, interpreted by 61 radiologists, from June 1, 2021, through May 31, 2022. A natural language processing (NLP) tool with manual review of NLP results was used to identify RAI in report impressions. Interradiologist variation in RAI rates was assessed. A generalized mixed-effects model was used to assess associations between RAI and examination, patient, and radiologist factors. RESULTS. A total of 2943 (7.5%) reports contained RAI. Individual radiologist RAI rates ranged from 0.8% to 22.0% (median, 7.1%; IQR, 5.2-10.2%), representing a 27.5-fold difference between minimum and a maximum values and 1.8-fold difference between 25th and 75th percentiles. In multivariable analysis, RAI likelihood was higher for CTA than for CT examinations (OR, 1.32), for examinations that included a trainee in report generation (OR, 1.23), and for patients with self-identified race of Black or African American versus White (OR, 1.25); was lower for male than female patients (OR, 0.90); and was associated with increasing patient age (OR, 1.09 per decade) and inversely associated with radiologist years since training (OR, 0.90 per 5 years). The model accounted for 10.9% of the likelihood of RAI. Of explainable likelihood of RAI, 25.7% was attributable to examination, patient, and radiologist factors; 74.3% was attributable to radiologist-specific behavior. CONCLUSION. Interradiologist variation in RAI rates for head and neck imaging was substantial. RAI appear to be more substantially associated with individual radiologist-specific behavior than with measurable systemic factors. CLINICAL IMPACT. Quality improvement initiatives, incorporating best practices for incidental findings management, may help reduce radiologist preference-sensitive decision-making in issuing RAI for head and neck imaging and associated care variation.

Risk of Malignancy in Incidentally Detected Lung Nodules in Patients Aged Younger Than 35 Years.

BACKGROUND: The risk of malignancy in pulmonary nodules incidentally detected on computed tomography (CT) in patients who are aged younger than 35 years is unclear. OBJECTIVE: The aim of this study was to evaluate the incidence of lung cancer in incidental pulmonary nodules in patients who are 15-34 years old. METHODS: This retrospective study included patients aged 15-34 years who had an incidental pulmonary nodule on chest CT from 2010 to 2018 at our hospital. Patients with prior, current, or suspected malignancy were excluded. A chart review identified patients with diagnosis of malignancy. Incidental pulmonary nodule was deemed benign if stable or resolved on a follow-up CT at least 2 years after initial or if there was a medical visit in our health care network at least 2 years after initial CT without diagnosis of malignancy.Receiver operating characteristic curve analysis was performed with nodule size. Association of categorical variables with lung cancer diagnosis was performed with Fisher exact test, and association of continuous variables was performed with logistic regression. RESULTS: Five thousand three hundred fifty-five chest CTs performed on patients aged 15-34 years between January 2010 and December 2018. After excluding patients without a reported pulmonary nodule and prior or current malignancy, there were a total of 779 patients. Of these, 690 (89%) had clinical or imaging follow-up after initial imaging. Of these, 545 (70% of total patients) patients had imaging or clinical follow-up greater than 2 years after their initial imaging.A malignant diagnosis was established in 2/779 patients (0.3%; 95% confidence interval, 0.1%-0.9%). Nodule size was strongly associated with malignancy (P = 0.007), with area under the receiver operating characteristic curve of 0.97. There were no malignant nodules that were less than 10 mm in size. Smoking history, number of nodules, and nodule density were not associated with malignancy. CONCLUSIONS: Risk of malignancy for incidentally detected pulmonary nodules in patients aged 15-34 years is extremely small (0.3%). There were no malignant nodules that were less than 10 mm in size. Routine follow-up of subcentimeter pulmonary nodules should be carefully weighed against the risks.

Frequency and Outcomes of BI-RADS Category 3 Assessments in Patients With a Personal History of Breast Cancer: Full-Field Digital Mammography Versus Digital Breast Tomosynthesis.

BACKGROUND. Studies establishing the validity of BI-RADS category 3 excluded patients with personal history of breast cancer (PHBC). Use of category 3 in patients with PHBC may be impacted not only by this population's increased breast cancer risk, but also by adoption of digital breast tomosynthesis (DBT) over full-field digital mammography (FFDM). OBJECTIVE. The purpose of this article was to compare the frequency, outcomes, and additional characteristics of BI-RADS category 3 assessments between FFDM and DBT in patients with PHBC. METHODS. This retrospective study included 14,845 mammograms in 10,118 patients (mean age, 63 years) with PHBC who had undergone mastectomy and/or lumpectomy. Of these, 8422 examinations were performed by FFDM from October 2014 to September 2016, and 6423 examinations by FFDM with DBT from February 2017 to December 2018, after interval conversion of the center's mammography units. Information was extracted from the EHR and radiology reports. FFDM and DBT groups were compared in the entire sample and among index category 3 lesions (i.e., earliest category 3 assessment per lesion). RESULTS. The frequency of category 3 assessment was lower for DBT than FFDM (5.6% vs 6.4%; p = .05). DBT, compared with FFDM, showed a lower malignancy rate for category 3 lesions (1.8% vs 5.0%; p = .04), higher malignancy rate for category 4 lesions (32.0% vs 23.2%; p = .03), and no difference in malignancy rate for category 5 lesions (100.0% vs 75.0%; p = .24). Analysis of index category 3 lesions included 438 and 274 lesions for FFDM and DBT, respectively. For category 3 lesions, DBT, compared with FFDM, showed lower PPV3 (13.9% vs 36.1%; p = .02) and a more frequent mammographic finding of mass (33.2% vs 23.1%; p = .003). CONCLUSION. The malignancy rate for category 3 lesions in patients with PHBC was less than the accepted limit (2%) for DBT (1.8%), but not FFDM (5.0%). A lower malignancy rate for category 3 lesions but higher malignancy rate for category 4 lesions for DBT supports more appropriate application of category 3 assessment in patients with PHBC through use of DBT. CLINICAL IMPACT. These insights may help establish whether category 3 assessments in patients with PHBC are within benchmarks for early detection of second cancers and reduction of benign biopsies.

Predictors of Completion of Clinically Necessary Radiologist-Recommended Follow-Up Imaging: Assessment Using an Automated Closed-Loop Communication and Tracking Tool.

BACKGROUND. Patients with adverse social determinants of health may be at increased risk of not completing clinically necessary follow-up imaging. OBJECTIVE. The purpose of this study was to use an automated closed-loop communication and tracking tool to identify patient-, referrer-, and imaging-related factors associated with lack of completion of radiologist-recommended follow-up imaging. METHODS. This retrospective study was performed at a single academic health system. A tool for automated communication and tracking of radiologist-recommended follow-up imaging was embedded in the PACS and electronic health record. The tool prompted referrers to record whether they deemed recommendations to be clinically necessary and assessed whether clinically necessary follow-up imaging was pursued. If imaging was not performed within 1 month after the intended completion date, the tool prompted a safety net team to conduct further patient and referrer follow-up. The study included patients for whom a follow-up imaging recommendation deemed clinically necessary by the referrer was entered with the tool from October 21, 2019, through June 30, 2021. The electronic health record was reviewed for documentation of eventual completion of the recommended imaging at the study institution or an outside institution. Multivariable logistic regression analysis was performed to identify factors associated with completion of follow-up imaging. RESULTS. Of 5856 recommendations entered during the study period, the referrer agreed with 4881 recommendations in 4599 patients (2929 women, 1670 men; mean age, 61.3 ± 15.6 years), who formed the study sample. Follow-up was completed for 74.8% (3651/4881) of recommendations. Independent predictors of lower likelihood of completing follow-up imaging included living in a socioeconomically disadvantaged neighborhood according to the area deprivation index (odds ratio [OR], 0.67 [95% CI, 0.54-0.84]), inpatient (OR, 0.25 [95% CI, 0.20-0.32]) or emergency department (OR, 0.09 [95% CI, 0.05-0.15]) care setting, and referrer surgical specialty (OR, 0.70 [95% CI, 0.58-0.84]). Patient age, race and ethnicity, primary language, and insurance status were not independent predictors of completing follow-up (p > .05). CONCLUSION. Socioeconomically disadvantaged patients are at increased risk of not completing recommended follow-up imaging that referrers deem clinically necessary. CLINICAL IMPACT. Initiatives for ensuring completion of follow-up imaging should be aimed at the identified patient groups to reduce disparities in missed and delayed diagnoses.

Development and External Validation of an Artificial Intelligence Model for Identifying Radiology Reports Containing Recommendations for Additional Imaging.

BACKGROUND. Reported rates of recommendations for additional imaging (RAIs) in radiology reports are low. Bidirectional encoder representations from transformers (BERT), a deep learning model pretrained to understand language context and ambiguity, has potential for identifying RAIs and thereby assisting large-scale quality improvement efforts. OBJECTIVE. The purpose of this study was to develop and externally validate an artificial intelligence (AI)-based model for identifying radiology reports containing RAIs. METHODS. This retrospective study was performed at a multisite health center. A total of 6300 radiology reports generated at one site from January 1, 2015, to June 30, 2021, were randomly selected and split by 4:1 ratio to create training (n = 5040) and test (n = 1260) sets. A total of 1260 reports generated at the center's other sites (including academic and community hospitals) from April 1 to April 30, 2022, were randomly selected as an external validation group. Referring practitioners and radiologists of varying sub-specialties manually reviewed report impressions for presence of RAIs. A BERT-based technique for identifying RAIs was developed by use of the training set. Performance of the BERT-based model and a previously developed traditional machine learning (TML) model was assessed in the test set. Finally, performance was assessed in the external validation set. The code for the BERT-based RAI model is publicly available. RESULTS. Among a total of 7419 unique patients (4133 women, 3286 men; mean age, 58.8 years), 10.0% of 7560 reports contained RAI. In the test set, the BERT-based model had 94.4% precision, 98.5% recall, and an F1 score of 96.4%. In the test set, the TML model had 69.0% precision, 65.4% recall, and an F1 score of 67.2%. In the test set, accuracy was greater for the BERT-based than for the TML model (99.2% vs 93.1%, p < .001). In the external validation set, the BERT-based model had 99.2% precision, 91.6% recall, an F1 score of 95.2%, and 99.0% accuracy. CONCLUSION. The BERT-based AI model accurately identified reports with RAIs, outperforming the TML model. High performance in the external validation set suggests the potential for other health systems to adapt the model without requiring institution-specific training. CLINICAL IMPACT. The model could potentially be used for real-time EHR monitoring for RAIs and other improvement initiatives to help ensure timely performance of clinically necessary recommended follow-up.

Comparing Tumor Characteristics and Rates of Breast Cancers Detected by Screening Digital Breast Tomosynthesis and Full-Field Digital Mammography.

OBJECTIVE. The purpose of this study was to compare the cancer detection rates (CDRs), tumor types, and characteristics between screening digital breast tomosynthesis (DBT) and screening full-field digital mammography (FFDM) in a matched patient population in a large academic breast imaging practice with mixed DBT and FFDM technology. MATERIALS AND METHODS. In this retrospective study, we reviewed consecutive screening FFDM and DBT examinations performed between October 2012 and September 2014. To control for nonrandomized selection of FFDM versus DBT examinations, we applied propensity score matching on the basis of patient age, imaging site, and prior imaging findings. An institutional breast cancer registry identified cancer diagnoses. CDR and tumor type, grade, receptor, nodal status, and size were compared between matched FFDM and DBT groups. RESULTS. Sixty-one cancers were detected in the matched screening cohort of DBT (n = 9817) and FFDM (n = 14,180) examinations. CDR was higher with DBT than with FFDM for invasive cancers (2.8 vs 1.3, p = 0.01), minimal cancers (2.4 vs 1.2, p = 0.03), estrogen receptor-positive invasive cancers (2.6 vs 1.1, p = 0.01), and node-negative invasive cancers (2.3 vs 1.1, p = 0.02.), respectively. The ratio of screen-detected invasive cancers to ductal carcinoma in situ on DBT (3.0) was not significantly different from that on FFDM (2.6) (p = 0.79). CONCLUSION. DBT results in an overall increase in CDR irrespective of the tumor type, size, or grade of cancer.

Variation in Follow-up Imaging Recommendations in Radiology Reports: Patient, Modality, and Radiologist Predictors.

Background Variation between radiologists when making recommendations for additional imaging and associated factors are, to the knowledge of the authors, unknown. Clear identification of factors that account for variation in follow-up recommendations might prevent unnecessary tests for incidental or ambiguous image findings. Purpose To determine incidence and identify factors associated with follow-up recommendations in radiology reports from multiple modalities, patient care settings, and imaging divisions. Materials and Methods This retrospective study analyzed 318 366 reports obtained from diagnostic imaging examinations performed at a large urban quaternary care hospital from January 1 to December 31, 2016, excluding breast and US reports. A subset of 1000 reports were randomly selected and manually annotated to train and validate a machine learning algorithm to predict whether a report included a follow-up imaging recommendation (training-and-validation set consisted of 850 reports and test set of 150 reports). The trained algorithm was used to classify 318 366 reports. Multivariable logistic regression was used to determine the likelihood of follow-up recommendation. Additional analysis by imaging subspecialty division was performed, and intradivision and interradiologist variability was quantified. Results The machine learning algorithm classified 38 745 of 318 366 (12.2%) reports as containing follow-up recommendations. Average patient age was 59 years ± 17 (standard deviation); 45.2% (143 767 of 318 366) of reports were from male patients. Among 65 radiologists, 57% (37 of 65) were men. At multivariable analysis, older patients had higher rates of follow-up recommendations (odds ratio [OR], 1.01 [95% confidence interval {CI}: 1.01, 1.01] for each additional year), male patients had lower rates of follow-up recommendations (OR, 0.9; 95% CI: 0.9, 1.0), and follow-up recommendations were most common among CT studies (OR, 4.2 [95% CI: 4.0, 4.4] compared with radiography). Radiologist sex (P = .54), presence of a trainee (P = .45), and years in practice (P = .49) were not significant predictors overall. A division-level analysis showed 2.8-fold to 6.7-fold interradiologist variation. Conclusion Substantial interradiologist variation exists in the probability of recommending a follow-up examination in a radiology report, after adjusting for patient, examination, and radiologist factors. © RSNA, 2019 See also the editorial by Russell in this issue.

Adoption of a Closed-Loop Communication Tool to Establish and Execute a Collaborative Follow-Up Plan for Incidental Pulmonary Nodules.

OBJECTIVE. The purpose of this study is to assess radiologists' adoption of a closed-loop communication and tracking system, Result Alert and Development of Automated Resolution (RADAR), for incidental pulmonary nodules and to measure its effect on the completeness of radiologists' follow-up recommendations. MATERIALS AND METHODS. This retrospective study was performed at a tertiary academic center that performs more than 600,000 radiology examinations annually. Before RADAR, the institution's standard of care was for radiologists to generate alerts for newly discovered incidental pulmonary nodules using a previously described PACS-embedded software tool. RADAR is a new closed-loop communication tool embedded in the PACS and enterprise provider workflow that enables establishing a collaborative follow-up plan between a radiologist and referring provider and helps automate collaborative follow-up plan tracking and execution. We assessed RADAR adoption for incidental pulmonary nodules, the primary outcome, in our thoracic radiology division (study period March 9, 2018, through August 2, 2018). The secondary outcome was the completeness of follow-up recommendation for incidental pulmonary nodules, defined as explicit imaging modality and time frame for follow-up. RESULTS. After implementation, 106 of 183 (58%) incidental pulmonary nodules alerts were generated using RADAR. RADAR adoption increased by 75% during the study period (40% in the first 3 weeks vs 70% in the last 3 weeks; p < 0.001 test for trend). All RADAR alerts had explicit documentation of imaging modality and follow-up time frame, compared with 71% for non-RADAR alerts for incidental pulmonary nodules (p < 0.001). CONCLUSION. A closed-loop communication system that enables establishing and executing a collaborative follow-up plan for incidental pulmonary nodules can be adopted and improves the quality of radiologists' follow-up recommendations.

Risk Stratification Model: Lower-Extremity Ultrasonography for Hospitalized Patients with Suspected Deep Vein Thrombosis.

BACKGROUND: The Wells score for deep venous thrombosis (DVT) has a high failure rate and low efficiency among inpatients. OBJECTIVE: To create and validate an inpatient-specific risk stratification model to help assess pre-test probability of DVT in hospitalized patients. DESIGN: Prospective cohort study of hospitalized patients undergoing lower-extremity ultrasonography studies (LEUS) for suspected DVT. Demographics, physical findings, medical history, medications, hospitalization, and laboratory and imaging results were collected. Samples were divided into model derivation (patients undergoing LEUS 11/1/2012-12/31/2013) and validation cohorts (LEUS 1/1/2014-5/31/2015). A DVT prediction rule was derived using the recursive partitioning algorithm (decision tree-type approach) and was then validated. PARTICIPANTS: Adult inpatients undergoing LEUS for suspected DVT from November 2012 to May 2015, excluding those with DVT in the prior 3 months, at a 793-bed, urban academic quaternary-care hospital with ~50,000 admissions annually. MAIN MEASURES: The primary outcome was the presence of proximal DVT, and the secondary outcome was the presence of any DVT (proximal or distal). Model sensitivity and specificity for predicting DVT were calculated. KEY RESULTS: Recursive partitioning yielded four variables (previous DVT, active cancer, hospitalization ≥ 6 days, age ≥ 46 years) that optimized the prediction of proximal DVT and yield in the derivation cohort. From this decision tree, we stratified a scoring system using the validation cohort, categorizing patients into low- and high-risk groups. The incidence rates of proximal DVT were 2.9% and 12.0%, and of any DVT were 5.2% and 21.0%, for the low- and high-risk groups, respectively. The AUC for the discriminatory accuracy of the Center for Evidence-Based Imaging (CEBI) score for risk of proximal DVT identified on LEUS was 0.73. Model sensitivity was 98.1% for proximal and 98.1% for any DVT. CONCLUSIONS: In hospitalized adults, specific factors can help clinicians predict risk of DVT, identifying those with low pre-test probability, in whom ultrasonography can be safely avoided.

Assessing Documentation of Critical Imaging Result Follow-up Recommendations in Emergency Department Discharge Instructions.

To facilitate follow-up of critical test results across transitions in patient care settings, we implemented an electronic discharge module that enabled care providers to include follow-up recommendations in the discharge instructions. We assessed the impact of this module on documentation of follow-up recommendations for critical imaging findings in Emergency Department (ED) discharge instructions. We studied 240 patients with critical imaging findings discharged from the ED before (n = 80) and after (n = 160) implementation of the module. We manually reviewed hand-written forms and electronic discharge instructions to determine if follow-up recommendations were documented. Follow-up recommendations in ED discharge instructions increased from 60.0% (48/80) to 73.8% (118/160) post-module implementation (p = 0.03), a relative increase of 23%. There was no significant change in the rate of documented critical imaging findings in the discharge instructions (77.5% [62/80] before the intervention and 76.9% [123/160] after the intervention; p = 0.91). Implementation of a discharge module was associated with increased documentation of critical imaging finding follow-up recommendations in ED discharge instructions. However, one in four patients still did not receive adequate follow-up recommendations, suggesting further opportunities for performance improvement exist.

Impact of a Health Information Technology Intervention on the Follow-up Management of Pulmonary Nodules.

Lung cancer is the leading cause of cancer deaths in the USA. The most common abnormalities suspicious for lung cancer on CT scan include pulmonary nodules. Recommendations to improve care for patients with pulmonary nodules require follow-up management. However, transitions in care, especially for patients undergoing transitions to ambulatory care sites from the emergency department (ED) and inpatient settings, can exacerbate failures in follow-up testing and compromise patient safety. We evaluate the impact of a discharge module that includes follow-up recommendations for further management of pulmonary nodules on the study outcome and follow-up management of patients with pulmonary nodules within 1 year after discharge. After IRB approval, we collected data on all patients undergoing chest or abdominal CT exams over a 12-month baseline and 12-month intervention period at an academic medical center. The inpatient discharge module was implemented in November 2011; the ED module was implemented in May 2012. Multivariable logistic regression was performed to account for care setting, imaging modality, recommendations, and patient demographics. Implementation of a discharge module resulted in improved follow-up of patients with pulmonary nodules within 1 year after discharge (OR = 1.64, p = 0.01); the ED implementation resulted in better follow-up compared to the inpatient module (OR = 2.24, p < 0.01). Twenty-seven percent of patients with pulmonary nodules received follow-up management, which, although significantly improved from the 18% baseline, remains low. An electronic discharge module is associated with improved follow-up management of patients with pulmonary nodules, and may be combined with interventions to further improve management of these patients.

Comparing Diagnostic Performance of Digital Breast Tomosynthesis and Full-Field Digital Mammography in a Hybrid Screening Environment.

OBJECTIVE: The purpose of this study is to compare the diagnostic performance of screening digital breast tomosynthesis (DBT) to that of full-field digital mammography (FFDM) in a mixed DBT and FFDM imaging environment. MATERIALS AND METHODS: This retrospective observational study consisted of all female patients undergoing screening DBT or FFDM at an academic medical center and outpatient imaging facility between October 2012 and May 2015. Patient demographics and personal history of breast cancer were collected from the electronic medical record. A natural language processing algorithm extracted patients' breast density, current or prior imaging findings, and BI-RADS category from their most recent prior imaging examinations. To control for differential selection of FFDM versus DBT, we applied propensity score matching based on patient age, imaging site, and prior imaging findings. An institutional breast cancer registry identified cancer diagnoses. Primary outcomes of recall rate, cancer detection rate, and positive predictive value 1 (PPV1) were compared between matched FFDM and DBT groups. RESULTS: Among 68,794 screening examinations, we matched 16,264 FFDM with 21,074 DBT examinations (total, 37,338 examinations) using nearest neighbor propensity score matching. Recall rates were 10.3% (1683/16,264) for FFDM and 10.7% (2254/21,074) for DBT (p = 0.26). Cancer detection rates (number of cancers/1000 examinations) were 1.8/1000 for FFDM and 3.8/1000 for DBT (p = 0.005). The PPV1 (number of cancers/number of recalls) was 1.8% (26/1478) for FFDM and 3.6% (37/1036) for DBT (p = 0.006). CONCLUSION: FFDM and DBT recall rates were not significantly different in a mixed FFDM and DBT breast imaging practice. However, the PPV1 of recalled cases and the cancer detection rate (the primary screening objective) were significantly higher with DBT compared with FFDM.

Medicare Imaging Demonstration: Assessing Attributes of Appropriate Use Criteria and Their Influence on Ordering Behavior.

OBJECTIVE: Persistent concern exists about the variable and possibly inappropriate utilization of high-cost imaging tests. The purpose of this study is to assess the influence of appropriate use criteria attributes on altering ambulatory imaging orders deemed inappropriate. MATERIALS AND METHODS: This secondary analysis included Medicare Imaging Demonstration data collected from three health care systems in 2011-2013 via the use of clinical decision support (CDS) during ambulatory imaging order entry. The CDS system captured whether orders were inappropriate per the appropriate use criteria of professional societies and provided advice during the intervention period. For orders deemed inappropriate, we assessed the impact of the availability of alternative test recommendations, conflicts with local best practices, and the strength of evidence for appropriate use criteria on the primary outcome of cancellation or modification of inappropriate orders. Expert review determined conflicts with local best practices for 250 recommendations for abdominal and thoracic CT orders. Strength of evidence was assessed for the 15 most commonly triggered recommendations that were deemed inappropriate. A chi-square test was used for univariate analysis. RESULTS: A total of 1691 of 63,222 imaging test orders (2.7%) were deemed inappropriate during the intervention period; this amount decreased from 364 of 11,675 test orders (3.1%) in the baseline period (p < 0.00001). Of 270 inappropriate recommendations with alternative test recommendations, 28 (10.4%) were modified, compared with four of 1024 inappropriate recommendations without alternatives (0.4%) (p < 0.0001). Seventy-eight of 250 recommendations (31%) conflicted with local best practices, but only six of 69 inappropriate recommendations (9%) conflicted (p < 0.001). No inappropriate recommendations that conflicted with local best practices were modified. All 15 commonly triggered recommendations had an Oxford Centre for Evidence-Based Medicine level of evidence of 5 (i.e., expert opinion). CONCLUSION: Orders for imaging tests that were deemed inappropriate were modified infrequently, more often with alternative recommendations present and only for appropriate use criteria consistent with local best practices.

JOURNAL CLUB: Predictors of Provider Response to Clinical Decision Support: Lessons Learned From the Medicare Imaging Demonstration.

OBJECTIVE: The efficacy of imaging clinical decision support (CDS) varies. Our objective was to identify CDS factors contributing to imaging order cancellation or modification. SUBJECTS AND METHODS: This pre-post study was performed across four institutions participating in the Medicare Imaging Demonstration. The intervention was CDS at order entry for selected outpatient imaging procedures. On the basis of the information entered, computerized alerts indicated to providers whether orders were not covered by guidelines, appropriate, of uncertain appropriateness, or inappropriate according to professional society guidelines. Ordering providers could override or accept CDS. We considered actionable alerts to be those that could generate an immediate order behavior change in the ordering physician (i.e., cancellation of inappropriate orders or modification of orders of uncertain appropriateness that had a recommended alternative). Chi-square and logistic regression identified predictors of order cancellation or modification after an alert. RESULTS: A total of 98,894 radiology orders were entered (83,114 after the intervention). Providers ignored 98.9%, modified 1.1%, and cancelled 0.03% of orders in response to alerts. Actionable alerts had a 10 fold higher rate of modification (8.1% vs 0.7%; p < 0.0001) or cancellation (0.2% vs 0.02%; p < 0.0001) orders compared with nonactionable alerts. Orders from institutions with preexisting imaging CDS had a sevenfold lower rate of cancellation or modification than was seen at sites with newly implemented CDS (1.4% vs 0.2%; p < 0.0001). In multivariate analysis, actionable alerts were 12 times more likely to result in order cancellation or modification. Orders at sites with preexisting CDS were 7.7 times less likely to be cancelled or modified (p < 0.0001). CONCLUSION: Using results from the Medicare Imaging Demonstration project, we identified potential factors that were associated with CDS effect on provider imaging ordering; these findings may have implications for future design of such computerized systems.

Variation in Mammographic Breast Density Assessments Among Radiologists in Clinical Practice: A Multicenter Observational Study.

Background: About half of the United States has legislation requiring radiology facilities to disclose mammographic breast density information to women, often with language recommending discussion of supplemental screening options for women with dense breasts. Objective: To examine variation in breast density assessment across radiologists in clinical practice. Design: Cross-sectional and longitudinal analyses of prospectively collected observational data. Setting: 30 radiology facilities within the 3 breast cancer screening research centers of the Population-based Research Optimizing Screening through Personalized Regimens (PROSPR) consortium. Participants: Radiologists who interpreted at least 500 screening mammograms during 2011 to 2013 (n = 83). Data on 216 783 screening mammograms from 145 123 women aged 40 to 89 years were included. Measurements: Mammographic breast density, as clinically recorded using the 4 Breast Imaging Reporting and Data System categories (heterogeneously dense and extremely dense categories were considered "dense" for analyses), and patient age, race, and body mass index (BMI). Results: Overall, 36.9% of mammograms were rated as showing dense breasts. Across radiologists, this percentage ranged from 6.3% to 84.5% (median, 38.7% [interquartile range, 28.9% to 50.9%]), with multivariable adjustment for patient characteristics having little effect (interquartile range, 29.9% to 50.8%). Examination of patient subgroups revealed that variation in density assessment across radiologists was pervasive in all but the most extreme patient age and BMI combinations. Among women with consecutive mammograms interpreted by different radiologists, 17.2% (5909 of 34 271) had discordant assessments of dense versus nondense status. Limitation: Quantitative measures of mammographic breast density were not available for comparison. Conclusion: There is wide variation in density assessment across radiologists that should be carefully considered by providers and policymakers when considering supplemental screening strategies. The likelihood of a woman being told she has dense breasts varies substantially according to which radiologist interprets her mammogram. Primary Funding Source: National Institutes of Health.

Assessing Inaccuracies in Automated Information Extraction of Breast Imaging Findings.

We previously identified breast imaging findings from radiology reports using an expert-based information extraction algorithm as part of the National Cancer Institute's Population-based Research Optimizing Screening through Personalized Regimens (PROSPR) initiative. We validate this algorithm and assess inaccuracies in a different institutional setting. Mammography, ultrasound (US), and breast magnetic resonance imaging (MRI) reports of patients at an academic health system between 4/2013 and 6/2013 were included for analysis. Accuracy of automatically extracting imaging findings using an algorithm developed at a different institution compared to manual gold standard review is reported. Extraction errors are further categorized based on manual review. Precision and recall for extracting BI-RADS categories remain between 0.9 and 1.0, except for MRI (0.7). F measures for extracting other findings are 0.9 for non-mass enhancement (in MRI) and 0.8-0.9 for cysts (in MRI and US). Extracting breast imaging findings resulted in lowest accuracy for findings of calcification (range 0.4-0.6 in mammography) and asymmetric density (0.5-0.7 in mammography). Majority of errors for extracting imaging findings were due to qualifier-based errors, descriptors which indicate absence of findings, missed by automated extraction (e.g., "benign" calcifications). Our information extraction algorithm provides an effective approach to extracting some breast imaging findings for populating a breast screening registry. However, errors in information extraction when utilizing methods in new settings demonstrate that further work is necessary to extract information content from unstructured multi-institutional radiology reports.

Gastrointestinal stromal tumor: optimizing the use of cross-sectional chest imaging during follow-up.

PURPOSE: To identify the frequency of and variables associated with thoracic metastasis in patients with gastrointestinal stromal tumor (GIST) to help optimize the use of cross-sectional chest imaging. MATERIALS AND METHODS: This retrospective institutional review board-approved study included 631 patients (343 men; mean age, 55 years; range, 19-94 years) with pathologically confirmed GIST who were identified with a natural language processing algorithm in a review of radiologic reports from January 2004 through October 2012, followed by manual confirmation. The requirement for informed consent was waived. Available imaging, pathologic, and clinical records were reviewed to confirm the presence of abdominal and thoracic metastases. The association of age; sex; size, location, mitotic count, and risk stratification of the primary tumor; initial treatment; presence of abdominal metastases; and bulky abdominal metastases (more than 10 lesions larger than 1 cm, or more than five lesions with at least one larger than 5 cm) with development of thoracic metastases, the primary outcome measure, was studied by using logistic regression. RESULTS: During median follow-up of 61.4 months (interquartile range, 37.8-93 months), 401 of 631 (63.5%) patients developed metastatic disease (median interval, 6.9 months; interquartile range, 0-25.6 months), all with peritoneal (n = 324) and/or hepatic metastases (n = 303). Bulky abdominal metastases were found in 218 (34.5%) patients. Although 579 (91.8%) patients underwent chest imaging, only 64 of 631 (10.1%) developed thoracic metastases (median, 51.4 months; interquartile range, 36-78.7 months); all had bulky abdominal metastases except one patient who presented with symptomatic scapular metastasis. Only bulky abdominal metastasis was significantly associated with the development of thoracic metastasis (P < .0001; odds ratio, 42.6; range, 8.6-211.5). CONCLUSION: Thoracic metastases are relatively uncommon in patients with GIST and are significantly associated only with presence of bulky abdominal metastases.

Smart stylet: the development and use of a bedside external ventricular drain image-guidance system.

BACKGROUND: Placement accuracy of ventriculostomy catheters is reported in a wide and variable range. Development of an efficient image-guidance system may improve physician performance and patient safety. OBJECTIVE: We evaluate the prototype of Smart Stylet, a new electromagnetic image-guidance system for use during bedside ventriculostomy. METHODS: Accuracy of the Smart Stylet system was assessed. System operators were evaluated for their ability to successfully target the ipsilateral frontal horn in a phantom model. RESULTS: Target registration error across 15 intracranial targets ranged from 1.3 to 4.6 mm (mean 3.1 mm). Using Smart Stylet guidance, a test operator successfully passed a ventriculostomy catheter to a shifted ipsilateral frontal horn 20/20 (100%) times from the frontal approach in a skull phantom. Without Smart Stylet guidance, the operator was successful 4/10 (40%) times from the right frontal approach and 6/10 (60%) times from the left frontal approach. In a separate experiment, resident operators were successful 2/4 (50%) times when targeting the shifted ipsilateral frontal horn with Smart Stylet guidance and 0/4 (0%) times without image guidance using a skull phantom. CONCLUSIONS: Smart Stylet may improve the ability to successfully target the ventricles during frontal ventriculostomy.

Evaluation of an Automated Information Extraction Tool for Imaging Data Elements to Populate a Breast Cancer Screening Registry.

Breast cancer screening is central to early breast cancer detection. Identifying and monitoring process measures for screening is a focus of the National Cancer Institute's Population-based Research Optimizing Screening through Personalized Regimens (PROSPR) initiative, which requires participating centers to report structured data across the cancer screening continuum. We evaluate the accuracy of automated information extraction of imaging findings from radiology reports, which are available as unstructured text. We present prevalence estimates of imaging findings for breast imaging received by women who obtained care in a primary care network participating in PROSPR (n = 139,953 radiology reports) and compared automatically extracted data elements to a "gold standard" based on manual review for a validation sample of 941 randomly selected radiology reports, including mammograms, digital breast tomosynthesis, ultrasound, and magnetic resonance imaging (MRI). The prevalence of imaging findings vary by data element and modality (e.g., suspicious calcification noted in 2.6% of screening mammograms, 12.1% of diagnostic mammograms, and 9.4% of tomosynthesis exams). In the validation sample, the accuracy of identifying imaging findings, including suspicious calcifications, masses, and architectural distortion (on mammogram and tomosynthesis); masses, cysts, non-mass enhancement, and enhancing foci (on MRI); and masses and cysts (on ultrasound), range from 0.8 to1.0 for recall, precision, and F-measure. Information extraction tools can be used for accurate documentation of imaging findings as structured data elements from text reports for a variety of breast imaging modalities. These data can be used to populate screening registries to help elucidate more effective breast cancer screening processes.

Journal club: Renal masses detected at abdominal CT: radiologists' adherence to guidelines regarding management recommendations and communication of critical results.

OBJECTIVE: The purpose of this study was to assess radiologists' adherence to published guidelines for managing renal masses detected at abdominal CT at one institution and to a critical results communication policy. MATERIALS AND METHODS: A validated natural language processing tool supplemented by manual review was used to randomly assemble a cohort of 97 radiology reports from all abdominal CT reports (n = 11,952) generated from July 2010 to June 2011. Critical renal mass findings warranted consideration for surgery, intervention, or imaging follow-up and required direct, separate, and timely communication to the referrer in addition to the radiology report. Primary outcomes were adherence to guidelines and institutional policy for communicating critical results. Sample size allowed a 95% CI ± 5% for primary outcome. Pearson chi-square test was performed to assess whether radiology subspecialization was predictive of the primary outcome. RESULTS: Of all abdominal CT reports, 35.6% contained at least one renal mass finding (4.3% critical). Guideline adherence was lower for patients with critical than for those with noncritical findings (48/57 [84.2%] vs 40/40 [100%]; p = 0.01). Adherence to critical result communication policy was 73.7% (42/57). For critical findings, abdominal radiologists had higher guideline adherence (40/43 [93.0%] vs 8/14 [57.1%]; p = 0.001) and critical result communication policy adherence (36/43 [83.7%] vs 6/14 [42.9%]; p = 0.002) than nonabdominal radiologists. CONCLUSION: In reporting renal masses detected at abdominal CT, radiologists largely adhered to management guidelines but did not adhere to the critical results communication policy in one of four reports. Subspecialization improved adherence to both management guidelines and the institution's critical result communication policy.