Patterns of Screening Recall Behavior Among Subspecialty Breast Radiologists.

RATIONALE AND OBJECTIVES: Determine whether there are patterns of lesion recall among breast imaging subspecialists interpreting screening mammography, and if so, whether recall patterns correlate to morphologies of screen-detected cancers. MATERIALS AND METHODS: This Institutional Review Board-approved, retrospective review included all screening examinations January 3, 2012-October 1, 2018 interpreted by fifteen breast imaging subspecialists at a large academic medical center and two outpatient imaging centers. Natural language processing identified radiologist recalls by lesion type (mass, calcifications, asymmetry, architectural distortion); proportions of callbacks by lesion types were calculated per radiologist. Hierarchical cluster analysis grouped radiologists based on recall patterns. Groups were compared to overall practice and each other by proportions of lesion types recalled, and overall and lesion-specific positive predictive value-1 (PPV1). RESULTS: Among 161,859 screening mammograms with 13,086 (8.1%) recalls, Hierarchical cluster analysis grouped 15 radiologists into five groups. There was substantial variation in proportions of lesions recalled: calcifications 13%-18% (Chi-square 45.69, p < 0.00001); mass 16%-44% (Chi-square 498.42, p < 0.00001); asymmetry 13%-47% (Chi-square 660.93, p < 0.00001) architectural distortion 6%-20% (Chi-square 283.81, p < 0.00001). Radiologist groups differed significantly in overall PPV1 (range 5.6%-8.8%; Chi-square 17.065, p = 0.0019). PPV1 by lesion type varied among groups: calcifications 9.2%-15.4% (Chi-square 2.56, p = 0.6339); mass 5.6%-8.5% (Chi-square 1.31, p = 0.8597); asymmetry 3.4%-5.9% (Chi-square 2.225, p = 0.6945); architectural distortion 5.6%-10.8% (Chi-square 5.810, p = 0.2138). Proportions of recalled lesions did not consistently correlate to proportions of screen-detected cancer. CONCLUSION: Breast imaging subspecialists have patterns for screening mammography recalls, suggesting differential weighting of imaging findings for perceived malignant potential. Radiologist recall patterns are not always predictive of screen-detected cancers nor lesion-specific PPV1s.

A Prospective Intervention to Reduce Burnout Among Academic Radiologists.

RATIONALE AND OBJECTIVES: Few studies have examined what constitutes effective interventions to reduce burnout among radiologists. We compared self-reported burnout among academic medical center radiologists before and after a series of departmental initiatives intended to increase wellbeing and professional satisfaction. MATERIALS AND METHODS: This Institutional Review Board-approved, prospective study took place 2017-2019 in a tertiary academic medical center. In pre- (2017) and post-intervention (2019) periods, we administered the previously-validated Stanford Physician Wellness Survey to faculty in our 11-division radiology department. Faculty rated their burnout level across 8 domains (professional fulfillment, emotional exhaustion, interpersonal disengagement, sleep difficulties, self-compassion, negative work impact on personal relations, organizational/personal values alignment, perceived quality of supervisory leadership). Between the two surveys, departmental initiatives focusing on culture, team building, work-life balance, and personal well-being were implemented (e.g., electronic medical record training, shorter work hours). Pre- and post-survey results were compared, using Whitney-Mann U test to calculate Z scores. RESULTS: Faculty members rated lower professional fulfillment (Z-3.04, p=0.002), higher emotional exhaustion (Z=2.52, p=0.012), increased sleep-related impairment (Z=2.38, p=0.012), and reduced organizational/personal values alignment (Z=-4.10, p<0.0001) between the two surveys. No significant differences were identified associated with interpersonal disengagement (Z=1.82, p=0.069), self-compassion (Z=1.39, p=0.164), negative impact of work on personal relationship (Z=0.89, p=0.372) and perceived supervisory leadership quality (Z=0.07, p=0.942). CONCLUSION: Despite numerous departmental initiatives intended to improve culture, workplace efficiency, work-life balance, and personal wellness, self-reported burnout was unchanged or worsened over time.Physician and employee wellness embedded into institutional culture maybe more effective than departmental improvement initiatives.

Self-reported Burnout: Comparison of Radiologists to Nonradiologist Peers at a Large Academic Medical Center.

RATIONALE AND OBJECTIVES: Relatively little data exist on factors associated with radiologists' burnout versus other medical specialties. We compared self-reported burnout among academic medical center radiologists versus nonradiologist peers to inform initiatives to increase wellbeing and professional satisfaction. MATERIALS AND METHODS: In 2017, our large urban academic medical center administered the Stanford Physician Wellness Survey to faculty in fifteen clinical departments (fourteen academic, one community-based). Faculty rated burnout via Likert scale (0-no burnout; 1-occasional stress/no burnout; 2-one or more burnout symptoms; 3-persistent burnout symptoms; 4-completely burned out); burnout defined as >=2. Responses in 11 domains (professional fulfillment, emotional exhaustion, interpersonal disengagement, sleep difficulties, self-compassion, negative work impact on personal relations, perceived appreciation, control over schedule, organizational/personal values alignment, electronic health record, perceived quality of supervisory leadership) compared radiologists versus nonradiologists for association with burnout, using Whitney-Mann U test to calculate Z scores. RESULTS: There was no significant difference in overall self-reported burnout between radiologists and nonradiologists, nor in self-rating for emotional exhaustion, interpersonal disengagement, self-compassion, control over schedule, organizational/personal values alignment, or electronic health record experience. Radiologists had significantly lower self-rating for work happiness (Z = -2.669, p = 0.0076), finding work meaningful (Z = -2.77351, p = 0.0055), perceiving physicians as highly valued (Z = -2.5486, p = 0.0108), and believing leadership treated them with respect and dignity (Z = -3.44149, p = 0.0006). CONCLUSION: Compared to nonradiologist colleagues, radiologists were less likely to find work meaningful and more likely to feel unhappy and undervalued in the workplace and by leadership. Initiatives to increase perceived appreciation, leadership relationships, and meaningfulness of work for radiologists may reduce burnout.

Appropriateness of inpatient stress testing: Implications for development of clinical decision support mechanisms and future criteria.

BACKGROUND: An upcoming national mandate will require consultation of appropriate use criteria (AUC) through a clinical decision support mechanism (CDSM) for advanced imaging. We aimed to evaluate our current ability to ascertain test appropriateness. METHODS: We prospectively collected data on 288 consecutive stress tests and coronary computed tomography angiography studies for medical inpatients. Study appropriateness was determined independently by two physicians using the 2013 Multimodality AUC. RESULTS: The median age of the study population was 66 years [interquartile range (IQR) 56, 75], 40.8% were female, and 52.8% had a history of coronary artery disease. Review of the electronic health record (EHR) alone was sufficient to deem appropriateness for 87.2% of cases. The most common reason it was insufficient was inability to determine if the patient could exercise (59.5%). After reviewing the EHR and pilot CDSM data together, appropriateness could be determined for 95.8% of the cases. The most common reason appropriateness could not be determined was that the exam indication was not addressed by an AUC criterion (83.3%). CONCLUSION: In preparing for the mandate, it will be important for future CDSM to obtain information on the patient's ability to exercise and for future AUC to include additional indications that are not currently addressed.

Predictors of Self-Reported Burnout Among Radiology Faculty at a Large Academic Medical Center.

OBJECTIVE: Determine predictors of self-reported burnout among academic radiologists. METHODS: In 2017, radiologists at an urban medical center completed the Stanford Wellness Survey, rating burnout via Likert scale (0: no burnout; 1: occasional stress, no burnout; 2: one or more burnout symptoms; 3: persistent burnout symptoms; 4: completely burned out). Univariate analyses assessed age, gender, family situation, clinical versus research focus, and academic rank for association with burnout (Likert scale ≥ 2). Responses in 11 domains querying definitions of burnout (professional fulfillment, emotional exhaustion, interpersonal disengagement), individual factors (sleep-related impairment, self-compassion, negative work impact on personal relationships), institutional factors (perceived appreciation, control over schedule, organizational or personal values alignment, electronic health record experience, supervisor's leadership quality) were evaluated for association with burnout, using χ2 and logistic regression to calculate odds ratios (ORs). RESULTS: In 159 of 204 (77.9%) completed radiologist surveys, 35.2% (56 of 159) reported burnout. Age < 40 years (P = .0068) and clinical focus (P = .0111) were significantly associated with burnout. In univariate analysis, all domains except electronic health record were statistically significant: emotional exhaustion (OR = 1.93, P < .0001); professional fulfillment (OR = 0.78, P < .0001); self-compassion (OR = 1.36, P < .0001); perceived appreciation (OR = 0.78, P < .0001); sleep-related impairment (OR = 1.20, P < .0001); supervisor's leadership quality (OR = 0.91, P < .0001); interpersonal disengagement (OR = 1.31, P < .0001); organizational or personal values alignment (OR = 0.87, P = .0004); negative work impact on personal relationships (OR = 1.10, P = .0070); control over schedule (OR = 0.80, P = .0054); electronic health record experience (OR=1.03, P = .5392). DISCUSSION: Nearly all questions significantly predicted self-reported burnout, observed in over one-third of academic radiologists. Younger age and clinical focus were associated with burnout. Initiatives targeting individual factors (eg, sleep impairment, self-compassion) and institutional factors (eg, physician appreciation, leadership-faculty interactions) may reduce burnout.

Can emergency department provider notes help to achieve more dynamic clinical decision support?

OBJECTIVE: Assess whether clinical data were present in emergency department (ED) provider notes at time of order entry for cervical spine (c-spine) imaging that could be used to augment or pre-populate clinical decision support (CDS) attributes. METHODS: This Institutional Review Board-approved retrospective study, performed in a quaternary hospital, included all encounters for adult ED patients seen April 1, 2013-September 30, 2014 for a chief complaint of trauma who received c-spine computed tomography (CT) or x-ray. We assessed proportion of ED encounters with at least 1 c-spine-specific CDS rule attribute in clinical notes available at the time of imaging order and agreement between attributes in clinical notes and data entered into CDS. RESULTS: A portion of the clinical note was submitted before imaging order in 42% (184/438) of encounters reviewed; 59.2% (109/184) of encounters with note portions submitted before imaging order had at least 1 positive CDS attribute identified supporting imaging study appropriateness; 34.8% (64/184) identified exclusion criteria where CDS appropriateness recommendations would not be applicable. 65.8% (121/184) of encounters had either a positive CDS attribute or an exclusion criterion. Concordance of c-spine CDS attributes when present in both notes and CDS was 68.4% (κ = 0.35 95% CI: 0.15-0.56; McNemar P = 0.23). CONCLUSIONS: Clinical notes are an underutilized source of clinical attributes needed for CDS, available in a substantial percentage of encounters at the time of imaging order. Automated pre-population of imaging order requisitions with relevant clinical information extracted from electronic health record provider notes may: (1) improve ordering efficiency by reducing redundant data entry, (2) help improve clinical relevance of CDS alerts, and (3) potentially reduce provider burnout from extraneous alerts.

Comparing Tumor Characteristics and Rates of Breast Cancers Detected by Screening Digital Breast Tomosynthesis and Full-Field Digital Mammography.

OBJECTIVE. The purpose of this study was to compare the cancer detection rates (CDRs), tumor types, and characteristics between screening digital breast tomosynthesis (DBT) and screening full-field digital mammography (FFDM) in a matched patient population in a large academic breast imaging practice with mixed DBT and FFDM technology. MATERIALS AND METHODS. In this retrospective study, we reviewed consecutive screening FFDM and DBT examinations performed between October 2012 and September 2014. To control for nonrandomized selection of FFDM versus DBT examinations, we applied propensity score matching on the basis of patient age, imaging site, and prior imaging findings. An institutional breast cancer registry identified cancer diagnoses. CDR and tumor type, grade, receptor, nodal status, and size were compared between matched FFDM and DBT groups. RESULTS. Sixty-one cancers were detected in the matched screening cohort of DBT (n = 9817) and FFDM (n = 14,180) examinations. CDR was higher with DBT than with FFDM for invasive cancers (2.8 vs 1.3, p = 0.01), minimal cancers (2.4 vs 1.2, p = 0.03), estrogen receptor-positive invasive cancers (2.6 vs 1.1, p = 0.01), and node-negative invasive cancers (2.3 vs 1.1, p = 0.02.), respectively. The ratio of screen-detected invasive cancers to ductal carcinoma in situ on DBT (3.0) was not significantly different from that on FFDM (2.6) (p = 0.79). CONCLUSION. DBT results in an overall increase in CDR irrespective of the tumor type, size, or grade of cancer.

Variation in Follow-up Imaging Recommendations in Radiology Reports: Patient, Modality, and Radiologist Predictors.

Background Variation between radiologists when making recommendations for additional imaging and associated factors are, to the knowledge of the authors, unknown. Clear identification of factors that account for variation in follow-up recommendations might prevent unnecessary tests for incidental or ambiguous image findings. Purpose To determine incidence and identify factors associated with follow-up recommendations in radiology reports from multiple modalities, patient care settings, and imaging divisions. Materials and Methods This retrospective study analyzed 318 366 reports obtained from diagnostic imaging examinations performed at a large urban quaternary care hospital from January 1 to December 31, 2016, excluding breast and US reports. A subset of 1000 reports were randomly selected and manually annotated to train and validate a machine learning algorithm to predict whether a report included a follow-up imaging recommendation (training-and-validation set consisted of 850 reports and test set of 150 reports). The trained algorithm was used to classify 318 366 reports. Multivariable logistic regression was used to determine the likelihood of follow-up recommendation. Additional analysis by imaging subspecialty division was performed, and intradivision and interradiologist variability was quantified. Results The machine learning algorithm classified 38 745 of 318 366 (12.2%) reports as containing follow-up recommendations. Average patient age was 59 years ± 17 (standard deviation); 45.2% (143 767 of 318 366) of reports were from male patients. Among 65 radiologists, 57% (37 of 65) were men. At multivariable analysis, older patients had higher rates of follow-up recommendations (odds ratio [OR], 1.01 [95% confidence interval {CI}: 1.01, 1.01] for each additional year), male patients had lower rates of follow-up recommendations (OR, 0.9; 95% CI: 0.9, 1.0), and follow-up recommendations were most common among CT studies (OR, 4.2 [95% CI: 4.0, 4.4] compared with radiography). Radiologist sex (P = .54), presence of a trainee (P = .45), and years in practice (P = .49) were not significant predictors overall. A division-level analysis showed 2.8-fold to 6.7-fold interradiologist variation. Conclusion Substantial interradiologist variation exists in the probability of recommending a follow-up examination in a radiology report, after adjusting for patient, examination, and radiologist factors. © RSNA, 2019 See also the editorial by Russell in this issue.

Use of Imaging in the Emergency Department: Do Individual Physicians Contribute to Variation?

OBJECTIVE. The purpose of this study is to investigate the magnitude of physician variation in the use of imaging and the factors associated with variation in an urban emergency department (ED) in the United States. MATERIALS AND METHODS. This retrospective cohort study was conducted from April 1, 2013, to March 31, 2014, in the ED of a level I adult trauma center in the northeastern United States. The study cohort included all patient visits to the ED during the study period. We built hierarchic and logistic regression models to determine per-physician utilization of low- and high-cost imaging, and we identified factors correlated with variation in use. Global (i.e., intraclass correlation coefficient) and individual variability metrics were used to profile physician variation after controlling for patient-, visit-, and physician-related covariates. RESULTS. A total of 56,793 patients presented to the ED during the study; of these patients, 49.5% (28,135) underwent imaging, with 38.2% (21,686) undergoing low-cost imaging and 21.9% (12,430) undergoing high-cost imaging. Statistically significant predictors of imaging orders were patient age and sex, number of secondary diagnoses, certain primary diagnoses, time of arrival in the ED, and ED crowding. Unadjusted and adjusted intraclass correlation coefficients were 0.0072 and 0.0066, respectively, for low-cost imaging, and 0.0097 and 0.0090, respectively, for high-cost imaging. The coefficient of variation for adjusted imaging odds ratios was 10.9% and 14.0% for low- and high-cost imaging, respectively, indicating a moderate degree of variation. CONCLUSION. Unexplained and moderate variation in imaging utilization exists among ED physicians, even after controlling for patient, visit, and physician characteristics. Improvement initiatives using well-defined ED imaging quality measures may help improve quality and reduce waste.

Can Automated Retrieval of Data from Emergency Department Physician Notes Enhance the Imaging Order Entry Process?

BACKGROUND: When a paucity of clinical information is communicated from ordering physicians to radiologists at the time of radiology order entry, suboptimal imaging interpretations and patient care may result. OBJECTIVES: Compare documentation of relevant clinical information in electronic health record (EHR) provider note to computed tomography (CT) order requisition, prior to ordering of head CT for emergency department (ED) patients presenting with headache. METHODS: In this institutional review board-approved retrospective observational study performed between April 1, 2013 and September 30, 2014 at an adult quaternary academic hospital, we reviewed data from 666 consecutive ED encounters for patients with headaches who received head CT. The primary outcome was the number of concept unique identifiers (CUIs) relating to headache extracted via ontology-based natural language processing from the history of present illness (HPI) section in ED notes compared with the number of concepts obtained from the imaging order requisition. RESULTS: Our analysis was conducted on cases where the HPI note section was completed prior to image order entry, which occurred in 23.1% (154/666) of encounters. For these 154 encounters, the number of CUIs specific to headache per note extracted from the HPI (median = 3, interquartile range [IQR]: 2-4) was significantly greater than the number of CUIs per encounter obtained from the imaging order requisition (median = 1, IQR: 1-2; Wilcoxon signed rank p < 0.0001). Extracted concepts from notes were distinct from order requisition indications in 92.9% (143/154) of cases. CONCLUSION: EHR provider notes are a valuable source of relevant clinical information at the time of imaging test ordering. Automated extraction of clinical information from notes to prepopulate imaging order requisitions may improve communication between ordering physicians and radiologists, enhance efficiency of ordering process by reducing redundant data entry, and may help improve clinical relevance of clinical decision support at the time of order entry, potentially reducing provider burnout from extraneous alerts.

Impact of a Multifaceted Information Technology-Enabled Intervention on the Adoption of ACR White Paper Follow-Up Recommendations for Incidental Adnexal Lesions Detected on CT.

OBJECTIVE. The objective of our study was to improve adherence to American College of Radiology (ACR) white paper follow-up imaging recommendations for incidental adnexal lesions seen on pelvic CT (herein referred to as "adherence rate to recommendations"). MATERIALS AND METHODS. This quality improvement project was conducted at a large academic teaching hospital. The baseline adherence rate to recommendations was assessed by screening all pelvic CT reports for the period from October 22, 2016, through December 22, 2016, for incidental adnexal findings, followed by manual review. Forty abdominal and cancer imaging radiologists were surveyed to understand the barriers to adoption of the recommendations. Interventions to address the most common identified barriers were implemented on December 23, 2016. The postintervention adherence rate was assessed from December 23, 2016, through February 15, 2017, by again screening CT pelvis reports for incidental adnexal lesions followed by manual review. The change in pre- and postintervention adherence rates was assessed using the Fisher exact test and statistical process control (SPC) p-chart with 3-sigma control limits. RESULTS. The adherence rate to recommendations at baseline was 67% (121/181). Of the 28 of 40 (70%) radiologists who completed the survey, only 29% (8/28) often or consistently used the recommendations. Not remembering the details of the recommendations or not having time to look them up accounted for 83.3% of the barriers cited by radiologists. Interventions consisted of radiologist education and creation of an easily accessible clinical decision support tool incorporated into radiology reporting workflow. The adherence rate to recommendations after the intervention increased to 87% (129/148; p < 0.0001), as also shown by the SPC chart. CONCLUSION. The rate of adherence to follow-up imaging recommendations significantly increased after radiologist education and incorporation of recommendations into the radiologist workflow.

Effect of Clinical Decision Support-Generated Report Cards Versus Real-Time Alerts on Primary Care Provider Guideline Adherence for Low Back Pain Outpatient Lumbar Spine MRI Orders.

OBJECTIVE: The purpose of this study is to determine whether the type of feedback on evidence-based guideline adherence influences adult primary care provider (PCP) lumbar spine (LS) MRI orders for low back pain (LBP). MATERIALS AND METHODS: Four types of guideline adherence feedback were tested on eight tertiary health care system outpatient PCP practices: no feedback during baseline (March 1, 2012-October 4, 2012), randomization by practice to either clinical decision support (CDS)-generated report cards comparing providers to peers only or real-time CDS alerts at order entry during intervention 1 (February 6, 2013-December 31, 2013), and both feedback types for all practices during intervention 2 (January 14, 2014-June 20, 2014, and September 4, 2014-January 21, 2015). International Classification of Disease codes identified LBP visits (excluding Medicare fee-for-service). The primary outcome of the likelihood of LS MRI order being made on the day of or 1-30 days after the outpatient LBP visit was adjusted by feedback type (none, report cards only, real-time alerts only, or both); patient age, sex, race, and insurance status; and provider sex and experience. RESULTS: Half of PCPs (54/108) remained for all three periods, conducting 9394 of 107,938 (8.7%) outpatient LBP visits. The proportion of LBP visits increased over the course of the study (p = 0.0001). In multilevel hierarchic regression, report cards resulted in a lower likelihood of LS MRI orders made the day of and 1-30 days after the visit versus baseline: 38% (p = 0.009) and 37% (p = 0.006) for report cards alone, and 27% (p = 0.020) and 27% (p = 0.016) with alerts, respectively. Real-time alerts alone did not affect MRI orders made the day of (p = 0.585) or 1-30 days after (p = 0.650) the visit. No patient or provider variables were associated with LS MRI orders being generated on the day of or 1-30 days after the LBP visit. CONCLUSION: CDS-generated evidence-based report cards can substantially reduce outpatient PCP LS MRI orders on the day of and 1-30 days after the LBP visit. Real-time CDS alerts do not.

Impact of a Health Information Technology-Enabled Appropriate Use Criterion on Utilization of Emergency Department CT for Renal Colic.

OBJECTIVE: The purpose of this study was to evaluate the impact of an appropriate use criterion (AUC) for renal colic based on local best practice, implemented as electronic clinical decision support (CDS), on the emergency department (ED) use of CT for patients with suspected nephrolithiasis. MATERIALS AND METHODS: This retrospective cohort study was performed in the EDs of a level I trauma center (study site) and local comparable hospital (control site). An AUC for patients younger than 50 years with a history of uncomplicated nephrolithiasis presenting with renal colic was developed by an interdisciplinary emergency medicine, emergency radiology, and urology team and embedded as CDS. AUC-consistent CT of ureter requests received no CDS alert. Otherwise, the orderer was alerted to consider a trial of symptomatic control or discharge without CT. A natural language processing tool mined ED notes for visits in September 2010-February 2012 (before AUC implementation) and April 2013-September 2014 (1 year after implementation) for concept unique identifiers of flank tenderness or renal or ureteral pain. Manual review excluded noneligible cases; the others were reviewed by a multidisciplinary team. Chi-square tests were used to assess for CT rate differences, the primary outcome. RESULTS: The final sample included 467 patients (194 study site) before and 306 (88 study site) after AUC implementation. The study site's CT of ureter rate decreased from 23.7% (46/194) to 14.8% (13/88) (p = 0.03) after implementation of the AUC. The rate at the control site remained unchanged, 49.8% (136/273) versus 48.2% (105/218) (p = 0.3). CONCLUSION: Implementing an AUC based on local best practice as CDS may effectively curb potential imaging overuse in a subset of ED patients with renal colic unlikely to have a complicated course or alternative dangerous diagnosis.

Patient, Radiologist, and Examination Characteristics Affecting Screening Mammography Recall Rates in a Large Academic Practice.

OBJECTIVE: The aims of this study were to evaluate patient, radiologist, and examination characteristics affecting screening mammography recall rates in an academic breast imaging practice and to identify modifiable factors that could reduce recall variation. METHODS: This institutional review board-approved retrospective study included screening mammographic examinations in female patients interpreted by 13 breast imaging specialists at an academic center and two outpatient centers from October 1, 2012, to May 31, 2015. Patient demographics were extracted via electronic medical record. Natural language processing captured breast density, BI-RADS assignment, and current and prior screening examination findings. Radiologists' annual screening volumes, clinical experience, and concentration in breast imaging were calculated. Risk aversion, stress from uncertainty, and malpractice concerns were derived via survey. Univariate and multivariate analyses assessed patient, radiologist, and examination characteristics associated with likelihood of mammography recall. The Pearson product-moment correlation coefficient was used to assess the relationship between cancer detection rate and recall rate. RESULTS: Overall, 5,678 of 61,198 screening examinations (9.3%) were recalled. In multivariate analysis, patient and radiologist characteristics associated with higher odds of recall included patient's age < 50 years (P < .0001), prior mammographic findings (calcification [P < .0001], mass [P < .0001], higher density category [P < .0001]), baseline examination (P < .0001), annual reading volume < 1,250 examinations (P = .0282), and <10 years of experience (P = .0036). Radiologist's risk aversion, stress from uncertainty, malpractice concerns, and cancer detection rates were not associated with higher recall rates (r = -0.36, P = .23). CONCLUSIONS: In addition to patient and examination factors, screening recall variations were associated with radiologists' annual reading volume and experience. Interventions targeting radiologist factors (screening volumes, second review of potential recalls) may help reduce unwarranted variation in screening recall.

Integrity of clinical information in computerized order requisitions for diagnostic imaging.

Objective: Assess information integrity (concordance and completeness of documented exam indications from the electronic health record [EHR] imaging order requisition, compared to EHR provider notes), and assess potential impact of indication inaccuracies on exam planning and interpretation. Methods: This retrospective study, approved by the Institutional Review Board, was conducted at a tertiary academic medical center. There were 139 MRI lumbar spine (LS-MRI) and 176 CT abdomen/pelvis orders performed 4/1/2016-5/31/2016 randomly selected and reviewed by 4 radiologists for concordance and completeness of relevant exam indications in order requisitions compared to provider notes, and potential impact of indication inaccuracies on exam planning and interpretation. Forty each LS-MRI and CT abdomen/pelvis were re-reviewed to assess kappa agreement. Results: Requisition indications were more likely to be incomplete (256/315, 81%) than discordant (133/315, 42%) compared to provider notes (p < 0.0001). Potential impact of discrepancy between clinical information in requisitions and provider notes was higher for radiologist's interpretation than for exam planning (135/315, 43%, vs 25/315, 8%, p < 0.0001). Agreement among radiologists for concordance, completeness, and potential impact was moderate to strong (Kappa 0.66-0.89). Indications in EHR order requisitions are frequently incomplete or discordant compared to physician notes, potentially impacting imaging exam planning, interpretation and accurate diagnosis. Such inaccuracies could also diminish the relevance of clinical decision support alerts if based on information in order requisitions. Conclusions: Improved availability of relevant documented clinical information within EHR imaging requisition is necessary for optimal exam planning and interpretation.

Assessing information sources to elucidate diagnostic process errors in radiologic imaging - a human factors framework.

Objective: To assess information sources that may elucidate errors related to radiologic diagnostic imaging, quantify the incidence of potential safety events from each source, and quantify the number of steps involved from diagnostic imaging chain and socio-technical factors. Materials and Methods: This retrospective, Institutional Review Board-approved study was conducted at the ambulatory healthcare facilities associated with a large academic hospital. Five information sources were evaluated: an electronic safety reporting system (ESRS), alert notification for critical result (ANCR) system, picture archive and communication system (PACS)-based quality assurance (QA) tool, imaging peer-review system, and an imaging computerized physician order entry (CPOE) and scheduling system. Data from these sources (January-December 2015 for ESRS, ANCR, QA tool, and the peer-review system; January-October 2016 for the imaging ordering system) were collected to quantify the incidence of potential safety events. Reviewers classified events by the step(s) in the diagnostic process they could elucidate, and their socio-technical factors contributors per the Systems Engineering Initiative for Patient Safety (SEIPS) framework. Results: Potential safety events ranged from 0.5% to 62.1% of events collected from each source. Each of the information sources contributed to elucidating diagnostic process errors in various steps of the diagnostic imaging chain and contributing socio-technical factors, primarily Person, Tasks, and Tools and Technology. Discussion: Various information sources can differentially inform understanding diagnostic process errors related to radiologic diagnostic imaging. Conclusion: Information sources elucidate errors in various steps within the diagnostic imaging workflow and can provide insight into socio-technical factors that impact patient safety in the diagnostic process.

High-Grade Serous Ovarian Cancer: Use of Machine Learning to Predict Abdominopelvic Recurrence on CT on the Basis of Serial Cancer Antigen 125 Levels.

PURPOSE: The aim of this study was to use machine learning to predict abdominal recurrence on CT on the basis of serial cancer antigen 125 (CA125) levels in patients with advanced high-grade serous ovarian cancer on surveillance. METHODS: This institutional review board-approved, HIPAA-compliant, retrospective, hypothesis-generating study included all 57 patients (mean age, 61 ± 11.2 years) with advanced high-grade serous ovarian cancer who underwent cytoreductive surgery from January to December 2012, followed by surveillance abdominopelvic CT and corresponding CA125 levels. A blinded radiologist reviewed abdominopelvic CT studies until recurrence was noted. Four measures of CA125 were assessed: actual CA125 levels at the time of CT, absolute change since prior CT, relative change since prior CT, and rate of change since prior CT. Using machine learning, support vector machine models were optimized and evaluated using 10-fold cross-validation to determine the CA125 measure most predictive of abdominal recurrence. The association of the most accurate CA125 measure was further analyzed using Cox proportional-hazards model along with age, tumor size, stage, and degree of cytoreduction. RESULTS: Rate of change in CA125 was most predictive of abdominal recurrence in a linear kernel support vector machine model and was significantly higher preceding CT studies showing abdominal recurrence (median 13.2 versus 0.6 units/month; P = .007). On multivariate analysis, a higher rate of CA125 increase was significantly associated with recurrence (hazard ratio, 1.02 per 10 units change; 95% confidence interval, 1.0006-1.04; P = .04). CONCLUSION: A higher rate of CA125 increase is associated with abdominal recurrence. The rate of increase of CA125 may help in the selection of patients who are most likely to benefit from abdominopelvic CT in surveillance of ovarian cancer.

Risk Stratification Model: Lower-Extremity Ultrasonography for Hospitalized Patients with Suspected Deep Vein Thrombosis.

BACKGROUND: The Wells score for deep venous thrombosis (DVT) has a high failure rate and low efficiency among inpatients. OBJECTIVE: To create and validate an inpatient-specific risk stratification model to help assess pre-test probability of DVT in hospitalized patients. DESIGN: Prospective cohort study of hospitalized patients undergoing lower-extremity ultrasonography studies (LEUS) for suspected DVT. Demographics, physical findings, medical history, medications, hospitalization, and laboratory and imaging results were collected. Samples were divided into model derivation (patients undergoing LEUS 11/1/2012-12/31/2013) and validation cohorts (LEUS 1/1/2014-5/31/2015). A DVT prediction rule was derived using the recursive partitioning algorithm (decision tree-type approach) and was then validated. PARTICIPANTS: Adult inpatients undergoing LEUS for suspected DVT from November 2012 to May 2015, excluding those with DVT in the prior 3 months, at a 793-bed, urban academic quaternary-care hospital with ~50,000 admissions annually. MAIN MEASURES: The primary outcome was the presence of proximal DVT, and the secondary outcome was the presence of any DVT (proximal or distal). Model sensitivity and specificity for predicting DVT were calculated. KEY RESULTS: Recursive partitioning yielded four variables (previous DVT, active cancer, hospitalization ≥ 6 days, age ≥ 46 years) that optimized the prediction of proximal DVT and yield in the derivation cohort. From this decision tree, we stratified a scoring system using the validation cohort, categorizing patients into low- and high-risk groups. The incidence rates of proximal DVT were 2.9% and 12.0%, and of any DVT were 5.2% and 21.0%, for the low- and high-risk groups, respectively. The AUC for the discriminatory accuracy of the Center for Evidence-Based Imaging (CEBI) score for risk of proximal DVT identified on LEUS was 0.73. Model sensitivity was 98.1% for proximal and 98.1% for any DVT. CONCLUSIONS: In hospitalized adults, specific factors can help clinicians predict risk of DVT, identifying those with low pre-test probability, in whom ultrasonography can be safely avoided.