Quality Improvement Report: Design and Implementation of a Radiology E-Consult Service.

Electronic consultations (e-consults) mediated through an electronic health record system or web-based platform allow synchronous or asynchronous physician-to-physician communication. E-consults have been explored in various clinical specialties, but relatively few instances in the literature describe e-consults to connect health care providers directly with radiologists.The authors outline how a radiology department can implement an e-consult service and review the development of such a service in a large academic health system. They describe the logistics, workflow, turnaround time expectations, stakeholder management, and pilot implementation and highlight challenges and lessons learned.

Ovarian Cancer Detection in Average-Risk Women: Classic- versus Nonclassic-appearing Adnexal Lesions at US.

Background Several US risk stratification schemas for assessing adnexal lesions exist. These multiple-subcategory systems may be more multifaceted than necessary for isolated adnexal lesions in average-risk women. Purpose To explore whether a US-based classification scheme of classic versus nonclassic appearance can be used to help appropriately triage women at average risk of ovarian cancer without compromising diagnostic performance. Materials and Methods This retrospective multicenter study included isolated ovarian lesions identified at pelvic US performed between January 2011 and June 2014, reviewed between September 2019 and September 2020. Lesions were considered isolated in the absence of ascites or peritoneal implants. Lesions were classified as classic or nonclassic based on sonographic appearance. Classic lesions included simple cysts, hemorrhagic cysts, endometriomas, and dermoids. Otherwise, lesions were considered nonclassic. Outcomes based on histopathologic results or clinical or imaging follow-up were recorded. Diagnostic performance and frequency of malignancy were calculated. Frequency of malignancy between age groups was compared using the χ2 test, and Poisson regression was used to explore relationships between imaging features and malignancy. Results A total of 970 isolated lesions in 878 women (mean age, 42 years ± 14 [SD]) were included. The malignancy rate for classic lesions was less than 1%. Of 970 lesions, 53 (6%) were malignant. The malignancy rate for nonclassic lesions was 32% (33 of 103) when blood flow was present and 8% (16 of 194) without blood flow (P < .001). For women older than 60 years, the malignancy rate was 50% (10 of 20 lesions) when blood flow was present and 13% (five of 38) without blood flow (P = .004). The sensitivity, specificity, positive predictive value, and negative predictive value of the classic-versus-nonclassic schema was 93% (49 of 53 lesions), 73% (669 of 917 lesions), 17% (49 of 297 lesions), and 99% (669 of 673 lesions), respectively, for detection of malignancy. Conclusion Using a US classification schema of classic- or nonclassic-appearing adnexal lesions resulted in high sensitivity and specificity in the diagnosis of malignancy in ovarian cancer. The highest risk of cancer was in isolated nonclassic lesions with blood flow in women older than 60 years. © RSNA, 2022 See also the editorial by Baumgarten in this issue.

Use of an Online Crowdsourcing Platform to Assess Patient Comprehension of Radiology Reports and Colloquialisms.

OBJECTIVE. The purpose of this study was to use an online crowdsourcing platform to assess patient comprehension of five radiology reporting templates and radiology colloquialisms. MATERIALS AND METHODS. In this cross-sectional study, participants were surveyed as patient surrogates using a crowdsourcing platform. Two tasks were completed within two 48-hour time periods. For the first crowdsourcing task, each participant was randomly assigned a set of radiology reports in a constructed reporting template and subsequently tested for comprehension. For the second crowdsourcing task, each participant was randomly assigned a radiology colloquialism and asked to indicate whether the phrase indicated a normal, abnormal, or ambivalent finding. RESULTS. A total of 203 participants enrolled for the first task and 1166 for the second within 48 hours of task publication. The payment totaled $31.96. Of 812 radiology reports read, 384 (47%) were correctly interpreted by the patient surrogates. Patient surrogates had higher rates of comprehension of reports written in the patient summary (57%, p < 0.001) and traditional unstructured in combination with patient summary (51%, p = 0.004) formats than in the traditional unstructured format (40%). Most of the patient surrogates (114/203 [56%]) expressed a preference for receiving a full radiology report via an electronic patient portal. Several radiology colloquialisms with modifiers such as "low," "underdistended," and "decompressed" had low rates of comprehension. CONCLUSION. Use of the crowdsourcing platform is an expeditious, cost-effective, and customizable tool for surveying laypeople in sentiment- or task-based research. Patient summaries can help increase patient comprehension of radiology reports. Radiology colloquialisms are likely to be misunderstood by patients.

Automated Detection of Radiology Reports that Require Follow-up Imaging Using Natural Language Processing Feature Engineering and Machine Learning Classification.

While radiologists regularly issue follow-up recommendations, our preliminary research has shown that anywhere from 35 to 50% of patients who receive follow-up recommendations for findings of possible cancer on abdominopelvic imaging do not return for follow-up. As such, they remain at risk for adverse outcomes related to missed or delayed cancer diagnosis. In this study, we develop an algorithm to automatically detect free text radiology reports that have a follow-up recommendation using natural language processing (NLP) techniques and machine learning models. The data set used in this study consists of 6000 free text reports from the author's institution. NLP techniques are used to engineer 1500 features, which include the most informative unigrams, bigrams, and trigrams in the training corpus after performing tokenization and Porter stemming. On this data set, we train naive Bayes, decision tree, and maximum entropy models. The decision tree model, with an F1 score of 0.458 and accuracy of 0.862, outperforms both the naive Bayes (F1 score of 0.381) and maximum entropy (F1 score of 0.387) models. The models were analyzed to determine predictive features, with term frequency of n-grams such as "renal neoplasm" and "evalu with enhanc" being most predictive of a follow-up recommendation. Key to maximizing performance was feature engineering that extracts predictive information and appropriate selection of machine learning algorithms based on the feature set.

Patient Factor Disparities in Imaging Follow-Up Rates After Incidental Abdominal Findings.

OBJECTIVE: The effect of demographics and societal determinants on imaging follow-up rates is not clear. The purpose of this study was to compare characteristics of patients with imaging findings representing possible cancer who undergo follow-up imaging versus those who do not to better understand factors that contribute to follow-up completion. MATERIALS AND METHODS: The records of 1588 patients with indeterminate abdominal imaging findings consecutively registered between July 1, 2013, and March 20, 2014, were reviewed. Several patient characteristics, including distance between patients' home zip codes and the flagship hospital of the health system were compared between the groups who did and did not undergo follow-up imaging. Subgroup analyses based on the location of the index examination were also performed. RESULTS: Among the 1513 (36.62%) included patients, 554 did not undergo follow-up abdominal imaging within 1 year of the index examination. The same was true of 270 of 938 (28.78%) outpatients and 168 of 279 (60.21%) emergency department patients. Eighty-nine of 959 (9.28%) patients who underwent follow-up imaging were younger than 40 years, compared with 76 of 554 (13.72%) patients who did not undergo follow-up imaging (p = 0.005). Fifty-four of 959 (5.63%) patients who underwent follow-up imaging were older than 80 years, compared with 70 of 554 (12.64%) patients who did not undergo follow-up imaging (p < 0.001). More white patients (587 of 959 vs 301 of 554, p = 0.007) and fewer black patients (204 of 554 versus 270 of 959, p < 0.001) were found in the follow-up imaging group. Greater distance from the flagship hospital correlated with less follow-up in the outpatient subgroup only (p = 0.03). CONCLUSION: Emergency department patients and patients at the extremes of age are less likely to complete follow-up imaging. Insurance status and race and ethnicity may affect follow-up completion rates. The relationship between distance to hospital and follow-up completion requires further investigation.

Effect of Clinical Decision Support-Generated Report Cards Versus Real-Time Alerts on Primary Care Provider Guideline Adherence for Low Back Pain Outpatient Lumbar Spine MRI Orders.

OBJECTIVE: The purpose of this study is to determine whether the type of feedback on evidence-based guideline adherence influences adult primary care provider (PCP) lumbar spine (LS) MRI orders for low back pain (LBP). MATERIALS AND METHODS: Four types of guideline adherence feedback were tested on eight tertiary health care system outpatient PCP practices: no feedback during baseline (March 1, 2012-October 4, 2012), randomization by practice to either clinical decision support (CDS)-generated report cards comparing providers to peers only or real-time CDS alerts at order entry during intervention 1 (February 6, 2013-December 31, 2013), and both feedback types for all practices during intervention 2 (January 14, 2014-June 20, 2014, and September 4, 2014-January 21, 2015). International Classification of Disease codes identified LBP visits (excluding Medicare fee-for-service). The primary outcome of the likelihood of LS MRI order being made on the day of or 1-30 days after the outpatient LBP visit was adjusted by feedback type (none, report cards only, real-time alerts only, or both); patient age, sex, race, and insurance status; and provider sex and experience. RESULTS: Half of PCPs (54/108) remained for all three periods, conducting 9394 of 107,938 (8.7%) outpatient LBP visits. The proportion of LBP visits increased over the course of the study (p = 0.0001). In multilevel hierarchic regression, report cards resulted in a lower likelihood of LS MRI orders made the day of and 1-30 days after the visit versus baseline: 38% (p = 0.009) and 37% (p = 0.006) for report cards alone, and 27% (p = 0.020) and 27% (p = 0.016) with alerts, respectively. Real-time alerts alone did not affect MRI orders made the day of (p = 0.585) or 1-30 days after (p = 0.650) the visit. No patient or provider variables were associated with LS MRI orders being generated on the day of or 1-30 days after the LBP visit. CONCLUSION: CDS-generated evidence-based report cards can substantially reduce outpatient PCP LS MRI orders on the day of and 1-30 days after the LBP visit. Real-time CDS alerts do not.

Toward Complete Structured Information Extraction from Radiology Reports Using Machine Learning.

Unstructured and semi-structured radiology reports represent an underutilized trove of information for machine learning (ML)-based clinical informatics applications, including abnormality tracking systems, research cohort identification, point-of-care summarization, semi-automated report writing, and as a source of weak data labels for training image processing systems. Clinical ML systems must be interpretable to ensure user trust. To create interpretable models applicable to all of these tasks, we can build general-purpose systems which extract all relevant human-level assertions or "facts" documented in reports; identifying these facts is an information extraction (IE) task. Previous IE work in radiology has focused on a limited set of information, and extracts isolated entities (i.e., single words such as "lesion" or "cyst") rather than complete facts, which require the linking of multiple entities and modifiers. Here, we develop a prototype system to extract all useful information in abdominopelvic radiology reports (findings, recommendations, clinical history, procedures, imaging indications and limitations, etc.), in the form of complete, contextualized facts. We construct an information schema to capture the bulk of information in reports, develop real-time ML models to extract this information, and demonstrate the feasibility and performance of the system.

Engaging Patients: Models for Patient- and Family-centered Care in Radiology.

This article examines how radiologists can meet the objectives of patient- and family-centered care set forth by the National Committee for Quality Assurance (NCQA) program for primary care providers. The breadth of initiatives in radiology that can be mapped to the NCQA objectives is impressive and invites the idea of creating a similar program in radiology. ©RSNA, 2018.

Clinical decision support increases diagnostic yield of computed tomography for suspected pulmonary embolism.

OBJECTIVE: Determine effects of evidence-based clinical decision support (CDS) on the use and yield of computed tomographic pulmonary angiography for suspected pulmonary embolism (CTPE) in Emergency Department (ED) patients. METHODS: This multi-site prospective quality improvement intervention conducted in three urban EDs used a pre/post design. For ED patients aged 18+years with suspected PE, CTPE use and yield were compared 19months pre- and 32months post-implementation of CDS intervention based on the Wells criteria, provided at the time of CTPE order, deployed in April 2012. Primary outcome was the yield (percentage of studies positive for acute PE). Secondary outcome was utilization (number of studies/100 ED visits) of CTPE. Chi-square and statistical process control chart assessed pre- and post-intervention differences. An interrupted time series analysis was also performed. RESULTS: Of 558,795 patients presenting October 2010-December 2014, 7987 (1.4%) underwent CTPE (mean age 52±17.5years, 66% female, 60.1% black); 34.7% of patients presented pre- and 65.3% post-CDS implementation. Overall CTPE diagnostic yield was 9.8% (779/7987 studies positive for PE). Yield increased a relative 30.8% after CDS implementation (8.1% vs. 10.6%; p=0.0003). There was no statistically significant change in CTPE utilization (1.4% pre- vs. 1.4% post-implementation; p=0.25). A statistical process control chart demonstrated immediate and sustained improvement in CTPE yield post-implementation. Interrupted time series analysis demonstrated the slope of PE findings versus time to be unchanged before and after the intervention (p=0.9). However, there was a trend that the intervention was associated with a 50% increased probability of PE finding (p=0.08), suggesting an immediate rather than gradual change after the intervention. CONCLUSIONS: Implementing evidence-based CDS in the ED was associated with an immediate, significant and sustained increase in CTPE yield without a measurable decrease in CTPE utilization. Further studies will be needed to assess whether stronger interventions could further improve appropriate use of CTPE.

Expanding the Scope of an Automated Radiology Recommendation-Tracking Engine: Initial Experiences and Lessons Learned.

An automated radiology recommendation-tracking engine for incidental focal masses in the liver, pancreas, kidneys, and adrenal glands was launched within our institution in July 2013. For 2 years, the majority of CT, MR, and US examination reports generated within our health system were mined by the engine. However, the need to expand the system beyond the initial four organs was soon identified. In July 2015, the second phase of the system was implemented and expanded to include additional anatomic structures in the abdomen and pelvis, as well as to provide non-radiology and non-imaging options for follow-up. The most frequent organs with incidental findings, outside of the original four, included the ovaries and the endometrium, which also correlated to the most frequently ordered imaging follow-up study of pelvic ultrasound and non-imaging follow-up study of endometrial biopsies, respectively. The second phase expansion has demonstrated new venues for augmenting and improving radiologist roles in optimal communication and management of incidental findings.

"Chasing a Ghost": Factors that Influence Primary Care Physicians to Follow Up on Incidental Imaging Findings.

Purpose To explore provider and patient characteristics that influence how primary care providers (PCPs) communicate and manage incidental imaging findings. Materials and Methods This HIPAA-compliant study was approved by the institutional review board. Through semistructured interviews, researchers explored concerns and perspectives of 30 PCPs on receiving and acting on incidental imaging findings. Open-ended questions were designed to elicit a range of responses rather than quantifiable data. Thematic codes were developed and explicitly defined. Three research assistants independently coded all 30 deidentified transcripts and resolved discrepancies (κ = 0.85). Codes pertaining to PCP and patient characteristics were organized into an explanatory model. Results Some PCPs felt compelled but frustrated to pursue costly follow-up for incidental imaging findings of limited clinical importance. Other PCPs did not act on findings that were unfamiliar or occurred in an unusual clinical context when follow-up recommendations were not given; the challenges of researching the clinical importance of these findings or seeking specialist consultation led to inaction. Some PCPs reported using a uniform approach to communicate and manage incidental findings, while others adapted their approach to the patient and the finding. Sometimes PCP characteristics such as follow-up style superseded patient characteristics. At other times patient characteristics such as health literacy superseded PCP characteristics. Conclusion PCPs cited a variety of objective and subjective factors that influence how they communicate and manage incidental imaging findings. These results suggest that some patients may receive inappropriate follow-up of incidental imaging findings and present an opportunity for radiologists to help PCPs and patients to best use the information conveyed in imaging reports. © RSNA, 2016 Online supplemental material is available for this article.

Willingness to Pay for CT Colonography: A Survey of Patient Preferences.

OBJECTIVE: CT colonography (CTC) has received mixed reviews both in the radiology literature and in clinical practice. CTC is less invasive than optical colonoscopy (OC), is better for identifying polyps, and does not require sedation. However, its cost-effectiveness has been called into question, and there is a residual need for OC if the CTC findings are positive. Some radiologists are hesitant to perform CTC because of the time-intensive nature of its interpretation. Results of willingness-to-pay surveys can be informative about preferences and value placed on intangibles. The purpose of this study was to use such a survey to elicit the preferences of radiologists about CTC versus OC. SUBJECTS AND METHODS: A vignette was presented in which the insurer covered OC at no charge but required out-of-pocket payment for CTC. The strengths and weaknesses of OC and CTC were listed. The respondents were asked how much they would be willing to pay for CTC; for CTC with perfect sensitivity, specificity, and accuracy; and for CTC that required no bowel preparation. RESULTS: Twenty-eight of 42 radiologists preferred OC to CTC. One radiologist was indifferent. Four radiologists would reverse their preference and choose CTC if it had 100% sensitivity. Sixteen radiologists said they would prefer CTC if it had 100% specificity. If CTC eliminated the need for bowel preparation, 57% would prefer it to OC. Thirty-one (74%) radiologists preferred perfect sensitivity to perfect specificity. CONCLUSION: Despite the less invasive nature of CTC, most radiologists who responded to the survey preferred OC for colorectal cancer screening, mostly because of the definitive nature of OC due to the capability of immediate biopsy of suspicious lesions and the lack of requirement for a second round of bowel preparation.

Developing an automated database for monitoring ultrasound- and computed tomography-guided procedure complications and diagnostic yield.

Monitoring complications and diagnostic yield for image-guided procedures is an important component of maintaining high quality patient care promoted by professional societies in radiology and accreditation organizations such as the American College of Radiology (ACR) and Joint Commission. These outcome metrics can be used as part of a comprehensive quality assurance/quality improvement program to reduce variation in clinical practice, provide opportunities to engage in practice quality improvement, and contribute to developing national benchmarks and standards. The purpose of this article is to describe the development and successful implementation of an automated web-based software application to monitor procedural outcomes for US- and CT-guided procedures in an academic radiology department. The open source tools PHP: Hypertext Preprocessor (PHP) and MySQL were used to extract relevant procedural information from the Radiology Information System (RIS), auto-populate the procedure log database, and develop a user interface that generates real-time reports of complication rates and diagnostic yield by site and by operator. Utilizing structured radiology report templates resulted in significantly improved accuracy of information auto-populated from radiology reports, as well as greater compliance with manual data entry. An automated web-based procedure log database is an effective tool to reliably track complication rates and diagnostic yield for US- and CT-guided procedures performed in a radiology department.

Liberating Structures in Strategic Planning: Methods to Foster Engagement.

This paper describes the innovative approach of using liberating structures to the development of the AUR 2023 strategic plan, and lessons learned in their application. The 2023 strategic plan built on the results and approach of the prior 2015 plan. Similar to the 2015 strategic plan, traditional tools such as a SWOT analysis and strategic retreat were used. In addition, the 2023 process included tools called liberating structures and was iteratively co-produced through a series of virtual meetings over 18 months. Advantages of liberating structures included increased creativity and speed in moving through meeting tasks, increased number of meaningful contributions from AUR members and increased engagement from participants during discussions and meetings. The 2023 AUR strategic plan is provided along with examples of completed goals and those under early implementation. Lessons learned from using these tools for strategic planning can be applied to other society and group meetings. Moving forward, the 2023 strategic plan will be a living document, which will be reviewed at each Board of Directors meeting and periodically adapted.

Predictors of CT colonography utilization among asymptomatic medicare beneficiaries.

BACKGROUND: Although the Centers for Medicare and Medicaid Services (CMS) denied coverage for screening computed tomography colonography (CTC) in March 2009, little is understood about whether CTC was targeted to the appropriate patient population prior to this decision. OBJECTIVE: Evaluate patient characteristics and known relative clinical indications for screening CTC among patients who received CTC compared to optical colonoscopy (OC). DESIGN/PARTICIPANTS: Cross-sectional study of all 10,538 asymptomatic Medicare beneficiaries who underwent CTC between January 2007 and December 2008, compared to a cohort of 160,113 asymptomatic beneficiaries who underwent OC, matched on county of residence and year of examination. MAIN MEASURES: Patient characteristics and known relative appropriate and inappropriate clinical indications for screening CTC. KEY RESULTS: CTC utilization was higher among women, patients > 65 years of age, white patients, and those with household income > 75 % (p = 0.001). Patients with relatively appropriate clinical indications for screening CTC were more likely to undergo CTC than OC including presumed incomplete OC (OR 80.7, 95 % CI 76.01-85.63); sedation risk (OR 1.11, 95 % CI 1.05-1.17); and chronic anticoagulation risk (OR 1.46, 95 % CI 1.38-1.54), after adjusting for patient characteristics and known clinical indications. Conversely, patients undergoing high-risk screening, an inappropriate indication, were less likely to receive CTC (OR 0.4, 95 % CI 0.37-0.42). Overall, 83 % of asymptomatic patients referred to CTC had at least one clinical indication relatively appropriate for CTC (8,772/10,538). CONCLUSION: During the 2 years preceding CMS denial for screening, CTC was targeted to asymptomatic patients with relatively appropriate clinical indications for CTC/not receiving OC. However, CTC utilization was lower among certain demographic groups, including minority patients. These findings raise the possibility that future coverage of screening CTC might exacerbate disparities in colorectal cancer screening while increasing overall screening rates.

Building Effective Teams in the Real World From Traps to Triumph.

Quality patient care and advancements in medical education, investigation, and innovation require effective teamwork. High-functioning teams navigate stressful environments, learning openly from failures and leveraging successes to fuel future initiatives. The authors review foundational concepts for implementing and sustaining successful teams, including emotional intelligence, trust, inclusivity, clear communication, and accountability. Focus is given to real-world examples and actionable, practical solutions.

Impact of Artificial Intelligence-Assisted Indication Selection on Appropriateness Order Scoring for Imaging Clinical Decision Support.

PURPOSE: The aim of this study was to assess appropriateness scoring and structured order entry after the implementation of an artificial intelligence (AI) tool for analysis of free-text indications. METHODS: Advanced outpatient imaging orders in a multicenter health care system were recorded 7 months before (March 1, 2020, to September 21, 2020) and after (October 20, 2020, to May 13, 2021) the implementation of an AI tool targeting free-text indications. Clinical decision support score (not appropriate, may be appropriate, appropriate, or unscored) and indication type (structured, free-text, both, or none) were assessed. The χ2 and multivariate logistic regression adjusting for covariables with bootstrapping were used. RESULTS: In total, 115,079 orders before and 150,950 orders after AI tool deployment were analyzed. The mean patient age was 59.3 ± 15.5 years, and 146,035 (54.9%) were women; 49.9% of orders were for CT, 38.8% for MR, 5.9% for nuclear medicine, and 5.4% for PET. After deployment, scored orders increased to 52% from 30% (P < .001). Orders with structured indications increased to 67.3% from 34.6% (P < .001). On multivariate analysis, orders were more likely to be scored after tool deployment (odds ratio [OR], 2.7, 95% CI, 2.63-2.78; P < .001). Compared with physicians, orders placed by nonphysician providers were less likely to be scored (OR, 0.80; 95% CI, 0.78-0.83; P < .001). MR (OR, 0.84; 95% CI, 0.82-0.87) and PET (OR, 0.12; 95% CI, 0.10-0.13) were less likely to be scored than CT (; P < .001). After AI tool deployment, 72,083 orders (47.8%) remained unscored, 45,186 (62.7%) with free-text-only indications. CONCLUSIONS: Embedding AI assistance within imaging clinical decision support was associated with increased structured indication orders and independently predicted a higher likelihood of scored orders. However, 48% of orders remained unscored, driven by both provider behavior and infrastructure-related barriers.

Development of automated detection of radiology reports citing adrenal findings.

The aim of this study was to determine the feasibility of automated detection of adrenal nodules, a common finding on CT, using a newly developed search engine that mines dictated radiology reports. To ensure Health Insurance Portability and Accountability Act compliance, we utilized a preexisting de-identified database of 32,974 CT reports from February 1, 2009 to February 28, 2010. Common adrenal descriptors from 29 staff radiologists were used to develop an automated rule-based algorithm targeting adrenal findings. Each sentence within the free text of reports was searched with an adapted NegEx negation algorithm. The algorithm was refined using a 2-week test period of reports and subsequently validated using a 6-week period. Manual review of the 3,693 CT reports in the validation period identified 222 positive reports while the algorithm detected 238 positive reports. The algorithm identified one true positive report missed on manual review for a total of 223 true positive reports. This resulted in a precision of 91% (217 of 238) and a recall of 97% (217 of 223). The sensitivity of the query was 97.3% (95% confidence interval (CI), 93.9-98.9%), and the specificity was 99.3% (95% CI, 99.1-99.6%). The positive predictive value of the algorithm was 91.0% (95% CI, 86.6-94.3%), and the negative predictive value was 99.8% (95% CI, 99.6-99.9%). The prevalence of true positive adrenal findings identified by the query (7.1%) was nearly identical to the true prevalence (7.2%). Automated detection of language describing common findings in imaging reports, such as adrenal nodules on CT, is feasible.

Development and Implementation of a Multisite Registry Using Structured Templates for Actionable Findings in the Kidney.

PURPOSE: The aim of this study was to scale structured report templates categorizing actionable renal findings across health systems and create a centralized registry of patient and report data. METHODS: In January 2017, three academic radiology departments agreed to prospectively include identical structured templates categorizing the malignant likelihood of renal findings in ≥90% of all adult ultrasound, MRI, and CT reports, a new approach for two sites. Between November 20, 2017, and September 30, 2019, deidentified HL7 report data were transmitted to a centralized ACR registry. An automated algorithm extracted categories. Radiologists were requested to addend reports with missing or incomplete templates after the first month. Separately, each site submitted patient sociodemographic and clinical data 12 months before and at least 3 months after enrollment. RESULTS: A total of 164,982 eligible radiology reports were transmitted to the registry; 4,159 (2.5%) were excluded because of missing categories or radiologist names. The final cohort included 160,823 examinations on 102,619 unique patients. Mean template use before and after addendum requests was 99.3% and 99.9% at SITE1, 86.5% and 94.6% at SITE2, and 91.4% and 96.0% at SITE3. Matching patient sociodemographic and clinical data were obtained on 96.9% of reports from SITE1, 94.2% from SITE2, and 96.0% from SITE3. Regulatory, cultural, and technology barriers to the creation of a multisite registry were identified. CONCLUSIONS: Barriers to the adoption of unified structured report templates for actionable kidney findings can be addressed. Deidentified report and patient data can be securely transmitted to an external registry. These data can facilitate the collection of diverse evidence-based population imaging outcomes.

Diagnostic Performance of the Ovarian-Adnexal Reporting and Data System (O-RADS) Ultrasound Risk Score in Women in the United States.

Importance: The American College of Radiology (ACR) Ovarian-Adnexal Reporting and Data System (O-RADS) ultrasound (US) risk scoring system has been studied in a selected population of women referred for suspected or known adnexal lesions. This population has a higher frequency of malignant neoplasms than women presenting to radiology departments for pelvic ultrasonography for a variety of indications, potentially impacting the diagnostic performance of the risk scoring system. Objective: To evaluate the risk of malignant neoplasm and diagnostic performance of O-RADS US risk scoring system in a multi-institutional, nonselected cohort. Design, Setting, and Participants: This multi-institutional cohort study included a population of nonselected women in the United States who presented to radiology departments for routine pelvic ultrasonography between 2011 and 2014, with pathology confirmation imaging follow up or 2 years of clinical follow up. Exposure: Analysis of 1014 adnexal lesions using the O-RADS US risk stratification system. Main Outcomes and Measures: Frequency of ovarian cancer and diagnostic performance of the O-RADS US risk stratification system. Results: This study included 913 women with 1014 adnexal lesions. The mean (SD) age of the patients was 42.4 (13.9 years), and 674 of 913 (73.8%) were premenopausal. The overall frequency of malignant neoplasm was 8.4% (85 of 1014 adnexal lesions). The frequency of malignant neoplasm for O-RADS US 2 was 0.5% (3 of 657 lesions; <1% expected); O-RADS US 3, 4.5% (5 of 112 lesions; <10% expected); O-RADS US 4, 11.6% (18 of 155; 10%-50% expected); and O-RADS 5, 65.6% (59 of 90 lesions; >50% expected). O-RADS US 4 was the optimum cutoff for diagnosing cancer with sensitivity of 90.6% (95% CI, 82.3%-95.9%), specificity of 81.9% (95% CI, 79.3%-84.3%), positive predictive value of 31.4% (95% CI, 25.7%-37.7%) and negative predictive value of 99.0% (95% CI, 98.0%-99.6%). Conclusions and Relevance: In this cohort study of a nonselected patient population, the O-RADS US risk stratification system performed within the expected range as published by the ACR O-RADS US committee. The frequency of malignant neoplasm was at the lower end of the published range, partially because of the lower prevalence of cancer in a nonselected population. However, a high negative predictive value was maintained, and when a lesion can be classified as an O-RADS US 2, the risk of cancer is low, which is reassuring for both clinician and patient.