False-Positive and False-Negative Contrast-enhanced Mammograms: Pitfalls and Strategies to Improve Cancer Detection.

Contrast-enhanced mammography (CEM) is a relatively new breast imaging modality that uses intravenous contrast material to increase detection of breast cancer. CEM combines the structural information of conventional mammography with the functional information of tumor neovascularity. Initial studies have demonstrated that CEM and MRI perform with similar accuracies, with CEM having a slightly higher specificity (fewer false positives), although larger studies are needed. There are various reasons for false positives and false negatives at CEM. False positives at CEM can be caused by benign lesions with vascularity, including benign tumors, infection or inflammation, benign lesions in the skin, and imaging artifacts. False negatives at CEM can be attributed to incomplete or inadequate visualization of lesions, marked background parenchymal enhancement (BPE) obscuring cancer, lack of lesion contrast enhancement due to technical issues or less-vascular cancers, artifacts, and errors of lesion perception or characterization. When possible, real-time interpretation of CEM studies is ideal. If additional views are necessary, they may be obtained while contrast material is still in the breast parenchyma. Until recently, a limitation of CEM was the lack of CEM-guided biopsy capability. However, in 2020, the U.S. Food and Drug Administration cleared two devices to support CEM-guided biopsy using a stereotactic biopsy technique. The authors review various causes of false-positive and false-negative contrast-enhanced mammograms and discuss strategies to reduce these diagnostic errors to improve cancer detection while mitigating unnecessary additional imaging and procedures. ©RSNA, 2023 Quiz questions for this article are available in the supplemental material.

Imaging Characteristics of and Multidisciplinary Management Considerations for Atypical Ductal Hyperplasia and Flat Epithelial Atypia: Review of Current Literature.

High-risk lesions of the breast are frequently encountered in percutaneous biopsy specimens. While benign, these lesions have historically undergone surgical excision due to their potential to be upgraded to malignancy. However, there is emerging evidence that a tailored management approach should be considered to reduce overtreatment of these lesions. Flat epithelial atypia (FEA) and atypical ductal hyperplasia (ADH) are two of the most commonly encountered high-risk lesions. FEA has been shown to have a relatively low rate of progression to malignancy, and some guidelines are now recommending observation over routine excision in select cases. Selective observation may be reasonable in cases where the target lesion is small and completely removed at biopsy and when there are no underlying risk factors, such as a history of breast cancer or genetic mutation or concurrent ADH. ADH has the highest potential upgrade rate to malignancy of all the high-risk lesions. Most society guidelines continue to recommend surgical excision of this lesion. More recently, some literature suggests that ADH lesions that appear completely removed at biopsy, involve limited foci (less than two or three) with no necrosis or significant atypia, manifest as a small group of mammographic calcifications, or demonstrate no enhancement at MRI may be reasonable for observation. Ultimately, management of all high-risk lesions must be based on a multidisciplinary approach that considers all patient, radiologic, clinical, and histopathologic factors. ©RSNA, 2023 Quiz questions for this article are available in the supplemental material.

Kaizen Process Improvement in Radiology: Primer for Creating a Culture of Continuous Quality Improvement.

Kaizen process improvement is an element of lean production that is an approach to creating continuous improvement. Kaizen is based on the idea that small ongoing positive changes in workflow and elimination of waste can yield major improvements over time. A focused Kaizen event, or rapid process improvement event, can lead to sustainable process improvement in health care settings that are resistant to change. This approach has been proven to be successful in health care. These events are led by a trained facilitator and coach who provides appropriate team education and engagement. To ensure success, the team must embrace the Kaizen culture, which emphasizes the development of a "learning organization" that is focused on relentless pursuit of perfection. The culture empowers all staff to improve the work they perform, with an emphasis on the process and not the individual. Respect for individual people is key in Kaizen. In radiology, this method has been successful in empowering frontline staff to improve their individual workflows. A 5-day Kaizen event has been successful in increasing on-time starts, decreasing lead time, increasing patient and staff satisfaction, and ensuring sustainability. Sustainable success can occur when the team stays true to lean principles, engages leaders, and empowers team members with the use of timely data to drive decision making. Online supplemental material is available for this article. ©RSNA, 2022.

Special Report of the RSNA COVID-19 Task Force: The Short- and Long-term Financial Impact of the COVID-19 Pandemic on Private Radiology Practices.

The coronavirus disease 2019 (COVID-19) pandemic resulted in widespread disruption to the global economy, including demand for imaging services. The resulting reduction in demand for imaging services had an abrupt and substantial impact on private radiology practices, which are heavily dependent on examination volumes for practice revenues. The goal of this report is to describe the specific experiences of radiologists working in various types of private radiology practices during the initial peak of the COVID-19 pandemic. Herein, the authors describe factors determining the impact of the pandemic on private practices, the challenges these practices faced, the cost levers leaders adjusted, and the government subsidies sought. In addition, the authors describe adjustments practices are making to their mid- and long-term strategic plans to pivot for long-term success while managing the COVID-19 pandemic. Private practices have crafted tiered strategies to respond to the impact of the pandemic by pulling various cost levers to adjust service availability, staffing, compensation, benefits, time off, and expense reductions. In addition, they have sought additional revenues, within the boundaries of their practice, to mitigate ongoing financial losses. The longer term impact of the pandemic will alter existing practices, making some of them more likely than others to succeed in the years ahead. This report synthesizes the collective experience of private-practice radiologists shared with members of the Radiological Society of North America COVID-19 Task Force, including discussions with colleagues and leaders of private-practice radiology groups from across the United States.

Contrast-Enhanced Mammography for Newly Diagnosed Breast Cancer in Women With Breast Augmentation: Preliminary Findings.

In 17 women with newly diagnosed breast cancer who underwent contrast-enhanced mammography (CEM) and MRI, both modalities were found to be concordant for the index cancer. In six of the 17 women, CEM showed an additional lesion that was confirmed by MRI. Of these six additional lesions, three were multifocal, one was multicentric, and two were contralateral; two of the six were malignant. MRI did not identify any additional cancers that were not identified on CEM. CEM may have a role in women with breast augmentation and either a contraindication or limited access to MRI.

Increasing Patient Access to MRI Examinations in an Integrated Multispecialty Practice.

A multidisciplinary team evaluated and improved the MRI processes within the authors' integrated health care system, with the aim to increase patient access to MRI. The authors created a SMART (specific, measurable, achievable, relevant, and time-based) goal of decreasing the average number of days to wait for MRI examination by 50%, from 15 to 7.5 days, while also creating capacity to meet demand for same-day and next-day MRI appointment requests. The current performance metrics and processes were compared with available benchmarking and best practice data. Several work groups were created to empower and support frontline teams to identify and capture improvement opportunities. Across all MRI processes, teams focused on creating standard work, advancing practice to top of scope, removing waste, improving communication, reducing rework, and improving patient experience. Patient access to MRI was monitored, measured as the average number of days to wait from the time of scheduling to the MRI examination and time to the third-available appointment. The authors also monitored secondary outcomes (patient satisfaction, throughput metrics) and a balancing measure (technical repeat examination rates). The access improved after intervention: the average number of days to wait for MRI access decreased from 14.2 days to 5.8 days after the intervention (-8.4 days, -59.2%, P < .0001) and third-available appointment decreased from 18 days to 0 days. Ten to 20 same-day and next-day appointments became routinely available. The throughput metrics improved, and balancing measures were not changed. This project resulted in significant improvements in patient access to MRI examinations. The findings demonstrate the value of a multidisciplinary team applying comprehensive improvement strategies to increase patient access to complex services, such as MRI. ©RSNA, 2021.

Randomized Clinical Trial of a Clinical Decision Support Tool for Improving the Appropriateness Scores for Ordering Imaging Studies in Primary and Specialty Care Ambulatory Clinics.

OBJECTIVE. The objective of our study was to evaluate whether the use of a clinical decision support (CDS) tool improved the appropriateness scores of orders for advanced imaging in clinical practice. MATERIALS AND METHODS. We used a stepped-wedge, cluster randomized clinical trial to evaluate the effectiveness of a CDS tool in an integrated health care system. Clinicians entered structured indications for each CT and MRI order, and the indications were electronically scored against appropriateness criteria to assign an appropriateness score. We compared the proportion of orders with adjusted appropriateness scores of 7 or greater (on a 1-9 scale) before and after activation of best practice alerts (BPAs) triggered for orders with low or marginal appropriateness scores. Secondary outcomes included the rate per month of orders for advanced imaging and the proportion of orders for which the radiology department requested changes. RESULTS. Between October 2015 and February 2016, 941 clinicians ordered 22,279 CT or MRI studies that met eligibility criteria. Before activation of the BPA, the mean proportion of appropriate orders (adjusted for time and clinic effect) was 77.0% (95% CI, 75.5-78.4%), which increased to 80.1% (95% CI, 78.7-81.5%) after activation (p = 0.001). There was no significant change in the rate of orders per month for advanced imaging. The proportion of order changes requested by the radiology department decreased from 5.7% (95% CI, 5.6-5.9%) before CDS implementation to 5.3% (95% CI, 5.1-5.5%) after CDS implementation (p < 0.001). CONCLUSION. Using an evidence-based CDS tool in clinical practice was associated with a modest but significant improvement in the appropriateness scores of advanced imaging orders.

Sustained Increases in the Quality of Imaging Histories Provided by Ordering Providers Resulting From Multidisciplinary Collaboration and Systems Design.

OBJECTIVE. The purpose of this quality improvement project was to create sustained improvement in the frequency of more complete imaging histories provided with imaging examinations submitted directly by ordering providers. A secondary purpose was increasing the number of characters submitted by ordering providers with imaging examinations. MATERIALS AND METHODS. A multidisciplinary team defined the components of a complete imaging history, a process that underwent several improvement cycles. Audits were regularly performed using consensus, and the project team regularly evaluated the completeness of the imaging histories. The final components of the definition of a complete imaging history included responses regarding what happened, when it happened, where the patient was experiencing pain, and the ordering provider's concern. Prompts were inserted into the electronic physician order entry process, and performance was monitored for an additional 18 months. RESULTS. A total of 10,236 orders were placed by providers in the study clinic from March 13, 2017, to December 16, 2018. Of the orders audited in the baseline period, 16.0% (64/397) of orders audited in the baseline period contained all four history components, which increased to 52.0% (2200/4234; absolute increase of 36.0%, relative increase of 225.0%; p < 0.0001) in the subsequent time periods, and improvement was sustained. The mean number of characters providers entered in the imaging histories they submitted increased from 45.4 characters per order during the baseline period to 75.4 (66.1% increase, p < 0.0001) after the intervention. CONCLUSION. By collaborating with a multidisciplinary team, we created a standardized definition of an imaging history, engineered our systems to include supportive prompts in the order entry interface, and sustainably improved the quality of imaging histories provided.

Breast MRI ordering practices in a large health care network.

The purpose of this study was to evaluate providers' ordering practices and perceptions of breast magnetic resonance imaging (MRI) in an academic network in order to better understand and educate a referral base. An online survey was distributed to primary care providers (PCPs) and specialists in our hospital and community practices. Questions included provider demographics, current ordering practices, challenges to ordering, and perceptions about breast MRI. Of 525 ordering providers, 134 responded (26% response rate). Of 134 providers, 57 (42%) order breast MRI in practice. Of those who do not, the most consistent reason was a lack of familiarity with the use of breast MRI (32/77 [42%] of cases). Of 57 cases, 45 (79%) order less than 10 exams annually. The most frequent indication is for high-risk screening (40/47 [84%]). PCPs order fewer breast MRI compared with specialists (P = 0.01). Both PCPs and specialists have mixed perceptions of the clinical utility of breast MRI. However, 30% of all providers are ordering more breast MRI since the enactment of breast density legislation in Massachusetts. Furthermore, 29% report they would order breast MRI more often to screen women with dense breasts if there was a low cost option. Referring provider surveys are useful tools for assessing a radiology practice. Our study suggests a growing clinical interest in breast MRI for screening; however, there is a need for provider education on the clinical utility of breast MRI. Increasing the radiologist's role in targeted educational interventions may help improve awareness and lead to more appropriate utilization of resources.

Implementation of a Peer Learning Program Replacing Score-Based Peer Review in a Multispecialty Integrated Practice.

OBJECTIVE: The aim of this study was to compare the efficacy of a peer learning program to score-based peer review in a radiology department. MATERIALS AND METHODS: Our score-based peer review program was transitioned to an open, inclusive, education- and improvement-oriented peer learning program. Program performance was evaluated by learning opportunity submissions, program participation, number of learning opportunity distributions to radiologists, number of practice improvement projects resulting from program, and radiologist survey results before and after interventions. RESULTS: Outcomes for the score-based peer review program compared with those of the peer learning program were as follows. The mean number of radiologists participating monthly increased from 5.0 to 35.2 (p < 0.01); submissions increased from 3.0 discrepancies per month to 36.0 learning opportunities per month (p < 0.01); the mean monthly learning opportunity distributions to radiologists increased from 18 to 352 (p < 0.01); improvement projects performed during the study periods increased from 5 to 61 (p < 0.01); and mean monthly continuing medical education credits earned by radiologists increased from 7.7 to 50.6 (p < 0.01). Radiologists felt peer learning accomplished goals better than did score-based peer review. CONCLUSION: In transitioning our score-based peer review program to a peer learning program, our radiologists identified, discussed, and converted more learning opportunities into practice improvement and perceived peer learning to be of higher value than score-based peer review.

Proposed biopsy performance benchmarks for MRI based on an audit of a large academic center.

Performance benchmarks exist for mammography (MG); however, performance benchmarks for magnetic resonance imaging (MRI) are not yet fully developed. The purpose of our study was to perform an MRI audit based on established MG and screening MRI benchmarks and to review whether these benchmarks can be applied to an MRI practice. An IRB approved retrospective review of breast MRIs was performed at our center from 1/1/2011 through 12/31/13. For patients with biopsy recommendation, core biopsy and surgical pathology results were reviewed. The data were used to derive mean performance parameter values, including abnormal interpretation rate (AIR), positive predictive value (PPV), cancer detection rate (CDR), percentage of minimal cancers and axillary node negative cancers and compared with MG and screening MRI benchmarks. MRIs were also divided by screening and diagnostic indications to assess for differences in performance benchmarks amongst these two groups. Of the 2455 MRIs performed over 3-years, 1563 were performed for screening indications and 892 for diagnostic indications. With the exception of PPV2 for screening breast MRIs from 2011 to 2013, PPVs were met for our screening and diagnostic populations when compared to the MRI screening benchmarks established by the Breast Imaging Reporting and Data System (BI-RADS) 5 Atlas® . AIR and CDR were lower for screening indications as compared to diagnostic indications. New MRI screening benchmarks can be used for screening MRI audits while the American College of Radiology (ACR) desirable goals for diagnostic MG can be used for diagnostic MRI audits. Our study corroborates established findings regarding differences in AIR and CDR amongst screening versus diagnostic indications.

Peer Feedback, Learning, and Improvement: Answering the Call of the Institute of Medicine Report on Diagnostic Error.

In September 2015, the Institute of Medicine (IOM) published a report titled "Improving Diagnosis in Health Care," in which it was recommended that "health care organizations should adopt policies and practices that promote a nonpunitive culture that values open discussion and feedback on diagnostic performance." It may seem counterintuitive that a report addressing a highly technical skill such as medical diagnosis would be focused on organizational culture. The wisdom becomes clearer, however, when examined in the light of recent advances in the understanding of human error and individual and organizational performance. The current dominant model for radiologist performance improvement is scoring-based peer review, which reflects a traditional quality assurance approach, derived from manufacturing in the mid-1900s. Far from achieving the goals of the IOM, which are celebrating success, recognizing mistakes as an opportunity to learn, and fostering openness and trust, we have found that scoring-based peer review tends to drive radiologists inward, against each other, and against practice leaders. Modern approaches to quality improvement focus on using and enhancing interpersonal professional relationships to achieve and maintain high levels of individual and organizational performance. In this article, the authors review the recommendations set forth by the recent IOM report, discuss the science and theory that underlie several of those recommendations, and assess how well they fit with the current dominant approach to radiology peer review. The authors also offer an alternative approach to peer review: peer feedback, learning, and improvement (or more succinctly, "peer learning"), which they believe is better aligned with the principles promoted by the IOM. © RSNA, 2016.

Increased Cancer Detection Rate and Variations in the Recall Rate Resulting from Implementation of 3D Digital Breast Tomosynthesis into a Population-based Screening Program.

PURPOSE: To compare the recall and cancer detection rates (CDRs) at screening with digital breast tomosynthesis (DBT) with those at screening with two-dimensional (2D) mammography and to evaluate variations in the recall rate (RR) according to patient age, risk factors, and breast density and among individual radiologists at a single U.S. academic medical center. MATERIALS AND METHODS: This institutional review board-approved, HIPAA-compliant prospective study with a retrospective cohort included 85 852 asymptomatic women who presented for breast cancer screening over a 3-year period beginning in 2011. A DBT unit was introduced into the existing 2D mammography screening program, and patients were assigned to the first available machine. Ten breast-subspecialized radiologists interpreted approximately 90% of the examinations. RRs were calculated overall and according to patient age, breast density, and individual radiologist. CDRs were calculated. Single and multiple mixed-effect logistic regression analyses, χ(2) tests, and Bonferroni correction were utilized, as appropriate. RESULTS: The study included 5703 (6.6%) DBT examinations and 80 149 (93.4%) 2D mammography examinations. The DBT subgroup contained a higher proportion of patients with risk factors for breast cancer and baseline examinations. DBT was used to detect 54.3% more carcinomas (+1.9 per 1000, P < .0018) than 2D mammography. The RR was 7.51% for 2D mammography and 6.10% for DBT (absolute change, 1.41%; relative change, -18.8%; P < .0001). The DBT subgroup demonstrated a significantly lower RR for patients with extremely or heterogeneously dense breasts and for patients in their 5th and 7th decades. CONCLUSION: Implementing DBT into a U.S. breast cancer screening program significantly decreased the screening RR overall and for certain patient subgroups, while significantly increasing the CDR. These findings may encourage more widespread adoption and reimbursement of DBT and facilitate improved patient selection.

Strategic planning and radiology practice management in the new health care environment.

Current comprehensive health care reform in the United States demands that policy makers, insurers, providers, and patients work in reshaping the health care system to deliver care that is both more affordable and of higher quality. A tectonic shift is under way that runs contrary to the traditional goal of radiology groups to perform and interpret large numbers of imaging examinations. In fact, radiology service requisitions now must be evaluated for their appropriateness, possibly resulting in a reduction in the number of imaging studies performed. To be successful, radiology groups will have to restructure their business practices and strategies to align with the emerging health care paradigm. This article outlines a four-stage strategic framework that has aided corporations in achieving their goals and that can be readily adapted and applied by radiologists. The four stages are (a) definition and articulation of a purpose, (b) clear definition of strategic goals, (c) prioritization of specific strategic enablers, and (d) implementation of processes for tracking progress and enabling continuous adaptation. The authors provide practical guidance for applying specific tools such as analyses of strengths, weaknesses, opportunities, and threats (so-called SWOT analyses), prioritization matrices, and balanced scorecards to accomplish each stage. By adopting and applying these tools within the strategic framework outlined, radiology groups can position themselves to succeed in the evolving health care environment.

Including the method of detection for breast cancer in the Surveillance, Epidemiology, and End Results database is long overdue.

Debates about breast cancer screening have continued in part because the Surveillance, Epidemiology, and End Results database, which began in 1974, has never included the method of detection so that it has been impossible to determine the role that early detection has played in the major decline in deaths from breast cancer that we have seen in the US since 1990. Method of detection should be added to the Surveillance, Epidemiology, and End Results database as soon as possible.

Breast Cancer Method of Detection: 5-Year Outcomes Across a Multisite Health Care Enterprise.

PURPOSE: To investigate the feasibility and accuracy of radiologists categorizing the method of detection (MOD) when performing image-guided breast biopsies. METHODS: This retrospective, observational study was conducted across a health care enterprise that provides breast imaging services at 18 imaging sites across four US states. Radiologists used standardized templates to categorize the MOD, defined as the first test, sign, or symptom that triggered the subsequent workup and recommendation for biopsy. All image-guided breast biopsies since the implementation of the MOD-inclusive standardized template-from October 31, 2017 to July 6, 2023-were extracted. A random sample of biopsy reports was manually reviewed to evaluate the accuracy of MOD categorization. RESULTS: A total of 29,999 biopsies were analyzed. MOD was reported in 29,423 biopsies (98.1%) at a sustained rate that improved over time. The 10 MOD categories in this study included the following: 15,184 mammograms (51.6%); 4,561 MRIs (15.5%); 3,473 ultrasounds (11.8%); 2,382 self-examinations (8.1%); 2,073 tomosynthesis studies (7.0%); 432 clinical examinations (1.5%); 421 molecular breast imaging studies (1.4%); 357 other studies (1.2%); 338 contrast-enhanced digital mammograms (1.1%); and 202 PET studies (0.7%). Original assignments of the MOD agreed with author assignments in 87% of manually reviewed biopsies (n = 100, 95% confidence interval: [80.4%, 93.6%]). CONCLUSIONS: This study demonstrates that US radiologists can consistently and accurately categorize the MOD over an extended time across a health care enterprise.

ACR Appropriateness Criteria® Imaging of Invasive Breast Cancer.

As the proportion of women diagnosed with invasive breast cancer increases, the role of imaging for staging and surveillance purposes should be determined based on evidence-based guidelines. It is important to understand the indications for extent of disease evaluation and staging, as unnecessary imaging can delay care and even result in adverse outcomes. In asymptomatic patients that received treatment for curative intent, there is no role for imaging to screen for distant recurrence. Routine surveillance with an annual 2-D mammogram and/or tomosynthesis is recommended to detect an in-breast recurrence or a new primary breast cancer in women with a history of breast cancer, and MRI is increasingly used as an additional screening tool in this population, especially in women with dense breasts. The American College of Radiology Appropriateness Criteria are evidence-based guidelines for specific clinical conditions that are reviewed annually by a multidisciplinary expert panel. The guideline development and revision process support the systematic analysis of the medical literature from peer reviewed journals. Established methodology principles such as Grading of Recommendations Assessment, Development, and Evaluation or GRADE are adapted to evaluate the evidence. The RAND/UCLA Appropriateness Method User Manual provides the methodology to determine the appropriateness of imaging and treatment procedures for specific clinical scenarios. In those instances where peer reviewed literature is lacking or equivocal, experts may be the primary evidentiary source available to formulate a recommendation.

Trainee Report Dashboard: Tool for Enhancing Feedback to Radiology Trainees about Their Reports.

During their radiology residency, trainees must learn multiple facets of radiology practice, including the writing of radiology reports. An important factor in the trainee's development of reporting skills is feedback from the attending radiologist on the trainee's preliminary reports. The quality and quantity of feedback may vary and are not typically documented. As radiology department workloads have increased and stricter limitations have been imposed on trainee work hours, less time is available for attending radiologists and trainees to perform a joint retrospective review of radiology reports. To compensate, the authors have developed a Web-based dashboard that provides trainees with case-specific feedback about their reports. Components include an attending radiologist-trainee report discrepancy logging and communication system that is integrated with the institutional picture archiving and communication system, an automated preliminary report-final report comparator, modules showing statistics related to the discrepancy logger and report comparator components, and a Web page that unifies these components with image and report display capabilities. Both the actual report feedback and the trainee's use of the system are documented, and the resultant data may be used for evaluating trainee competence in written communication, as mandated by the Accreditation Council for Graduate Medical Education. With these tools, trainees can obtain near-real-time feedback, which may pinpoint issues that can be corrected to improve the quality of their radiology reporting. This system, although it does not supplant face-to-face training sessions with attending radiologists, can augment traditional methods of learning. © RSNA, 2013.

Trainee report dashboard: tool for enhancing feedback to radiology trainees about their reports.

During their radiology residency, trainees must learn multiple facets of radiology practice, including the writing of radiology reports. An important factor in the trainee's development of reporting skills is feedback from the attending radiologist on the trainee's preliminary reports. The quality and quantity of feedback may vary and are not typically documented. As radiology department workloads have increased and stricter limitations have been imposed on trainee work hours, less time is available for attending radiologists and trainees to perform a joint retrospective review of radiology reports. To compensate, the authors have developed a Web-based dashboard that provides trainees with case-specific feedback about their reports. Components include an attending radiologist-trainee report discrepancy logging and communication system that is integrated with the institutional picture archiving and communication system, an automated preliminary report-final report comparator, modules showing statistics related to the discrepancy logger and report comparator components, and a Web page that unifies these components with image and report display capabilities. Both the actual report feedback and the trainee's use of the system are documented, and the resultant data may be used for evaluating trainee competence in written communication, as mandated by the Accreditation Council for Graduate Medical Education. With these tools, trainees can obtain near-real-time feedback, which may pinpoint issues that can be corrected to improve the quality of their radiology reporting. This system, although it does not supplant face-to-face training sessions with attending radiologists, can augment traditional methods of learning.

QRSE: a novel metric for the evaluation of trainee radiologist reporting skills.

Diagnostic radiology training programs must produce highly skilled diagnostic radiologists capable of interpreting radiological examinations and communicating results to clinicians. Established training performance tools evaluate interpretive skills, but trainees' competency in reporting skills is also essential. Our semi-automated passive electronic tool entitled the Quantitative Reporting Skills Evaluation (QRSE) allows radiology training programs to evaluate the quantity of edits made to trainee preliminary reports by attending physicians as a metric to evaluate trainee reporting performance. Consecutive report pairs and metadata extracted from the radiology information system were anonymized and exported to a MySQL database. To perform the QRSE, for each report pair, open source software was first utilized to calculate the Levenshtein Percent (LP), the percent of character changes required to convert each preliminary report to its corresponding final report. The average LP (ALP), ALP for each trainee, and standard deviations were calculated. Eighty-four trainees and 56 attending radiologists interpreted 228,543 radiological examinations during the study period. The overall ALP was 6.38 %. Trainee-specific ALPs ranged from 1.1 to 15.3 %. Among trainee-specific ALPs, the standard deviation was 3.7 %. Our analysis identified five trainees with trainee-specific ALPs above 2 standard deviations from the mean and 14 trainees with trainee-specific ALPs less than 1 standard deviation below the mean. The QRSE methodology allows for the passive, quantitative, and longitudinal evaluation of the reporting skills of trainees during diagnostic radiology residency training. The QRSE identifies trainees with high and low levels of edits to their preliminary reports, as a marker for trainee overall reporting skills, and thus represents a novel performance metric for radiology training programs.