What Went Right? A Mixed-Methods Study of Positive Feedback Data in a Hospital-Wide Mortality Review Survey.

BACKGROUND: Toxic work culture contributes to healthcare worker burnout and attrition, but little is known about how healthcare organizations can systematically create and promote a culture of civility and collegiality. OBJECTIVE: To analyze peer-to-peer positive feedback collected as part of a systematized mortality review survey to identify themes and recognition dynamics that can inform positive organizational culture change. DESIGN: Convergent mixed-methods study design. PARTICIPANTS: A total of 388 physicians, 212 registered nurses, 64 advanced practice providers, and 1 respiratory therapist at four non-profit hospitals (2 academic and 2 community). INTERVENTION: Providing optional positive feedback in the mortality review survey. MAIN MEASURES: Key themes and subthemes that emerged from positive feedback data, associations between key themes and positive feedback respondent characteristics, and recognition dynamics between positive feedback respondents and recipients. KEY RESULTS: Approximately 20% of healthcare workers provided positive feedback. Three key themes emerged among responses with free text comments: (1) providing extraordinary patient and family-centered care; (2) demonstrating self-possession and mastery; and (3) exhibiting empathic peer support and effective team collaboration. Compared to other specialties, most positive feedback from medicine (70.2%), neurology (65.2%), hospice and palliative medicine (64.3%), and surgery (58.8%) focused on providing extraordinary patient and family-centered care (p = 0.02), whereas emergency medicine (59.1%) comments predominantly focused on demonstrating self-possession and mastery (p = 0.06). Registered nurses (40.2%) provided multidirectional positive feedback more often than other clinician types in the hospital hierarchy (p < 0.001). CONCLUSIONS: Analysis of positive feedback from a mortality review survey provided meaningful insights into a health system's culture of teamwork and values related to civility and collegiality when providing end-of-life care. Systematic collection and sharing of positive feedback is feasible and has the potential to promote positive culture change and improve healthcare worker well-being.

A Public Health Critical Race Praxis Informed Congestive Heart Failure Quality Improvement Initiative on Inpatient General Medicine.

BACKGROUND: Prior evaluation at our hospital demonstrated that, compared to White patients, Black and Latinx patients with congestive heart failure (CHF) were less likely to be admitted to the cardiology service rather than the general medicine service (GMS). Patients admitted to GMS (compared to cardiology) had inferior rates of cardiology follow-up and 30-day readmission. OBJECTIVE: To develop and test the feasibility and impacts of using quality improvement (QI) methods, in combination with the Public Health Critical Race Praxis (PHCRP) framework, to engage stakeholders in developing an intervention for ensuring guideline-concordant inpatient CHF care across all patient groups. METHODS: We compared measures for all patients admitted with CHF to GMS between September 2019 and March 2020 (intervention group) to CHF patients admitted to GMS in the previous year (pre-intervention group) and those admitted to cardiology during the pre-intervention and intervention periods (cardiology group). Our primary measures were 30-day readmissions and 14- and 30-day post-discharge cardiology follow-up. RESULTS: There were 79 patients admitted with CHF to GMS during the intervention period, all of whom received the intervention. There were similar rates of Black and Latinx patients across the three groups. Compared to pre-intervention, intervention patients had a significantly lower 30-day readmission rate (18.9% vs. 24.8%; p=0.024), though the cardiology group also had a decrease in 30-day readmissions from the pre-intervention to intervention period. Compared to pre-intervention, intervention patients had significantly higher 14-day and 30-day post-discharge follow-up visits scheduled with cardiology (36.7% vs. 24.8%, p=0.005; 55.7% vs. 42.3%, p=0.0029), but no improvement in appointment attendance. CONCLUSION: This study provides a first test of applying the PHCRP framework within a stakeholder-engaged QI initiative for improving CHF care across races and ethnicities. Our study design cannot evaluate causation. However, the improvements in 30-day readmission, as well as in processes of care that may affect it, provide optimism that inclusion of a racism-conscious framework in QI initiatives is feasible and may enhance QI measures.

Predictors of Completion of Clinically Necessary Radiologist-Recommended Follow-Up Imaging: Assessment Using an Automated Closed-Loop Communication and Tracking Tool.

BACKGROUND. Patients with adverse social determinants of health may be at increased risk of not completing clinically necessary follow-up imaging. OBJECTIVE. The purpose of this study was to use an automated closed-loop communication and tracking tool to identify patient-, referrer-, and imaging-related factors associated with lack of completion of radiologist-recommended follow-up imaging. METHODS. This retrospective study was performed at a single academic health system. A tool for automated communication and tracking of radiologist-recommended follow-up imaging was embedded in the PACS and electronic health record. The tool prompted referrers to record whether they deemed recommendations to be clinically necessary and assessed whether clinically necessary follow-up imaging was pursued. If imaging was not performed within 1 month after the intended completion date, the tool prompted a safety net team to conduct further patient and referrer follow-up. The study included patients for whom a follow-up imaging recommendation deemed clinically necessary by the referrer was entered with the tool from October 21, 2019, through June 30, 2021. The electronic health record was reviewed for documentation of eventual completion of the recommended imaging at the study institution or an outside institution. Multivariable logistic regression analysis was performed to identify factors associated with completion of follow-up imaging. RESULTS. Of 5856 recommendations entered during the study period, the referrer agreed with 4881 recommendations in 4599 patients (2929 women, 1670 men; mean age, 61.3 ± 15.6 years), who formed the study sample. Follow-up was completed for 74.8% (3651/4881) of recommendations. Independent predictors of lower likelihood of completing follow-up imaging included living in a socioeconomically disadvantaged neighborhood according to the area deprivation index (odds ratio [OR], 0.67 [95% CI, 0.54-0.84]), inpatient (OR, 0.25 [95% CI, 0.20-0.32]) or emergency department (OR, 0.09 [95% CI, 0.05-0.15]) care setting, and referrer surgical specialty (OR, 0.70 [95% CI, 0.58-0.84]). Patient age, race and ethnicity, primary language, and insurance status were not independent predictors of completing follow-up (p > .05). CONCLUSION. Socioeconomically disadvantaged patients are at increased risk of not completing recommended follow-up imaging that referrers deem clinically necessary. CLINICAL IMPACT. Initiatives for ensuring completion of follow-up imaging should be aimed at the identified patient groups to reduce disparities in missed and delayed diagnoses.

Development and External Validation of an Artificial Intelligence Model for Identifying Radiology Reports Containing Recommendations for Additional Imaging.

BACKGROUND. Reported rates of recommendations for additional imaging (RAIs) in radiology reports are low. Bidirectional encoder representations from transformers (BERT), a deep learning model pretrained to understand language context and ambiguity, has potential for identifying RAIs and thereby assisting large-scale quality improvement efforts. OBJECTIVE. The purpose of this study was to develop and externally validate an artificial intelligence (AI)-based model for identifying radiology reports containing RAIs. METHODS. This retrospective study was performed at a multisite health center. A total of 6300 radiology reports generated at one site from January 1, 2015, to June 30, 2021, were randomly selected and split by 4:1 ratio to create training (n = 5040) and test (n = 1260) sets. A total of 1260 reports generated at the center's other sites (including academic and community hospitals) from April 1 to April 30, 2022, were randomly selected as an external validation group. Referring practitioners and radiologists of varying sub-specialties manually reviewed report impressions for presence of RAIs. A BERT-based technique for identifying RAIs was developed by use of the training set. Performance of the BERT-based model and a previously developed traditional machine learning (TML) model was assessed in the test set. Finally, performance was assessed in the external validation set. The code for the BERT-based RAI model is publicly available. RESULTS. Among a total of 7419 unique patients (4133 women, 3286 men; mean age, 58.8 years), 10.0% of 7560 reports contained RAI. In the test set, the BERT-based model had 94.4% precision, 98.5% recall, and an F1 score of 96.4%. In the test set, the TML model had 69.0% precision, 65.4% recall, and an F1 score of 67.2%. In the test set, accuracy was greater for the BERT-based than for the TML model (99.2% vs 93.1%, p < .001). In the external validation set, the BERT-based model had 99.2% precision, 91.6% recall, an F1 score of 95.2%, and 99.0% accuracy. CONCLUSION. The BERT-based AI model accurately identified reports with RAIs, outperforming the TML model. High performance in the external validation set suggests the potential for other health systems to adapt the model without requiring institution-specific training. CLINICAL IMPACT. The model could potentially be used for real-time EHR monitoring for RAIs and other improvement initiatives to help ensure timely performance of clinically necessary recommended follow-up.

Improving hepatitis B vaccination rates for advanced chronic kidney disease patients: a quality improvement initiative.

INTRODUCTION: Chronic kidney disease (CKD) patients are vulnerable to hepatitis B, and immunization prior to end stage kidney disease is recommended to optimize seroconversion. Our institution undertook a process improvement approach to increase hepatitis B vaccination in stage 4 and 5 CKD patients. METHODS: Four strategies were utilized such as: (1) Electronic health record (EHR)-based CKD registry to identify patients, (2) EHR-based physician/nurse reminders, (3) a co-located nurse appointment for vaccine administration, and (4) information sharing and provider awareness effort. The CKD registry was utilized to identify patients with stage 4 or 5 CKD, with at least two clinic visits in the prior 2 years, who had not received the hepatitis B vaccine or did not have serologic evidence of immunity. Target monthly vaccination rate was set at 75%, based on clinic leadership, nephrologist, and nurse consensus. RESULTS: A total of 239 patients were included in the study period, from November 2018 to January 2019 (observation period) and from February 2019 to September 2019 (intervention period). Monthly vaccination rate improved from 48% in November 2018 to the target rate of 75% by the end of the intervention (August and September 2019). There was a statistically significant increase from the rate of vaccination at a unique patient level in the first month of the baseline period, compared to the last month of the intervention period (51 vs. 75% p = 0.03). CONCLUSIONS: Utilizing a nurse-led approach to hepatitis B vaccination, coupled with EHR-based tools, along with continuous monitoring of performance, helped to improve hepatitis B vaccination among CKD stage 4 and 5 patients.

Quality improvement for rheumatoid arthritis care: results from a focus group.

OBJECTIVES: Complex treatment decisions in rheumatoid arthritis (RA) affect aspects of patients' physical, psychological and emotional well-being. We aimed to identify key attributes of patient-centered rheumatologic care for adults with RA through a qualitative study using patient focus group discussions in order to guide quality improvement efforts around optimisation of disease management. METHODS: Patients with RA were recruited from a large academic medical centre rheumatology clinic and its affiliate sites over one month and allocated into focus groups led by an experienced moderator. Focus groups were held until thematic saturation was reached. Patients' responses were examined, categorised into themes, and codified independently by three reviewers. We extracted statements identifying common themes from transcripts. RESULTS: Thirteen patients with RA were recruited and allocated into three focus groups. Mean age was 59.1±10.1 years and average RA disease duration was 17.8 years. All participants had experience taking at least one disease-modifying anti-rheumatic drug (DMARD). Following reviewer analysis of patients' responses, six common themes about quality RA care were identified including: the role and use of self-management strategies, the clinical environment, the health care delivery process, attitudes towards medication, insurance and medication access issues, and the impact of disease on lifestyle. CONCLUSIONS: Themes uncovered in focus group discussions related predominantly to the clinical environment and patient-provider communication. These preliminary results identified the need to incorporate operational aspects of health care delivery into our assessment of the RA patient experience and formed the basis of a RA quality improvement programme targeting medication optimisation.

Adoption of a Closed-Loop Communication Tool to Establish and Execute a Collaborative Follow-Up Plan for Incidental Pulmonary Nodules.

OBJECTIVE. The purpose of this study is to assess radiologists' adoption of a closed-loop communication and tracking system, Result Alert and Development of Automated Resolution (RADAR), for incidental pulmonary nodules and to measure its effect on the completeness of radiologists' follow-up recommendations. MATERIALS AND METHODS. This retrospective study was performed at a tertiary academic center that performs more than 600,000 radiology examinations annually. Before RADAR, the institution's standard of care was for radiologists to generate alerts for newly discovered incidental pulmonary nodules using a previously described PACS-embedded software tool. RADAR is a new closed-loop communication tool embedded in the PACS and enterprise provider workflow that enables establishing a collaborative follow-up plan between a radiologist and referring provider and helps automate collaborative follow-up plan tracking and execution. We assessed RADAR adoption for incidental pulmonary nodules, the primary outcome, in our thoracic radiology division (study period March 9, 2018, through August 2, 2018). The secondary outcome was the completeness of follow-up recommendation for incidental pulmonary nodules, defined as explicit imaging modality and time frame for follow-up. RESULTS. After implementation, 106 of 183 (58%) incidental pulmonary nodules alerts were generated using RADAR. RADAR adoption increased by 75% during the study period (40% in the first 3 weeks vs 70% in the last 3 weeks; p < 0.001 test for trend). All RADAR alerts had explicit documentation of imaging modality and follow-up time frame, compared with 71% for non-RADAR alerts for incidental pulmonary nodules (p < 0.001). CONCLUSION. A closed-loop communication system that enables establishing and executing a collaborative follow-up plan for incidental pulmonary nodules can be adopted and improves the quality of radiologists' follow-up recommendations.

Understanding Test Results Follow-Up in the Ambulatory Setting: Analysis of Multiple Perspectives.

BACKGROUND: Delayed or incomplete test result follow-up, which can lead to missed and/or delayed diagnosis, is an important issue in the ambulatory setting. Delayed test result follow-up has been linked to poorer patient outcomes and increased risk of mortality and accounts for a large portion of medical malpractice claims. Yet improvements are difficult, reflecting the complexity of the test result follow-up process. Test result follow-up safety culture was investigated using qualitative and quantitative patient safety and quality of care data at an academic medical center. METHODS: After an environmental scan, five sources of data were used to compass multiple perspectives on safety culture-two national surveys (AHRQ MO SOPS for safety culture and CG-CAHPS for patient satisfaction); patient and family complaints; safety reports; and provider response times to test message results in the electronic health record. RESULTS: The following metrics were inspected: how patients and providers estimated the frequency for providing timely test results; how patients' satisfaction with their provider correlated with their provider's response time to test result messages; and qualitative themes in patient complaints and safety reports filed by clinic. The institution was compared to national benchmarks using surveys. As test result response time decreased, patient satisfaction increased (p = 0.0073). CONCLUSION: Test result follow-up culture was investigated using tools typically used to examine patient satisfaction and experience and staff culture. Use of these five sources of data led to an examination of multiple perspectives in follow-up culture and identification of possible explanations for inappropriate follow-up. These data sources can be further explored to identify possible solutions.

Impact of a Health Information Technology Intervention on the Follow-up Management of Pulmonary Nodules.

Lung cancer is the leading cause of cancer deaths in the USA. The most common abnormalities suspicious for lung cancer on CT scan include pulmonary nodules. Recommendations to improve care for patients with pulmonary nodules require follow-up management. However, transitions in care, especially for patients undergoing transitions to ambulatory care sites from the emergency department (ED) and inpatient settings, can exacerbate failures in follow-up testing and compromise patient safety. We evaluate the impact of a discharge module that includes follow-up recommendations for further management of pulmonary nodules on the study outcome and follow-up management of patients with pulmonary nodules within 1 year after discharge. After IRB approval, we collected data on all patients undergoing chest or abdominal CT exams over a 12-month baseline and 12-month intervention period at an academic medical center. The inpatient discharge module was implemented in November 2011; the ED module was implemented in May 2012. Multivariable logistic regression was performed to account for care setting, imaging modality, recommendations, and patient demographics. Implementation of a discharge module resulted in improved follow-up of patients with pulmonary nodules within 1 year after discharge (OR = 1.64, p = 0.01); the ED implementation resulted in better follow-up compared to the inpatient module (OR = 2.24, p < 0.01). Twenty-seven percent of patients with pulmonary nodules received follow-up management, which, although significantly improved from the 18% baseline, remains low. An electronic discharge module is associated with improved follow-up management of patients with pulmonary nodules, and may be combined with interventions to further improve management of these patients.

Processes for Identifying and Reviewing Adverse Events and Near Misses at an Academic Medical Center.

BACKGROUND: Conferences, processes, and/or meetings in which adverse events and near misses are reviewed within clinical programs at a single academic medical center were identified. METHODS: Leaders of conferences, processes, or meetings-"process leaders"-in which adverse events and near misses were reviewed were surveyed. RESULTS: On the basis of responses from all 45 process leaders, processes were classified into (1) Morbidity and Mortality Conferences (MMCs), (2) Quality Assurance (QA) Meetings, and (3) Educational Conferences. Some 22% of the clinical programs used more than one of these three processes to identify and review adverse events and near misses, while 10% had no consistent participation in any of them. Explicit criteria for identifying and selecting cases to be reviewed were used by 58% of MMCs and 69% of QA Meetings. The explicit criteria used by MMCs and QA Meetings varied widely. Many MMCs (54%, 13/24), QA Meetings (54%, 7/13), and Educational Conferences (70%, 7/10) did not review all the adverse events or near misses that were identified, and several MMCs (46%, 6/13), QA Meetings (29%, 2/7), and Educational Conferences (57%, 4/7) had no other process within their clinical program by which to review these remaining cases. CONCLUSIONS: There was wide variation regarding how clinical programs identify and review adverse events and near misses within the MMCs, QA Meetings, and Educational Conferences, and some programs had no such processes. A well-designed, coordinated process across all clinical areas that incorporates accepted approaches for event analysis may improve the quality and safety of patient care.

Ambulatory Medication Reconciliation: Using a Collaborative Approach to Process Improvement at an Academic Medical Center.

BACKGROUND: Incomplete medication reconciliation has been identified as a source of adverse drug events and a threat to patient safety. How best to measure and improve rates of medication reconciliation in ambulatory care remains unknown. METHODS: An institutional collaborative improvement effort to develop and implement medication reconciliation processes was designed and facilitated across all 148 Brigham and Women's Hospital (Boston) ambulatory specialty practices: 63 underwent a more rigorous approach, a modified approach was undertaken in another 71 specialty practices, and a less intensive approach took place in the 14 primary care practices. The level of intervention varied on the basis of preexisting improvement infrastructure and practice prescription rates. Two electronically measured metrics were created to evaluate ambulatory visits to a provider in which there was a medication change: (1) Measure 1: the percentage of active medications prescribed by that provider that were reconciled; and (2) Measure 2: how often all the medications prescribed by that provider were reconciled. After the collaborative was completed, performance data were routinely shared with frontline staff and hospital leadership, and medication reconciliation rates became part of an institutional financial incentive program. RESULTS: For Measure 1, specialty practices improved from 71% to 90% (September 2012-August 2014; 24-month period). Primary care practice performance improved from 62% to 91% (December 2012-August 2014; 20-month period). For Measure 2, overall performance across all ambulatory practices increased from 81% to 90% during the first 12 months of the financial incentive program (October 2013- September 2014). CONCLUSION: A collaborative model of process improvement paired with financial incentives can successfully increase rates of ambulatory medication reconciliation.

Increasing pneumococcal vaccination for immunosuppressed patients: a cluster quality improvement trial.

OBJECTIVE: Pneumococcal vaccination is important for patients taking immunosuppressive medications, but prior studies suggest that most patients do not undergo vaccination. The aim of this study was to evaluate the effects of a point-of-care paper reminder form as a quality improvement (QI) strategy to increase the numbers of immunosuppressed patients being kept up-to-date with pneumococcal vaccination in a rheumatology practice. METHODS: Selected rheumatologists at 5 ambulatory practice sites received a point-of-care paper reminder form to be applied to patients who were not up-to-date with pneumococcal vaccination. Interrupted time-series analyses were used to measure the effect of the intervention on the pneumococcal vaccination rates among patients, comparing the rates in the intervention group with those in a control group of rheumatologists who did not receive the intervention. Adjusted Cox proportional hazards models were examined to identify independent predictors of being up-to-date with pneumococcal vaccination. RESULTS: We evaluated a total of 3,717 patients (66.0% with rheumatoid arthritis) who were taking immunosuppressive medications (74.1% women, mean age 53.7 years). Rheumatologists who received the intervention had a significant increase in the rate of patients who were up-to-date with pneumococcal vaccination, from 67.6% to 80.0% (P=0.006), in the time period following the intervention, compared to a rate that remained stable, from 52.3% to 52.0% (P=0.90), among patients in the nonintervention control group during this same time period. In regression models, positive predictors of being up-to-date with pneumococcal vaccination at the patient level included the following: having received the intervention (hazard ratio [HR] 3.58, 95% confidence interval [95% CI] 2.46-5.20), having a primary care physician affiliated with Brigham and Women's Hospital (HR 1.68, 95% CI 1.44-1.97), having a diagnosis of diabetes mellitus (HR 1.57, 95% CI 1.02-2.41), and being age 56-65 years at baseline, compared to age≤45 years (HR 1.24, 95% CI 1.01-1.51). CONCLUSION: A QI strategy involving a simple point-of-care paper reminder form significantly increased the rate of being up-to-date with pneumococcal vaccination among patients receiving immunosuppressive medications in our rheumatology practices over a 6-month period.

Neuroinvasive Bacillus cereus Infection in Immunocompromised Hosts: Epidemiologic Investigation of 5 Patients With Acute Myeloid Leukemia.

Background: Bacillus cereus is a ubiquitous gram-positive rod-shaped bacterium that can cause sepsis and neuroinvasive disease in patients with acute leukemia or neutropenia. Methods: A single-center retrospective review was conducted to evaluate patients with acute leukemia, positive blood or cerebrospinal fluid test results for B cereus, and abnormal neuroradiographic findings between January 2018 and October 2022. Infection control practices were observed, environmental samples obtained, a dietary case-control study completed, and whole genome sequencing performed on environmental and clinical Bacillus isolates. Results: Five patients with B cereus neuroinvasive disease were identified. All patients had acute myeloid leukemia (AML), were receiving induction chemotherapy, and were neutropenic. Neurologic involvement included subarachnoid or intraparenchymal hemorrhage or brain abscess. All patients were treated with ciprofloxacin and survived with limited or no neurologic sequelae. B cereus was identified in 7 of 61 environmental samples and 1 of 19 dietary protein samples-these were unrelated to clinical isolates via sequencing. No point source was identified. Ciprofloxacin was added to the empiric antimicrobial regimen for patients with AML and prolonged or recurrent neutropenic fevers; no new cases were identified in the ensuing year. Conclusions: B cereus is ubiquitous in the hospital environment, at times leading to clusters with unrelated isolates. Fastidious infection control practices addressing a range of possible exposures are warranted, but their efficacy is unknown and they may not be sufficient to prevent all infections. Thus, including B cereus coverage in empiric regimens for patients with AML and persistent neutropenic fever may limit the morbidity of this pathogen.

Ambulatory Medication Safety: Vaccination and Laboratory Screening for Patients Receiving Immunomodulatory Therapies.

OBJECTIVE: Immunomodulatory therapies improve the management of chronic diseases but can be associated with infectious risk. The present study was undertaken to examine the laboratory screening practices for hepatitis B virus (HBV), hepatitis C virus (HCV), and tuberculosis (TB) and rates of vaccination for pneumococcal and influenza in patients prescribed select immunosuppressive agents at our institution. METHODS: A retrospective analysis was conducted to review patients who were prescribed a select immunosuppressive over 3 years. Data were extracted from electronic health records to identify rates of screening and vaccination prior to initiation or at any time. Logistic regression models were developed to identify predictors of screening and vaccination. RESULTS: We identified 2,396 patients prescribed immunosuppressive medications by rheumatology (52.6%) and non-rheumatology specialties. Rates of screening at any time point were 84.5% (2,025 of 2,396) for HBV, 76.7% (1,838 of 2,396) for HCV, and 71.8% (1,720 of 2,396) for TB. Patients who had either in-system primary care providers (PCPs) or rheumatologists were more likely to receive pneumococcal vaccinations (odds ratio [OR] 1.98 [95% confidence interval (95% CI) 1.55-2.54] and OR 4.08 [95% CI 2.76-6.02], respectively). Patients with dermatologic (OR 1.67 [95% CI 1.14-2.45]) or rheumatologic providers (OR 2.5 [95% CI 1.86-3.36]) were more likely to be vaccinated for influenza. CONCLUSION: More than 70% of patients were screened for either HBV, HCV, or TB at some point. Rates of pneumococcal vaccination were better than rates of influenza vaccination. Patients with in-system PCPs were more likely to be screened and vaccinated. Establishing and executing consistent processes for screening and vaccination prior to immunosuppressive treatment remains a priority in ambulatory settings.

Patient and System Factors Associated With Completed Follow-Up of Probably Benign (BI-RADS 3) Breast Imaging Findings.

OBJECTIVE: Evaluate patient factors and health system test ordering and scheduling processes associated with completed BI-RADS 3 breast imaging follow-up. METHODS: Retrospective review of reports from January 1, 2021, to July 31, 2021, identified BI-RADS 3 findings corresponding to unique patient encounters (index examinations). The electronic health record was queried for patient, examination, and health system ordering or scheduling data including follow-up order status (order placed, performed; order placed, scheduled, but not performed; order placed, unscheduled; no order placed); ordering provider specialty and health system affiliation (primary care versus other, internal versus external to health system); and ordering department (radiology staff versus referring physician staff). Patient home addresses were categorized by area deprivation index (University of Wisconsin's Neighborhood Atlas). Univariable and multivariable analysis identified patient, examination, and ordering or scheduling factors associated with completed follow-up imaging within 15 months of BI-RADS 3 assessment. RESULTS: There were 3,104 unique BI-RADS 3 assessments, 2,561 (82.5%) with completed BI-RADS 3 follow-up within 15 months of study examination. In multivariable analysis, factors associated with incomplete follow-up included ultrasound (odds ratio [OR] 0.48; 95% confidence interval [95% CI] 0.38-0.60; P < .001) and MRI (OR 0.71; 95% CI 0.50-1.00; P = .049) versus mammogram; patients living in the highest disadvantaged neighborhoods (OR 0.70; 95% CI 0.50-0.98; P = .04); patients <40 years (OR 0.14; 95% CI 0.11-0.19; P < .001); Asian race (OR 0.55; 95% CI 0.37-0.81; P = .003); order placement >3 months (OR, 0.05; 95% CI 0.02-0.16; P < .001) after index examination or scheduling >6 months after order placement (OR, 0.35; 95% CI 0.14-0.87; P = .02); order placement by breast oncology or breast surgery departments (OR 0.35; 95% CI 0.17-0.73; P = .01) versus radiology department. DISCUSSION: Incomplete BI-RADS 3 follow-up is associated with ultrasound or MRI, most socioeconomically disadvantaged patients, younger patients, Asian race, delayed order entry, and follow-up examination ordering and scheduling by non-radiology departments.

Development and Assessment of an Information Technology Intervention to Improve the Clarity of Radiologist Follow-up Recommendations.

Importance: It is challenging to ensure timely performance of radiologist-recommended additional imaging when radiologist recommendation language is incomplete or ambiguous. Objective: To evaluate whether voluntary use of an information technology tool with forced structured entry of recommendation attributes was associated with improved completeness of recommendations for additional imaging over time. Design, Setting, and Participants: This cohort study of imaging report data was performed at an academic quaternary care center in Boston, Massachusetts, and included consecutive adults with radiology examinations performed from September 12 to 13, 2019 (taxonomy validation), October 14 to 17, 2019 (before intervention), April 5 to 7, 2021 (1 week after intervention), and April 4 to 7, 2022 (1 year after intervention), with reports containing recommendations for additional imaging. A radiologist scored the 3 groups (preintervention group, 1-week postintervention group, and 1-year postintervention group) of 336 consecutive radiology reports (n = 1008) with recommendations for additional imaging. Intervention: Final implementation on March 27, 2021, of a voluntary closed-loop communication tool embedded in radiologist clinical workflow that required structured entry of recommendation attributes. Main Outcomes and Measures: The a priori primary outcome was completeness of recommendations for additional imaging, defined in a taxonomy created by a multidisciplinary expert panel. To validate the taxonomy, 2 radiologists independently reviewed and scored language attributes as present or absent in 247 consecutive radiology reports containing recommendations for additional imaging. Agreement was assessed with Cohen κ. Recommendation completeness over time was compared with with 1-sided Fisher exact tests and significance set at P < .05. Results: Radiology-related information for consecutive radiology reports from the 4 time periods was collected from the radiology department data warehouse, which does not include data on patient demographic characteristics or other nonimaging patient medical information. The panel defined 5 recommendation language attributes: complete (contains imaging modality, time frame, and rationale), ambiguous (equivocal, vague language), conditional (qualifying language), multiplicity (multiple options), and alternate (language favoring a different examination to that ordered). Two radiologists had more than 90% agreement (κ > 0.8) for these attributes. Completeness with use of the tool increased more than 3-fold, from 14% (46 of 336) before the intervention to 46% (153 of 336) (P < .001) 1 year after intervention; completeness in the corresponding free-text report language increased from 14% (46 of 336) before the intervention to 25% (85 of 336) (P < .001) 1 year after the intervention. Conclusions and Relevance: This study suggests that supplementing free-text dictation with voluntary use of a structured entry tool was associated with improved completeness of radiologist recommendations for additional imaging as assessed by an internally validated taxonomy. Future research is needed to assess the association with timely performance of clinically necessary recommendations and diagnostic errors. The taxonomy can be used to evaluate and build interventions to modify radiologist reporting behaviors.

Impact of an Automated Closed-Loop Communication and Tracking Tool on the Rate of Recommendations for Additional Imaging in Thoracic Radiology Reports.

OBJECTIVE: Assess the effects of feedback reports and implementing a closed-loop communication system on rates of recommendations for additional imaging (RAIs) in thoracic radiology reports. METHODS: In this retrospective, institutional review board-approved study at an academic quaternary care hospital, we analyzed 176,498 thoracic radiology reports during a pre-intervention (baseline) period from April 1, 2018, to November 30, 2018; a feedback report only period from December 1, 2018, to September 30, 2019; and a closed-loop communication system plus feedback report (IT intervention) period from October 1, 2019, to December 31, 2020, promoting explicit documentation of rationale, time frame, and imaging modality for RAI, defined as complete RAI. A previously validated natural language processing tool was used to classify reports with an RAI. Primary outcome of rate of RAI was compared using a control chart. Multivariable logistic regression determined factors associated with likelihood of RAI. We also estimated the completeness of RAI in reports comparing IT intervention to baseline using χ2 statistic. RESULTS: The natural language processing tool classified 3.2% (5,682 of 176,498) reports as having an RAI; 3.5% (1,783 of 51,323) during the pre-intervention period, 3.8% (2,147 of 56,722) during the feedback report only period (odds ratio: 1.1, P = .03), and 2.6% (1,752 of 68,453) during the IT intervention period (odds ratio: 0.60, P < .001). In subanalysis, the proportion of incomplete RAI decreased from 84.0% (79 of 94) during the pre-intervention period to 48.5% (47 of 97) during the IT intervention period (P < .001). DISCUSSION: Feedback reports alone increased RAI rates, and an IT intervention promoting documentation of complete RAI in addition to feedback reports led to significant reductions in RAI rate, incomplete RAI, and improved overall completeness of the radiology recommendations.

Financial Impact of a Radiology Safety Net Program for Resolution of Clinically Necessary Follow-up Imaging Recommendations.

OBJECTIVE: Health care safety net (SN) programs can potentially improve patient safety and decrease risk associated with missed or delayed follow-up care, although they require financial resources. This study aimed to assess whether the revenue generated from completion of clinically necessary recommendations for additional imaging (RAI) made possible by an IT-enabled SN program could fund the required additional labor resources. METHODS: Clinically necessary RAI generated October 21, 2019, to September 24, 2021, were tracked to resolution as of April 13, 2023. A new radiology SN team worked with existing schedulers and care coordinators, performing chart review and patient and provider outreach to ensure RAI resolution. We applied relevant Current Procedural Terminology, version 4 codes of the completed imaging examinations to estimate total revenue. Coprimary outcomes included revenue generated by total performed examinations and estimated revenue attributed to SN involvement. We used Student's t test to compare the secondary outcome, RAI time interval, for higher versus lower revenue-generating modalities. RESULTS: In all, 24% (3,243) of eligible follow-up recommendations (13,670) required SN involvement. Total estimated revenue generated by performed recommended examinations was $6,116,871, with $980,628 attributed to SN. Net SN-generated revenue per 1.0 full-time equivalent was an estimated $349,768. Greatest proportion of performed examinations were cross-sectional modalities (CT, MRI, PET/CT), which were higher revenue-generating than non-cross-sectional modalities (x-ray, ultrasound, mammography), and had shorter recommendation time frames (153 versus 180 days, P < .001). DISCUSSION: The revenue generated from completion of RAI facilitated by an IT-enabled quality and safety program supplemented by an SN team can fund the required additional labor resources to improve patient safety. Realizing early revenue may require 5 to 6 months postimplementation.

Ambulatory Virtual Care During a Pandemic: Patient Safety Considerations.

BACKGROUND: The COVID-19 pandemic prompted sudden and fundamental changes in health care, including a rapid rise in the utilization of telehealth services in the ambulatory setting. With the unprecedented and significant decline in traditional office-based visits and procedures, novel patient safety risks and challenges emerged. METHODS: The ambulatory practices at our quaternary care, academic medical center experienced a 200-fold increase in virtual visit volume between February and April 2020. We convened a multidisciplinary working group dedicated to evaluating quality and safety when providing virtual visits during a pandemic. Our primary outcome was patient experience with virtual care delivery, which was assessed by leveraging patient complaint data and patient satisfaction survey data. RESULTS: For our main focus of patient experience and satisfaction, survey data were analyzed from the approximately 76,616 virtual visit encounters that occurred between March 1, 2020, and April 21, 2020. During this period, 5 patient complaints were filed to the Patient Advocacy Department. Overall, patient satisfaction with telehealth remained stable and high at >93% from February to May 2020. As we assessed these data each month, our working group developed risk mitigation strategies in response to the novel challenges presented by the use of telemedicine due to the COVID-19 pandemic while working to maintain patient satisfaction with care. We identified quality and safety issues around patient factors including optimal triage of patients and use of technology. We also evaluated accessibility to virtual platforms and logistics such as coordination of care for diagnostic testing. Finally, a guidance document was created and communicated to our diverse ambulatory practices to support clinicians. CONCLUSIONS: Ambulatory virtual care delivery requires a dynamic, flexible model of care through continuous rapid-cycle process improvement to mitigate patient safety risks during a pandemic, incorporating both provider and patient perspectives.