Do patients who read visit notes on the patient portal have a higher rate of "loop closure" on diagnostic tests and referrals in primary care? A retrospective cohort study.

OBJECTIVES: The 2021 US Cures Act may engage patients to help reduce diagnostic errors/delays. We examined the relationship between patient portal registration with/without note reading and test/referral completion in primary care. MATERIALS AND METHODS: Retrospective cohort study of patients with visits from January 1, 2018 to December 31, 2021, and order for (1) colonoscopy, (2) dermatology referral for concerning lesions, or (3) cardiac stress test at 2 academic primary care clinics. We examined differences in timely completion ("loop closure") of tests/referrals for (1) patients who used the portal and read ≥1 note (Portal + Notes); (2) those with a portal account but who did not read notes (Portal Account Only); and (3) those who did not register for the portal (No Portal). We estimated the predictive probability of loop closure in each group after adjusting for socio-demographic and clinical factors using multivariable logistic regression. RESULTS: Among 12 849 tests/referrals, loop closure was more common among Portal+Note-readers compared to their counterparts for all tests/referrals (54.2% No Portal, 57.4% Portal Account Only, 61.6% Portal+Notes, P < .001). In adjusted analysis, compared to the No Portal group, the odds of loop closure were significantly higher for Portal Account Only (OR 1.2; 95% CI, 1.1-1.4), and Portal+Notes (OR 1.4; 95% CI, 1.3-1.6) groups. Beyond portal registration, note reading was independently associated with loop closure (P = .002). DISCUSSION AND CONCLUSION: Compared to no portal registration, the odds of loop closure were 20% higher in tests/referrals for patients with a portal account, and 40% higher in tests/referrals for note readers, after controlling for sociodemographic and clinical factors. However, important safety gaps from unclosed loops remain, requiring additional engagement strategies.

Low Rate of Completion of Recommended Tests and Referrals in an Academic Primary Care Practice with Resident Trainees.

BACKGROUND: A frequent, preventable cause of diagnostic errors involves failure to follow up on diagnostic tests, referrals, and symptoms-termed "failure to close the diagnostic loop." This is particularly challenging in a resident practice where one third of physicians graduate annually, and rates of patient loss due to these transitions may lead to more opportunities for failure to close diagnostic loops. The aim of this study was to determine the prevalence of failure of loop closure in a resident primary care clinic compared to rates in the faculty practice and identify factors contributing to failure. METHODS: This retrospective cohort study included all patient visits from January 1, 2018, to December 31, 2021, at two academic medical center-based primary care practices where residents and faculty practice in the same setting. The primary outcome was prevalence of failure to close the loop for (1) dermatology referrals, (2) colonoscopy, and (3) cardiac stress testing. The primary predictor was resident vs. faculty status of the ordering provider. The authors present an unadjusted analysis and the results of a multivariable logistic regression analysis incorporating all patient factors to determine their association with loop closure. RESULTS: Of 12,282 orders for referrals and tests for the three studied areas, 1,929 (15.7%) were ordered by a resident physician. Of resident orders for all three tests, 52.9% were completed within the designated time vs. 58.4% for orders placed by attending physicians (p < 0.01). In an unadjusted analysis by test type, a similar trend was seen for colonoscopy (51.4% completion rate for residents vs. 57.5% for attending physicians, p < 0.01) and for cardiac stress testing (55.7% completion rate for residents vs. 61.2% for attending physicians), though a difference was not seen for dermatology referrals (64.2% completion rate for residents vs. 63.7% for attending physicians). In an adjusted analysis, patients with resident orders were less likely than attendings to close the loop for all test types combined (odds ratio 0.88, 95% confidence interval 0.79-0.98), with low rates of test completion for both physician groups. CONCLUSION: Loop closure for three diagnostic interventions was low for patients in both faculty and resident primary care clinics, with lower loop closure rates in resident clinics. Failure to close diagnostic loops presents a safety challenge in primary care and is of particular concern for training programs.

Completion of Recommended Tests and Referrals in Telehealth vs In-Person Visits.

Importance: Use of telehealth has increased substantially in recent years. However, little is known about whether the likelihood of completing recommended tests and specialty referrals-termed diagnostic loop closure-is associated with visit modality. Objectives: To examine the prevalence of diagnostic loop closure for tests and referrals ordered at telehealth visits vs in-person visits and identify associated factors. Design, Setting, and Participants: In a retrospective cohort study, all patient visits from March 1, 2020, to December 31, 2021, at 1 large urban hospital-based primary care practice and 1 affiliated community health center in Boston, Massachusetts, were evaluated. Main Measures: Prevalence of diagnostic loop closure for (1) colonoscopy referrals (screening and diagnostic), (2) dermatology referrals for suspicious skin lesions, and (3) cardiac stress tests. Results: The study included test and referral orders for 4133 patients (mean [SD] age, 59.3 [11.7] years; 2163 [52.3%] women; 203 [4.9%] Asian, 1146 [27.7%] Black, 2362 [57.1%] White, and 422 [10.2%] unknown or other race). A total of 1151 of the 4133 orders (27.8%) were placed during a telehealth visit. Of the telehealth orders, 42.6% were completed within the designated time frame vs 58.4% of those ordered during in-person visits and 57.4% of those ordered without a visit. In an adjusted analysis, patients with telehealth visits were less likely to close the loop for all test types compared with those with in-person visits (odds ratio, 0.55; 95% CI, 0.47-0.64). Conclusions: The findings of this study suggest that rates of loop closure were low for all test types across all visit modalities but worse for telehealth. Failure to close diagnostic loops presents a patient safety challenge in primary care that may be of particular concern during telehealth encounters.

Urgent referrals from primary care to dermatology for lesions suspicious for skin cancer: patterns, outcomes, and need for systems improvement.

Ideally, urgent dermatology referrals for evaluation of a lesion concerning for skin cancer should be triaged and processed with appropriate urgency by primary care and dermatology, respectively. We performed a retrospective single-institution study by conducting chart reviews of all dermatology referrals designated by primary care as urgent for evaluation of a lesion concerning for skin cancer. We identified 320 referrals placed between January 1 and December 31, 2018. Dermatology encounters for these patients occurred on or before 30 days for 50.6% of referrals and on or after 31 days for 38.4% of referrals, with 10.9% never completed. The percentage of all races excluding whites, non-Hispanic in the delayed appointment group (≥ 31 days) was 15.1% higher (95% CI 5.3-24.9) than in the timely appointment group (≤ 30 days). Similarly, the percentage of non-English languages in the delayed group was 7.1% higher (95% CI 0.5-13.7) than in the timely group. Overall, 15.8% of these referrals yielded diagnoses of malignancy, while 76.8% and 7.4% resulted in benign and pre-malignant diagnoses, respectively. The primary care team documented referral status (i.e., completed, incomplete, or pending) during their subsequent visits with the patients in only 37.5% of these referrals. Our findings demonstrate the need to improve the reliability of urgent referrals to ensure they occur in a timely manner with confirmation of "referral loop" closure at the referring clinician's end.

Early recognition and response to increases in surgical site infections using optimised statistical process control charts-The early 2RIS trial: A multicentre stepped wedge cluster randomised controlled trial.

Background: Traditional approaches for surgical site infection (SSI) surveillance have deficiencies that delay detection of SSI outbreaks and other clinically important increases in SSI rates. We investigated whether use of optimised statistical process control (SPC) methods and feedback for SSI surveillance would decrease rates of SSI in a network of US community hospitals. Methods: We conducted a stepped wedge cluster randomised trial of patients who underwent any of 13 types of common surgical procedures across 29 community hospitals in the Southeastern United States. We divided the 13 procedures into six clusters; a cluster of procedures at a single hospital was the unit of randomisation and analysis. In total, 105 clusters were randomised to 12 groups of 8-10 clusters. All participating clusters began the trial in a 12-month baseline period of control or "traditional" SSI surveillance, including prospective analysis of SSI rates and consultative support for SSI outbreaks and investigations. Thereafter, a group of clusters transitioned from control to intervention surveillance every three months until all clusters received the intervention. Electronic randomisation by the study statistician determined the sequence by which clusters crossed over from control to intervention surveillance. The intervention was the addition of weekly application of optimised SPC methods and feedback to existing traditional SSI surveillance methods. Epidemiologists were blinded to hospital identity and randomisation status while adjudicating SPC signals of increased SSI rates, but blinding was not possible during SSI investigations. The primary outcome was the overall SSI prevalence rate (PR=SSIs/100 procedures), evaluated via generalised estimating equations with a Poisson regression model. Secondary outcomes compared traditional and optimised SPC signals that identified SSI rate increases, including the number of formal SSI investigations generated and deficiencies identified in best practices for SSI prevention. This trial was registered at ClinicalTrials.gov, NCT03075813. Findings: Between Mar 1, 2016, and Feb 29, 2020, 204,233 unique patients underwent 237,704 surgical procedures. 148,365 procedures received traditional SSI surveillance and feedback alone, and 89,339 procedures additionally received the intervention of optimised SPC surveillance. The primary outcome of SSI was assessed for all procedures performed within participating clusters. SSIs occurred after 1171 procedures assigned control surveillance (prevalence rate [PR] 0.79 per 100 procedures), compared to 781 procedures that received the intervention (PR 0·87 per 100 procedures; model-based PR ratio 1.10, 95% CI 0.94-1.30, p=0.25). Traditional surveillance generated 24 formal SSI investigations that identified 120 SSIs with deficiencies in two or more perioperative best practices for SSI prevention. In comparison, optimised SPC surveillance generated 74 formal investigations that identified 458 SSIs with multiple best practice deficiencies. Interpretation: The addition of optimised SPC methods and feedback to traditional methods for SSI surveillance led to greater detection of important SSI rate increases and best practice deficiencies but did not decrease SSI rates. Additional research is needed to determine how to best utilise SPC methods and feedback to improve adherence to SSI quality measures and prevent SSIs. Funding: Agency for Healthcare Research and Quality.

Lessons Learned in Implementing a Chronic Opioid Therapy Management System.

OBJECTIVES: Opioid misuse has resulted in significant morbidity and mortality in the United States, and safer opioid use represents an important challenge in the primary care setting. This article describes a research collaborative of health service researchers, systems engineers, and clinicians seeking to improve processes for safer chronic opioid therapy management in an academic primary care center. We present implementation results and lessons learned along with an intervention toolkit that others may consider using within their organization. METHODS: Using iterative improvement lifecycles and systems engineering principles, we developed a risk-based workflow model for patients on chronic opioids. Two key safe opioid use process metrics-percent of patients with recent opioid treatment agreements and urine drug tests-were identified, and processes to improve these measures were designed, tested, and implemented. Focus groups were conducted after the conclusion of implementation, with barriers and lessons learned identified via thematic analysis. RESULTS: Initial surveys revealed a lack of knowledge regarding resources available to patients and prescribers in the primary care clinic. In addition, 18 clinicians (69%) reported largely "inheriting" (rather than initiating) their chronic opioid therapy patients. We tracked 68 patients over a 4-year period. Although process measures improved, full adherence was not achieved for the entire population. Barriers included team structure, the evolving opioid environment, and surveillance challenges, along with disruptions resulting from the 2019 novel coronavirus. CONCLUSIONS: Safe primary care opioid prescribing requires ongoing monitoring and management in a complex environment. The application of a risk-based approach is possible but requires adaptability and redundancies to be reliable.

Evaluating a patient safety learning laboratory to create an interdisciplinary ecosystem for health care innovation.

BACKGROUND: In response to the complexity, challenges, and slow pace of innovation, health care organizations are adopting interdisciplinary team approaches. Systems engineering, which is oriented to creating new, scalable processes that perform with higher reliability and lower costs, holds promise for driving innovation in the face of challenges to team performance. A patient safety learning laboratory (lab) can be an essential aspect of fostering interdisciplinary team innovation across multiple projects and organizations by creating an ecosystem focused on deploying systems engineering methods to accomplish process redesign. PURPOSE: We sought to identify the role and activities of a learning ecosystem that support interdisciplinary team innovation through evaluation of a patient safety learning lab. METHODS: Our study included three participating learning lab project teams. We applied a mixed-methods approach using a convergent design that combined data from qualitative interviews of team members conducted as teams neared the completion of their redesign projects, as well as evaluation questionnaires administered throughout the 4-year learning lab. RESULTS: Our results build on learning theories by showing that successful learning ecosystems continually create alignment between interdisciplinary teams' activities, organizational context, and innovation project objectives. The study identified four types of alignment, interpersonal/interprofessional, informational, structural, and processual, and supporting activities for alignment to occur. CONCLUSION: Interdisciplinary learning ecosystems have the potential to foster health care improvement and innovation through alignment of team activities, project goals, and organizational contexts. PRACTICE IMPLICATIONS: This study applies to interdisciplinary teams tackling multilevel system challenges in their health care organization and suggests that the work of such teams benefits from the four types of alignment. Alignment on all four dimensions may yield best results.

Increasing efficiency and accuracy of magnetic interaction calculations in colloidal simulation through machine learning.

Calculating the magnetic interaction between magnetic particles that are positioned in close proximity to one another is a surprisingly challenging task. Exact solutions for this interaction exist either through numerical expansion of multipolar interactions or through solving Maxwell's equations with a finite element solver. These approaches can take hours for simple configurations of three particles. Meanwhile, across a range of scientific and engineering problems, machine learning approaches have been developed as fast computational platforms for solving complex systems of interest when large data sets are available. In this paper, we bring the touted benefits of recent advances in science-based machine learning algorithms to bear on the problem of modeling the magnetic interaction between three particles. We investigate this approach using diverse machine learning systems including physics informed neural networks. We find that once the training data has been collected and the model has been initiated, simulation times are reduced from hours to mere seconds while maintaining remarkable accuracy. Despite this promise, we also try to lay bare the current challenges of applying machine learning to these and more complex colloidal systems.

Systems engineering analysis of diagnostic referral closed-loop processes.

BACKGROUND: Closing loops to complete diagnostic referrals remains a significant patient safety problem in most health systems, with 65%-73% failure rates and significant delays common despite years of improvement efforts, suggesting new approaches may be useful. Systems engineering (SE) methods increasingly are advocated in healthcare for their value in studying and redesigning complex processes. OBJECTIVE: Conduct a formative SE analysis of process logic, variation, reliability and failures for completing diagnostic referrals originating in two primary care practices serving different demographics, using dermatology as an illustrating use case. METHODS: An interdisciplinary team of clinicians, systems engineers, quality improvement specialists, and patient representatives collaborated to understand processes of initiating and completing diagnostic referrals. Cross-functional process maps were developed through iterative group interviews with an urban community-based health centre and a teaching practice within a large academic medical centre. Results were used to conduct an engineering process analysis, assess variation within and between practices, and identify common failure modes and potential solutions. RESULTS: Processes to complete diagnostic referrals involve many sub-standard design constructs, with significant workflow variation between and within practices, statistical instability and special cause variation in completion rates and timeliness, and only 21% of all process activities estimated as value-add. Failure modes were similar between the two practices, with most process activities relying on low-reliability concepts (eg, reminders, workarounds, education and verification/inspection). Several opportunities were identified to incorporate higher reliability process constructs (eg, simplification, consolidation, standardisation, forcing functions, automation and opt-outs). CONCLUSION: From a systems science perspective, diagnostic referral processes perform poorly in part because their fundamental designs are fraught with low-reliability characteristics and mental models, including formalised workaround and rework activities, suggesting a need for different approaches versus incremental improvement of existing processes. SE perspectives and methods offer new ways of thinking about patient safety problems, failures and potential solutions.

Patient engagement in system redesign teams: a process of social identity.

PURPOSE: Studies demonstrate how patient roles in system redesign teams reflect a continuum of involvement and influence. This research shows the process by which patients move through this continuum and effectively engage within redesign projects. DESIGN/METHODOLOGY/APPROACH: The authors studied members of redesign teams, consisting of 5-10 members: clinicians, systems engineers, health system staff and patient(s), from three health systems working on separate projects in a patient safety learning lab. Weekly team meetings were observed, January 2016-April 2018, 17 semi-structured interviews were conducted and findings through a patient focus group were refined. Grounded theory was used to analyze field notes and transcripts. FINDINGS: Results show how the social identity process enables patients to move through stages in a patient engagement continuum (informant, partner and active change agent). Initially, patient and team member perceptions of the patient's role influence their respective behaviors (activating, directing, framing and sharing). Subsequently, patient and team member behaviors influence patient contributions on the team, which can redefine patient and team member perceptions of the patient's role. ORIGINALITY/VALUE: As health systems grow increasingly complex and become more interested in responding to patient expectations, understanding how to effectively engage patients on redesign teams gains importance. This research investigates how and why patient engagement on redesign teams changes over time and what makes different types of patient roles valuable for team objectives. Findings have implications for how redesign teams can better prepare, anticipate and support the changing role of engaged patients.

Systems Analysis of a Dedicated Ambulatory Respiratory Unit for Seeing and Ensuring Follow-up of Patients With COVID-19 Symptoms.

COVID-19 necessitated significant care redesign, including new ambulatory workflows to handle surge volumes, protect patients and staff, and ensure timely reliable care. Opportunities also exist to harvest lessons from workflow innovations to benefit routine care. We describe a dedicated COVID-19 ambulatory unit for closing testing and follow-up loops characterized by standardized workflows and electronic communication, documentation, and order placement. More than 85% of follow-ups were completed within 24 hours, with no observed staff, nor patient infections associated with unit operations. Identified issues include role confusion, staffing and gatekeeping bottlenecks, and patient reluctance to visit in person or discuss concerns with phone screeners.

Antibiotic stewardship from a decision-making, behavioral economics, and incentive design perspective.

Antibiotic-resistant infections cause over 20 thousand deaths and $20 billion annually in the United States. Antibiotic prescribing decision making can be described as a "tragedy of the commons" behavioral economics problem, for which individual best interests affecting human decision-making lead to suboptimal societal antibiotic overuse. In 2015, the U.S. federal government announced a $1.2 billion National Action Plan to combat resistance and reduce antibiotic use by 20% in inpatient settings and 50% in outpatient settings by 2020. We develop and apply a behavioral economics model based on game theory and "tragedy of the commons" concepts to help illustrate why rational individuals may not practice ideal stewardship and how to potentially structure three specific alternate approaches to accomplish these objectives (collective cooperative management, usage taxes, resistance penalties), based on Ostrom's economic governance principles. Importantly, while each approach can effectively incentivize ideal stewardship, the latter two do so with 10-30% lower utility to all providers. Encouraging local or state-level self-managed cooperative stewardship programs thus is preferred to national taxes and penalties, in contrast with current trends and with similar implications in other countries.

A mixed methods analysis of access barriers to dermatology care in a rural state.

AIMS: To identify significant patient and system access barriers and facilitators to dermatology care in one rural health system with limited dermatology appointment availability. DESIGN: Mixed methods study using data from electronic medical records, patient surveys, stakeholder semi-structured interviews, and service area dermatologist demographics. Retrospective data were collected between 1 January 2017-1 March 2018, and interviews and surveys were conducted between June 1-August 31, 2018. Participants were recruited from two primary care practices in one rural Maine regional health system. METHODS: Findings from thematic analyses, descriptive statistics, and statistical modelling were integrated using Chi-square tests for homogeneity to develop a unified understanding. Statistical modelling using odd-ratio logistic and linear regression were performed for each outcome variable of interest. RESULTS: Urgent referrals by primary care increased the likelihood of dermatology care overall (OR: 6.771; p = .007) and at nearby sites with limited availability (OR: 4.024; p = .024), but not at geographically further sites with higher capacities (p = .844). Referral under-diagnosis occurred in 20.8% of those biopsied. Older (p = .041) or non-working (p = .021) patients were more likely to remain unevaluated than seek more available but geographically further care. CONCLUSIONS: In rural areas with scarce appointment availability, primary care provider diagnostic accuracy may be an important barrier of dermatology care receipt and health outcomes, especially among at-risk populations. IMPACT: Although melanoma mortality rates are decreasing throughout the US, little is known about why rates in Maine continue to rise. This study applied a comprehensive approach to identify several patient and system access barriers to dermatology care in one underserved rural regional health system. While specific to this population and large service area, these findings will inform improvement efforts here and support broader future research efforts aimed at understanding and improving health outcomes in this rural state.

Early recognition and response to increases in surgical site infections using optimized statistical process control charts-the Early 2RIS Trial: a multicenter cluster randomized controlled trial with stepped wedge design.

BACKGROUND: Surgical site infections (SSIs) cause significant patient suffering. Surveillance and feedback of SSI rates is an evidence-based strategy to reduce SSIs, but traditional surveillance methods are slow and prone to bias. The objective of this cluster randomized controlled trial (RCT) is to determine if using optimized statistical process control (SPC) charts for SSI surveillance and feedback lead to a reduction in SSI rates compared to traditional surveillance. METHODS: The Early 2RIS Trial is a prospective, multicenter cluster RCT using a stepped wedge design. The trial will be performed in 29 hospitals in the Duke Infection Control Outreach Network (DICON) and 105 clusters over 4 years, from March 2016 through February 2020; year one represents a baseline period; thereafter, 8-9 clusters will be randomized to intervention every 3 months over a 3-year period using a stepped wedge randomization design. All patients who undergo one of 13 targeted procedures at study hospitals will be included in the analysis; these procedures will be included in one of six clusters: cardiac, orthopedic, gastrointestinal, OB-GYN, vascular, and spinal. All clusters will undergo traditional surveillance for SSIs; once randomized to intervention, clusters will also undergo surveillance and feedback using optimized SPC charts. Feedback on surveillance data will be provided to all clusters, regardless of allocation or type of surveillance. The primary endpoint is the difference in rates of SSI between the SPC intervention compared to traditional surveillance and feedback alone. DISCUSSION: The traditional approach for SSI surveillance and feedback has several major deficiencies because SSIs are rare events. First, traditional statistical methods require aggregation of measurements over time, which delays analysis until enough data accumulate. Second, traditional statistical tests and resulting p values are difficult to interpret. Third, analyses based on average SSI rates during predefined time periods have limited ability to rapidly identify important, real-time trends. Thus, standard analytic methods that compare average SSI rates between arbitrarily designated time intervals may not identify an important SSI rate increase on time unless the "signal" is very strong. Therefore, novel strategies for early identification and investigation of SSI rate increases are needed to decrease SSI rates. While SPC charts are used throughout industry and healthcare to improve and optimize processes, including other types of healthcare-associated infections, they have not been evaluated as a tool for SSI surveillance and feedback in a randomized trial. TRIAL REGISTRATION: ClinicalTrials.gov NCT03075813 , Registered March 9, 2017.

Potential Community and Campus Covid-19 Outcomes Under University and College Reopening Scenarios.

BACKGROUND: Significant uncertainty exists in many countries about the safety of, and best strategies for, reopening college and university campuses until the Covid-19 pandemic is better controlled. Little also is known about the effects on-campus students may have on local higher-risk communities. We aimed to estimate potential community and campus Covid-19 exposures, infections, and mortality due to various university reopening and precaution plans under current ranges of assumptions and uncertainties. METHODS: We developed and calibrated campus-only, community-only, and campus-x-community epidemic differential equation and agent-based models. Input parameters for campus and surrounding communities were estimated via published and grey literature, scenario development, expert opinion, accuracy optimization algorithms, and Monte Carlo simulation; models were cross-validated against each other using February-June 2020 data from heterogeneous U.S. counties and states. Campus opening plans (spanning various fully open, hybrid, and fully virtual approaches) were identified from websites and publications. All scenarios were simulated assuming 16-week semesters and estimated ranges for Covid-19 prevalence among community residents and arriving students, precaution compliance, contact frequency, virus attack rates, and tracing and isolation effectiveness. Additional student and community exposures, infections, and mortality were estimated under each scenario, with 10% trimmed medians, standard deviations, and probability intervals computed to omit extreme outlier scenarios. Factorial analyses were conducted to identify intervention inputs with largest and smallest effects. RESULTS: As a base case with no precautions (or no compliance), predicted 16-week student infections and mortality under normal operations ranged significantly from 471 to 9,495 (median: 2,286, SD: 2,627) and 0 to 123 (median: 9, SD: 14) per 10,000 students, respectively. The maximum active exposures across a semester was 15.76% of all students warranting tracing. Total additional community exposures, infections, and mortality ranged from 1 to 187, 13 to 820, and 1 to 21 per 10,000 residents, respectively. 1% and 5% of on-campus students were infected after a mean (SD) of 11 (3) and 76 (17) days, respectively; >10% students infected by the end of a semester in 34.8% of scenarios, with the greatest increase (first inflection point) occurring on aver-age on day 84 (SD: 10.2 days). Common reopening precautions reduced infections by 24% to 26% and mortality by 36% to 50% in both populations. Uncertainties in many factors, however, produced tremendous variability in all results, ranging from medians by -67% to +342%. CONCLUSIONS: Consequences on community and student Covid-19 exposures, infections, and mortality of reopening physical campuses are very highly unpredictable, depending on a combination of random chance, controllable (e.g. physical layouts), and uncontrollable (e.g. human behavior) factors. Implications include needs for criteria to adapt campus operations mid-semester, methods to detect when necessary, and contingency plans for doing so.

Assessing the cognitive and work load of an inpatient safety dashboard in the context of opioid management.

For health information technology to realize its potential to improve flow, care, and patient safety, applications should be intuitive to use and burden neutral for frontline clinicians. We assessed the impact of a patient safety dashboard on clinician cognitive and work load within a simulated information-seeking task for safe inpatient opioid medication management. Compared to use of an electronic health record for the same task, the dashboard was associated with significantly reduced time on task, mouse clicks, and mouse movement (each p < 0.001), with no significant increases in cognitive load nor task inaccuracy. Cognitive burden was higher for users with less experience, possibly partly attributable to usability issues identified during this study. Findings underscore the importance of assessing the usability, cognitive, and work load analysis during the design and implementation of health information technology applications.

Use, Perceived Usability, and Barriers to Implementation of a Patient Safety Dashboard Integrated within a Vendor EHR.

BACKGROUND: Preventable adverse events continue to be a threat to hospitalized patients. Clinical decision support in the form of dashboards may improve compliance with evidence-based safety practices. However, limited research describes providers' experiences with dashboards integrated into vendor electronic health record (EHR) systems. OBJECTIVE: This study was aimed to describe providers' use and perceived usability of the Patient Safety Dashboard and discuss barriers and facilitators to implementation. METHODS: The Patient Safety Dashboard was implemented in a cluster-randomized stepped wedge trial on 12 units in neurology, oncology, and general medicine services over an 18-month period. Use of the Dashboard was tracked during the implementation period and analyzed in-depth for two 1-week periods to gather a detailed representation of use. Providers' perceptions of tool usability were measured using the Health Information Technology Usability Evaluation Scale (rated 1-5). Research assistants conducted field observations throughout the duration of the study to describe use and provide insight into tool adoption. RESULTS: The Dashboard was used 70% of days the tool was available, with use varying by role, service, and time of day. On general medicine units, nurses logged in throughout the day, with many logins occurring during morning rounds, when not rounding with the care team. Prescribers logged in typically before and after morning rounds. On neurology units, physician assistants accounted for most logins, accessing the Dashboard during daily brief interdisciplinary rounding sessions. Use on oncology units was rare. Satisfaction with the tool was highest for perceived ease of use, with attendings giving the highest rating (4.23). The overall lowest rating was for quality of work life, with nurses rating the tool lowest (2.88). CONCLUSION: This mixed methods analysis provides insight into the use and usability of a dashboard tool integrated within a vendor EHR and can guide future improvements and more successful implementation of these types of tools.

Impact of molecular testing on reported Clostridoides difficile infection rates.

BACKGROUND: The reported incidence of Clostridoides difficile infection (CDI) has increased in recent years, partly due to broadening adoption of nucleic acid amplification tests (NAATs) replacing enzyme immunoassay (EIA) methods. Our aim was to quantify the impact of this switch on reported CDI rates using a large, multihospital, empirical dataset. METHODS: We analyzed 9 years of retrospective CDI data (2009-2017) from 47 hospitals in the southeastern United States; 37 hospitals switched to NAAT during this period, including 24 with sufficient pre- and post-switch data for statistical analyses. Poisson regression was used to quantify the NAAT-over-EIA incidence rate ratio (IRR) at hospital and network levels while controlling for longitudinal trends, the proportion of intensive care unit patient days, changes in surveillance methodology, and previously detected infection cluster periods. We additionally used change-point detection methods to identify shifts in the mean and/or slope of hospital-level CDI rates, and we compared results to recorded switch dates. RESULTS: For hospitals that transitioned to NAAT, average unadjusted CDI rates increased substantially after the test switch from 10.9 to 23.9 per 10,000 patient days. Individual hospital IRRs ranged from 0.75 to 5.47, with a network-wide IRR of 1.75 (95% confidence interval, 1.62-1.89). Reported CDI rates significantly changed 1.6 months on average after switching to NAAT testing (standard deviation, 1.9 months). CONCLUSION: Hospitals that switched from EIA to NAAT testing experienced an average postswitch increase of 75% in reported CDI rates after adjusting for other factors, and this increase was often gradual or delayed.

Large-scale empirical optimisation of statistical control charts to detect clinically relevant increases in surgical site infection rates.

OBJECTIVE: Surgical site infections (SSIs) are common costly hospital-acquired conditions. While statistical process control (SPC) use in healthcare has increased, limited rigorous empirical research compares and optimises these methods for SSI surveillance. We sought to determine which SPC chart types and design parameters maximise the detection of clinically relevant SSI rate increases while minimising false alarms. DESIGN: Systematic retrospective data analysis and empirical optimisation. METHODS: We analysed 12 years of data on 13 surgical procedures from a network of 58 community hospitals. Statistically significant SSI rate increases (signals) at individual hospitals initially were identified using 50 different SPC chart variations (Shewhart or exponentially weighted moving average, 5 baseline periods, 5 baseline types). Blinded epidemiologists evaluated the clinical significance of 2709 representative signals of potential outbreaks (out of 5536 generated), rating them as requiring 'action' or 'no action'. These ratings were used to identify which SPC approaches maximised sensitivity and specificity within a broader set of 3600 individual chart variations (additional baseline variations and chart types, including moving average (MA), and five control limit widths) and over 32 million dual-chart combinations based on different baseline periods, reference data (network-wide vs local hospital SSI rates), control limit widths and other calculation considerations. Results were validated with an additional year of data from the same hospital cohort. RESULTS: The optimal SPC approach to detect clinically important SSI rate increases used two simultaneous MA charts calculated using lagged rolling baseline windows and 1 SD limits. The first chart used 12-month MAs with 18-month baselines and best identified small sustained increases above network-wide SSI rates. The second chart used 6-month MAs with 3-month baselines and best detected large short-term increases above individual hospital SSI rates. This combination outperformed more commonly used charts, with high sensitivity (0.90; positive predictive value=0.56) and practical specificity (0.67; negative predictive value=0.94). CONCLUSIONS: An optimised combination of two MA charts had the best performance for identifying clinically relevant small but sustained above-network SSI rates and large short-term individual hospital increases.