A mixed-methods study of challenges experienced by clinical teams in measuring improvement.

OBJECTIVE: Measurement is an indispensable element of most quality improvement (QI) projects, but it is undertaken to variable standards. We aimed to characterise challenges faced by clinical teams in undertaking measurement in the context of a safety QI programme that encouraged local selection of measures. METHODS: Drawing on an independent evaluation of a multisite improvement programme (Safer Clinical Systems), we combined a qualitative study of participating teams' experiences and perceptions of measurement with expert review of measurement plans and analysis of data collected for the programme. Multidisciplinary teams of frontline clinicians at nine UK NHS sites took part across the two phases of the programme between 2011 and 2016. RESULTS: Developing and implementing a measurement plan against which to assess their improvement goals was an arduous task for participating sites. The operational definitions of the measures that they selected were often imprecise or missed important details. Some measures used by the teams were not logically linked to the improvement actions they implemented. Regardless of the specific type of data used (routinely collected or selected ex novo), the burdensome nature of data collection was underestimated. Problems also emerged in identifying and using suitable analytical approaches. CONCLUSION: Measurement is a highly technical task requiring a degree of expertise. Simply leveraging individual clinicians' motivation is unlikely to defeat the persistent difficulties experienced by clinical teams when attempting to measure their improvement efforts. We suggest that more structural initiatives and broader capability-building programmes should be pursued by the professional community. Improving access to, and ability to use repositories of validated measures, and increasing transparency in reporting measurement attempts, is likely to be helpful.

Effectiveness of a Hospital-Based Computerized Decision Support System on Clinician Recommendations and Patient Outcomes: A Randomized Clinical Trial.

Importance: Sophisticated evidence-based information resources can filter medical evidence from the literature, integrate it into electronic health records, and generate recommendations tailored to individual patients. Objective: To assess the effectiveness of a computerized clinical decision support system (CDSS) that preappraises evidence and provides health professionals with actionable, patient-specific recommendations at the point of care. Design, Setting, and Participants: Open-label, parallel-group, randomized clinical trial among internal medicine wards of a large Italian general hospital. All analyses in this randomized clinical trial followed the intent-to-treat principle. Between November 1, 2015, and December 31, 2016, patients were randomly assigned to the intervention group, in which CDSS-generated reminders were displayed to physicians, or to the control group, in which reminders were generated but not shown. Data were analyzed between February 1 and July 31, 2018. Interventions: Evidence-Based Medicine Electronic Decision Support (EBMEDS), a commercial CDSS covering a wide array of health conditions across specialties, was integrated into the hospital electronic health records to generate patient-specific recommendations. Main Outcomes and Measures: The primary outcome was the resolution rate, the rate at which medical problems identified and alerted by the CDSS were addressed by a change in practice. Secondary outcomes included the length of hospital stay and in-hospital all-cause mortality. Results: In this randomized clinical trial, 20 563 patients were admitted to the hospital. Of these, 6480 (31.5%) were admitted to the internal medicine wards (study population) and randomized (3242 to CDSS and 3238 to control). The mean (SD) age of patients was 70.5 (17.3) years, and 54.5% were men. In total, 28 394 reminders were generated throughout the course of the trial (median, 3 reminders per patient per hospital stay; interquartile range [IQR], 1-6). These messages led to a change in practice in approximately 4 of 100 patients. The resolution rate was 38.0% (95% CI, 37.2%-38.8%) in the intervention group and 33.7% (95% CI, 32.9%-34.4%) in the control group, corresponding to an odds ratio of 1.21 (95% CI, 1.11-1.32; P < .001). The length of hospital stay did not differ between the groups, with a median time of 8 days (IQR, 5-13 days) for the intervention group and a median time of 8 days (IQR, 5-14 days) for the control group (P = .36). In-hospital all-cause mortality also did not differ between groups (odds ratio, 0.95; 95% CI, 0.77-1.17; P = .59). Alert fatigue did not differ between early and late study periods. Conclusions and Relevance: An international commercial CDSS intervention marginally influenced routine practice in a general hospital, although the change did not statistically significantly affect patient outcomes. Trial Registration: ClinicalTrials.gov identifier: NCT02577198.

How to be a very safe maternity unit: An ethnographic study.

Maternity care continues to be associated with avoidable harm that can result in serious disability and profound anguish for women, their children, and their families, and in high costs for healthcare systems. As in other areas of healthcare, improvement efforts have typically focused either on implementing and evaluating specific interventions, or on identifying the contextual features that may be generative of safety (e.g. structures, processes, behaviour, practices, and values), but the dialogue between these two approaches has remained limited. In this article, we report a positive deviance case study of a high-performing UK maternity unit to examine how it achieved and sustained excellent safety outcomes. Based on 143 h of ethnographic observations in the maternity unit, 12 semi-structured interviews, and two focus groups with staff, we identified six mechanisms that appeared to be important for safety: collective competence; insistence on technical proficiency; monitoring, coordination, and distributed cognition; clearly articulated and constantly reinforced standards of practice, behaviour, and ethics; monitoring multiple sources of intelligence about the unit's state of safety; and a highly intentional approach to safety and improvement. These mechanisms were nurtured and sustained through both a specific intervention (known as the PROMPT programme) and, importantly, the unit's contextual features: intervention and context shaped each other in both direct and indirect ways. The mechanisms were also influenced by the unit's structural conditions, such as staffing levels and physical environment. This study enhances understanding of what makes a maternity unit safe, paving the way for better design of improvement approaches. It also advances the debate on quality and safety improvement by offering a theoretically and empirically grounded analysis of the interplay between interventions and context of implementation.

Learning from high risk industries may not be straightforward: a qualitative study of the hierarchy of risk controls approach in healthcare.

OBJECTIVE: Though healthcare is often exhorted to learn from 'high-reliability' industries, adopting tools and techniques from those sectors may not be straightforward. We sought to examine the hierarchies of risk controls approach, used in high-risk industries to rank interventions according to supposed effectiveness in reducing risk, and widely advocated as appropriate for healthcare. DESIGN: Classification of risk controls proposed by clinical teams following proactive detection of hazards in their clinical systems. Classification was based on a widely used hierarchy of controls developed by the US National Institute for Occupational Safety and Health (NIOSH). SETTING AND PARTICIPANTS: A range of clinical settings in four English NHS hospitals. RESULTS: The four clinical teams in our study planned a total of 42 risk controls aimed at addressing safety hazards. Most (n = 35) could be classed as administrative controls, thus qualifying among the weakest type of interventions according to the HoC approach. Six risk controls qualified as 'engineering' controls, i.e. the intermediate level of the hierarchy. Only risk control qualified as 'substitution', classified as the strongest type of intervention by the HoC. CONCLUSIONS: Many risk controls introduced by clinical teams may cluster towards the apparently weaker end of an established hierarchy of controls. Less clear is whether the HoC approach as currently formulated is useful for the specifics of healthcare. Valuable opportunities for safety improvement may be lost if inappropriate hierarchical models are used to guide the selection of patient safety improvement interventions. Though learning from other industries may be useful, caution is needed.

What hinders the uptake of computerized decision support systems in hospitals? A qualitative study and framework for implementation.

BACKGROUND: Advanced Computerized Decision Support Systems (CDSSs) assist clinicians in their decision-making process, generating recommendations based on up-to-date scientific evidence. Although this technology has the potential to improve the quality of patient care, its mere provision does not guarantee uptake: even where CDSSs are available, clinicians often fail to adopt their recommendations. This study examines the barriers and facilitators to the uptake of an evidence-based CDSS as perceived by diverse health professionals in hospitals at different stages of CDSS adoption. METHODS: Qualitative study conducted as part of a series of randomized controlled trials of CDSSs. The sample includes two hospitals using a CDSS and two hospitals that aim to adopt a CDSS in the future. We interviewed physicians, nurses, information technology staff, and members of the boards of directors (n = 30). We used a constant comparative approach to develop a framework for guiding implementation. RESULTS: We identified six clusters of experiences of, and attitudes towards CDSSs, which we label as "positions." The six positions represent a gradient of acquisition of control over CDSSs (from low to high) and are characterized by different types of barriers to CDSS uptake. The most severe barriers (prevalent in the first positions) include clinicians' perception that the CDSSs may reduce their professional autonomy or may be used against them in the event of medical-legal controversies. Moving towards the last positions, these barriers are substituted by technical and usability problems related to the technology interface. When all barriers are overcome, CDSSs are perceived as a working tool at the service of its users, integrating clinicians' reasoning and fostering organizational learning. CONCLUSIONS: Barriers and facilitators to the use of CDSSs are dynamic and may exist prior to their introduction in clinical contexts; providing a static list of obstacles and facilitators, irrespective of the specific implementation phase and context, may not be sufficient or useful to facilitate uptake. Factors such as clinicians' attitudes towards scientific evidences and guidelines, the quality of inter-disciplinary relationships, and an organizational ethos of transparency and accountability need to be considered when exploring the readiness of a hospital to adopt CDSSs.