Using the Operating Room Black Box to Assess Surgical Team Member Adaptation Under Uncertainty: An Observational Study.

OBJECTIVE: Identify how surgical team members uniquely contribute to teamwork and adapt their teamwork skills during instances of uncertainty. BACKGROUND: The importance of surgical teamwork in preventing patient harm is well documented. Yet, little is known about how key roles (nurse, anesthesiologist, surgeon, and medical trainee) uniquely contribute to teamwork during instances of uncertainty, particularly when adapting to and rectifying an intraoperative adverse event (IAE). METHODS: Audiovisual data of 23 laparoscopic cases from a large community teaching hospital were prospectively captured using OR Black Box. Human factors researchers retrospectively coded videos for teamwork skills (backup behavior, coordination, psychological safety, situation assessment, team decision-making, and leadership) by team role under 2 conditions of uncertainty: associated with an IAE versus no IAE. Surgeons identified IAEs. RESULTS: In all, 1015 instances of teamwork skills were observed. Nurses adapted to IAEs by expressing more backup behavior skills (5.3× increase; 13.9 instances/hour during an IAE vs 2.2 instances/hour when no IAE) while surgeons and medical trainees expressed more psychological safety skills (surgeons: 3.6× increase; 30.0 instances/hour vs 6.6 instances/hour and trainees: 6.6× increase; 31.2 instances/hour vs 4.1 instances/hour). All roles expressed fewer situation assessment skills during an IAE versus no IAE. CONCLUSIONS: OR Black Box enabled the assessment of critically important details about how team members uniquely contribute during instances of uncertainty. Some teamwork skills were amplified, while others dampened when dealing with IAEs. The knowledge of how each role contributes to teamwork and adapts to IAEs should be used to inform the design of tailored interventions to strengthen interprofessional teamwork.

Cognitive biases in surgery: systematic review.

BACKGROUND: Although numerous studies have established cognitive biases as contributors to surgical adverse events, their prevalence and impact in surgery are unknown. This review aimed to describe types of cognitive bias in surgery, their impact on surgical performance and patient outcomes, their source, and the mitigation strategies used to reduce their effect. METHODS: A literature search was conducted on 9 April and 6 December 2021 using MEDLINE, Embase, PsycINFO, Scopus, Web of Science, Cochrane Central Register of Controlled Trials, and the Cochrane Database of Systematic Reviews. Included studies investigated how cognitive biases affect surgery and the mitigation strategies used to combat their impact. The National Institutes of Health tools were used to assess study quality. Inductive thematic analysis was used to identify themes of cognitive bias impact on surgical performance. RESULTS: Thirty-nine studies were included, comprising 6514 surgeons and over 200 000 patients. Thirty-one types of cognitive bias were identified, with overconfidence, anchoring, and confirmation bias the most common. Cognitive biases differentially influenced six themes of surgical performance. For example, overconfidence bias associated with inaccurate perceptions of ability, whereas anchoring bias associated with inaccurate risk-benefit estimations and not considering alternative options. Anchoring and confirmation biases associated with actual patient harm, such as never events. No studies investigated cognitive bias source or mitigation strategies. CONCLUSION: Cognitive biases have a negative impact on surgical performance and patient outcomes across all points of surgical care. This review highlights the scarcity of research investigating the sources that give rise to cognitive biases in surgery and the mitigation strategies that target these factors.

Effects of Unit Census and Patient Acuity Levels on Discussions During Patient Rounds.

OBJECTIVES: PICU teams adapt the duration of patient rounding discussions to accommodate varying contextual factors, such as unit census and patient acuity. Although studies establish that shorter discussions can lead to the omission of critical patient information, little is known about how teams adapt their rounding discussions about essential patient topics (i.e., introduction/history, acute clinical status, care plans) in response to changing contexts. To fill this gap, we examined how census and patient acuity impact time spent discussing essential topics during individual patient encounters. DESIGN: Observational study. SETTING: PICU at a university-affiliated children's hospital, Toronto, ON, Canada. SUBJECTS: Interprofessional morning rounding teams. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We observed 165 individual patient encounters during morning rounds over 10 weeks. Regardless of census or patient acuity, the duration of patient introductions/history did not change. When census was high versus low, acute clinical status discussions significantly decreased for both low acuity patients (00 min:50 s high census; 01 min:39 s low census; -49.5% change) and high acuity patients (01 min:10 s high census; 02 min:02 s low census; -42.6% change). Durations of care plan discussions significantly reduced as a function of census (01 min:19 s high census; 02 min:52 s low census; -54.7% change) for low but not high acuity patients. CONCLUSIONS: Under high census and patient acuity levels, rounding teams disproportionately shorten time spent discussing essential patient topics. Of note, while teams preserved time to plan the care for acute patients, they cut care plan discussions of low acuity patients. This study provides needed detail regarding how rounding teams adapt their discussions of essential topics and establishes a foundation for consideration of varying contextual factors in the design of rounding guidelines. As ICUs are challenged with increasing census and patient acuity levels, it is critical that we turn our attention to these contextual aspects and understand how these adaptations impact clinical outcomes to address them.

Analyzing interprofessional teamwork in the operating room: An exploratory observational study using conventional and alternative approaches.

Intraoperative teamwork is vital for patient safety. Conventional tools for studying intraoperative teamwork typically rely on behaviorally anchored rating scales applied at the individual or team level, while others capture narrative information across several units of analysis. This prospective observational study characterizes teamwork using two conventional tools (Operating Theatre Team Non-Technical Skills Assessment Tool [NOTECHS]; Team Emergency Assessment Measure [TEAM]), and one alternative approach (modified-Systems Engineering Initiative for Patient Safety [SEIPS] model). We aimed to explore the advantages and disadvantages of each for providing feedback to improve teamwork practice. Fifty consecutive surgical cases at a Canadian academic hospital were recorded with the OR Black Box®, analyzed by trained raters, and summarized descriptively. Teamwork performance was consistently high within and across cases rated with NOTECHS and TEAMS. For cases analyzed with the modified-SEIPS tool, both optimal and suboptimal teamwork behaviors were identified, and team resilience was frequently observed. NOTECHS and TEAM provided summative assessments and overall pattern descriptions, while SEIPS facilitated a deeper understanding of teamwork processes. As healthcare organizations continue to prioritize teamwork improvement, SEIPS may provide valuable insights regarding teamwork behavior and the broader context influencing performance. This may ultimately enhance the development and effectiveness of multi-level teamwork interventions.

Reusable personal protective equipment in Canadian healthcare: Safe, secure, and sustainable.

Personal Protective Equipment (PPE) that was intentionally designed and manufactured as reusable, including gowns, goggles, face shields, and elastomeric respirators, took on a heightened role during the pandemic. Healthcare workers who had access to these products and infrastructure for cleaning and sterilizing them had a greater sense of confidence to undertake their jobs due to an increased sense of personal safety. Using multiple data sources, including a literature review, roundtables, interviews, surveys, and Internet-based research, the project team investigated the impact of disposable PPE and role of reusable PPE during the pandemic in Canada. This research supports the claim that adopting and supporting reusable PPE systems throughout the health sector can, if used appropriately on an ongoing basis, provide continuous access to reusable PPE while also contributing many co-benefits, including lower costs, domestic jobs, and improved environmental performance such as reduced waste and greenhouse gas emissions.

Human Factors Analysis of Latent Safety Threats in a Pediatric Critical Care Unit.

OBJECTIVES: To identify unique latent safety threats spanning routine pediatric critical care activities and categorize them according to their underlying work system factors (i.e., "environment, organization, person, task, tools/technology") and associated clinician behavior (i.e., "legal": expected compliance with or "illegal-normal": deviation from and "illegal-illegal": disregard for standard policies and protocols). DESIGN: A prospective observational study with contextual inquiry of clinical activities over a 5-month period. SETTING: Two PICUs (i.e., medical-surgical ICU and cardiac ICU) in an urban free-standing quaternary children's hospital. SUBJECTS: Attending physicians and trainees, nurse practitioners, registered nurses, respiratory therapists, dieticians, pharmacists, and patient services assistants were observed. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Conducted 188 hours of observations to prospectively identify unique latent safety threats. Qualitative observational notes were analyzed by human factors experts using a modified framework analysis methodology to summarize latent safety threats and categorize them based on associated clinical activity, predominant work system factor, and clinician behavior. Two hundred twenty-six unique latent safety threats were observed. The latent safety threats were categorized into 13 clinical activities and attributed to work system factors as follows: "organization" (n = 83; 37%), "task" (n = 52; 23%), "tools/technology" (n = 40; 18%), "person" (n = 32; 14%), and "environment" (n = 19; 8%). Twenty-three percent of latent safety threats were identified when staff complied with policies and protocols (i.e., "legal" behavior) and 77% when staff deviated from policies and protocols (i.e., "illegal-normal" behavior). There was no "illegal-illegal" behavior observed. CONCLUSIONS: Latent safety threats span various pediatric critical care activities and are attributable to many underlying work system factors. Latent safety threats are present both when staff comply with and deviate from policies and protocols, suggesting that simply reinforcing compliance with existing policies and protocols, the common default intervention imposed by healthcare organizations, will be insufficient to mitigate safety threats. Rather, interventions must be designed to address the underlying work system threats. This human factors informed framework analysis of observational data is a useful approach to identifying and understanding latent safety threats and can be used in other clinical work systems.

Developing the first pan-Canadian survey on patient engagement in patient safety.

BACKGROUND: Patient safety is a worldwide problem, and the patient contribution to mitigate the risk of patient harm is now recognized as a cornerstone to its solution. In order to understand the nature of integrating patients into patient safety and healthcare organizations and to monitor their integration, a Canadian survey tool has been co-constructed by patients, researchers and the Canadian Patient Safety Institute (CPSI). This questionnaire has been adapted from the French version of the patient engagement (PE) in patient safety (PS) questionnaire created for the province of Quebec, Canada. METHODOLOGY: The pan-Canadian PE in PS survey tool was developed in a five-step process: (1) a literature review and revision of the initial tool developed in the province of Quebec; (2) translation of the French questionnaire into an English version tool; (3) creation of a Canadian expert advisory group; (4) adaptation of the English version tool based on feedback from the expert advisory group (assessment and development of the construct's dimensions, wording assessment and adaptation for pan-Canadian use, technical testing of the online platform for the survey); and (5) pilot testing and pre-validation of the tool before pan-Canadian use. RESULTS AND CONCLUSION: Eight pan-Canadian PE in PS surveys were completed from five Canadian provinces by the expert advisory group and six surveys were completed during the pilot project by participants from different provinces in Canada. This survey tool comprises 5 sections: (1) demographic identification of the participants (Q1 to Q5); (2) general questions (Q6 to Q17); (3) the patient engagement process (experience level of participants and organizational incentives for PE in general) (Q18 to Q33); (4) PE in PS processes, such as current activities, strategies, structures, resources and factors (Q34 to Q67); and (5) the context and impact of PE in PS initiatives in Canadian healthcare organizations (CHOs) (Q68 to Q75), including outcome identification, improvement mechanisms and strategies, evaluation mechanisms, and indicators.

System Factors Affecting Intraoperative Risk and Resilience: Applying a Novel Integrated Approach to Study Surgical Performance and Patient Safety.

OBJECTIVE: To identify and categorize system factors in complex laparoscopic surgery that have the potential to either threaten patient safety or support system resilience. BACKGROUND: The operating room is a uniquely complex sociotechnical work system wherein surgical successes prevail despite pervasive safety threats. Holistically characterizing intraoperative factors that thus support system resilience in addition to those that threaten patient safety using contextual methodologies is critical for optimizing surgical safety overall. METHOD: In this prospective descriptive interdisciplinary study, 19 audio/video recordings of complex laparoscopic general surgical procedures were directly observed and transcribed. Using a qualitative systems-based approach, intraoperative human factors with the potential to impact patient safety, either as a safety threat or as a support for resilience, were identified. Adverse events were further assessed for shared threats and supports. Data collection was guided by the Systems Engineering Initiative for Patient Safety 2.0 work system model. RESULTS: A total of 1083 relevant observations were made over 39.8 hours of operative time, enabling the identification of 79 distinct safety threats and 67 resilience supports within the surgical system. Safety threats associated with the physical environment, tasks, organization, and equipment were prevalent and observed in equal measure, whereas supports for resilience were predominantly attributed to clinician behaviors, including proactive team management and skills coaching. Two subclinical adverse events were identified; shared safety threats included suboptimal technology design, whereas shared resilience supports included calm clinician behavior and redundant intraoperative resourcing. CONCLUSIONS: Safety threats and resilience supports were found to be systematic in the surgical setting. Identified safety threats should be prioritized for remediation, and clinician behaviors that contribute to fostering resilience should be valued and protected.

Untangling Infusion Confusion: A Comparative Evaluation of Interventions in a Simulated Intensive Care Setting.

OBJECTIVES: Assess interventions' impact on preventing IV infusion identification and disconnection mix-ups. DESIGN: Experimental study with repeated measures design. SETTING: High fidelity simulated adult ICU. SUBJECTS: Forty critical care nurses. INTERVENTIONS: Participants had to correctly identify infusions and disconnect an infusion in four different conditions: baseline (current practice); line labels/organizers; smart pump; and light-linking system. MEASUREMENTS AND MAIN RESULTS: Participants identified infusions with significantly fewer errors when using line labels/organizers (0; 0%) than in the baseline (12; 7.7%) and smart pump conditions (10; 6.4%) (p < 0.01). The light-linking system did not significantly affect identification errors (5; 3.2%) compared with the other conditions. Participants were significantly faster identifying infusions when using line labels/organizers (0:31) than in the baseline (1:20), smart pump (1:29), and light-linking (1:22) conditions (p < 0.001). When disconnecting an infusion, there was no significant difference in errors between conditions, but participants were significantly slower when using the smart pump than all other conditions (p < 0.001). CONCLUSIONS: The results suggest that line labels/organizers may increase infusion identification accuracy and efficiency.

How can systems engineering inform the methods of programme evaluation in health professions education?

CONTEXT: We evaluate programmes in health professions education (HPE) to determine their effectiveness and value. Programme evaluation has evolved from use of reductionist frameworks to those addressing the complex interactions between programme factors. Researchers in HPE have recently suggested a 'holistic programme evaluation' aiming to better describe and understand the implications of 'emergent processes and outcomes'. FRAMEWORK: We propose a programme evaluation framework informed by principles and tools from systems engineering. Systems engineers conceptualise complexity and emergent elements in unique ways that may complement and extend contemporary programme evaluations in HPE. We demonstrate how the abstract decomposition space (ADS), an engineering knowledge elicitation tool, provides the foundation for a systems engineering informed programme evaluation designed to capture both planned and emergent programme elements. METHODS: We translate the ADS tool to use education-oriented language, and describe how evaluators can use it to create a programme-specific ADS through iterative refinement. We provide a conceptualisation of emergent elements and an equation that evaluators can use to identify the emergent elements in their programme. Using our framework, evaluators can analyse programmes not as isolated units with planned processes and planned outcomes, but as unfolding, complex interactive systems that will exhibit emergent processes and emergent outcomes. Subsequent analysis of these emergent elements will inform the evaluator as they seek to optimise and improve the programme. CONCLUSION: Our proposed systems engineering informed programme evaluation framework provides principles and tools for analysing the implications of planned and emergent elements, as well as their potential interactions. We acknowledge that our framework is preliminary and will require application and constant refinement. We suggest that our framework will also advance our understanding of the construct of 'emergence' in HPE research.

An integrative review of nurses' prosocial behaviours contributing to work environment optimization, organizational performance and quality of care.

AIM: To rigorously review the literature on the prosocial workplace behaviours of nurses. BACKGROUND: Prosocial workplace behaviours, predominantly organisational citizenship behaviours have been theoretically and empirically found to promote individual and group level performance in various industries. However, little consensus exists in the literature regarding the impact of nurses' workplace behaviours on the work environment and organisational performance. EVALUATION: An integrative literature review was conducted on studies between 1980 and 2016. Nineteen articles were included related to nurses' prosocial behaviours and performance. RESULTS: A positive relationship was noted between workplace behaviours and individual level performance and unit level performance. Albeit multifactorial, leadership and the social structure of the work environment are important factors contributing to the workplace behaviour-performance relationship. CONCLUSIONS: Prosocial behaviours influence the social functioning of the work environment and offer insights into the delivery of quality care. IMPLICATIONS FOR NURSING MANAGEMENT: Nurse managers should recognize the influence of leadership style and characteristics in the work environment that encourage employee participation in prosocial behaviours. These additional voluntary efforts by nursing staff may improve organisational effectiveness and quality of care. Inclusion of these behaviours in performance reviews and as cultural norms may help to foster a more collaborative work environment.

Factors constraining patient engagement in implantable medical device discussions and decisions: interviews with physicians.

OBJECTIVE: Patient engagement (PE) is warranted when treatment risks and outcomes are uncertain, as is the case for higher risk medical devices. Previous research found that patients were not engaged in discussions or decisions about implantable medical devices. This study explored physician views about engaging patients in such discussions. DESIGN: Qualitative interviews using a basic descriptive approach. SETTING: Canada. PARTICIPANTS: Practicing cardiovascular and orthopaedic physicians. MAIN OUTCOME MEASURES: Level, processes and determinants of PE in medical device discussions and decisions. RESULTS: Views were largely similar among 10 cardiovascular and 12 orthopaedic physicians interviewed. Most said that it was feasible to inform and sometimes involve patients in discussions, but not to partner with them in medical device decision-making. PE was constrained by patient (comfort with PE, technical understanding, physiologic/demographic characteristics, prognosis), physician (device preferences, time), health system (purchasing contracts) and device factors (number of devices on market, comparative advantage). A framework was generated to help physicians engage patients in discussions about medical devices, even when decisions may not be preference sensitive due to multiple constraints on choice. CONCLUSIONS: This study identified that patients are not engaged in discussions or decisions about implantable medical devices. This may be due to multiple constraints. Further research should establish the legitimacy, prevalence and impact of constraining factors, and examine whether and how different levels and forms of PE are needed and feasible.

MEDICAL DEVICE RECALLS IN CANADA FROM 2005 TO 2015.

OBJECTIVES: Medical devices are ubiquitous in modern medical care. However, little is known about the epidemiology of medical devices in the healthcare marketplace, including the rate at which medical devices are subject to recalls or other advisories. We sought to study the epidemiology of medical devices in Canada, focusing on device recalls. In Canada, a recall may signify a variety of events, ranging from relatively minor field safety notifications, to removal of a product from the marketplace. METHODS: We used data from Health Canada to study medical device recalls in Canada from 2005 to 2015. We analyzed the risks of medical device recalls according to the risk class of the device (I lowest; IV highest) and the hazard priority of the recall (Type I highest potential harm; Type III lowest potential harm). RESULTS: During a 10-year period, there were 7,226 medical device recalls. Most recalls were for intermediate risk class (Class II, 40.1 percent; Class III, 38.7 percent) medical devices. Among recalled devices, 5.0 percent were judged to have a reasonable probability of serious adverse health consequences or death (Type I recall Hazard Priority classification). While the number of medical devices marketed in Canada is not known, over a similar 10-year period, 24,849 new Class II, II, and IV medical device licenses were issued by Health Canada. CONCLUSIONS: Several hundred medical device recalls occur in Canada each year. Further research is needed to characterize the nature of medical device recalls, and to explore how consumers use information about recalls.

"Usability of data integration and visualization software for multidisciplinary pediatric intensive care: a human factors approach to assessing technology".

BACKGROUND: Intensive care clinicians use several sources of data in order to inform decision-making. We set out to evaluate a new interactive data integration platform called T3™ made available for pediatric intensive care. Three primary functions are supported: tracking of physiologic signals, displaying trajectory, and triggering decisions, by highlighting data or estimating risk of patient instability. We designed a human factors study to identify interface usability issues, to measure ease of use, and to describe interface features that may enable or hinder clinical tasks. METHODS: Twenty-two participants, consisting of bedside intensive care physicians, nurses, and respiratory therapists, tested the T3™ interface in a simulation laboratory setting. Twenty tasks were performed with a true-to-setting, fully functional, prototype, populated with physiological and therapeutic intervention patient data. Primary data visualization was time series and secondary visualizations were: 1) shading out-of-target values, 2) mini-trends with exaggerated maxima and minima (sparklines), and 3) bar graph of a 16-parameter indicator. Task completion was video recorded and assessed using a use error rating scale. Usability issues were classified in the context of task and type of clinician. A severity rating scale was used to rate potential clinical impact of usability issues. RESULTS: Time series supported tracking a single parameter but partially supported determining patient trajectory using multiple parameters. Visual pattern overload was observed with multiple parameter data streams. Automated data processing using shading and sparklines was often ignored but the 16-parameter data reduction algorithm, displayed as a persistent bar graph, was visually intuitive. However, by selecting or automatically processing data, triggering aids distorted the raw data that clinicians use regularly. Consequently, clinicians could not rely on new data representations because they did not know how they were established or derived. CONCLUSIONS: Usability issues, observed through contextual use, provided directions for tangible design improvements of data integration software that may lessen use errors and promote safe use. Data-driven decision making can benefit from iterative interface redesign involving clinician-users in simulated environments. This study is a first step in understanding how software can support clinicians' decision making with integrated continuous monitoring data. Importantly, testing of similar platforms by all the different disciplines who may become clinician users is a fundamental step necessary to understand the impact on clinical outcomes of decision aids.

Multiple constraints compromise decision-making about implantable medical devices for individual patients: qualitative interviews with physicians.

BACKGROUND: Little research has examined how physicians choose medical devices for treating individual patients to reveal if interventions are needed to support decision-making and reduce device-associated morbidity and mortality. This study explored factors that influence choice of implantable device from among available options. METHODS: A descriptive qualitative approach was used. Physicians who implant orthopedic and cardiovascular devices were identified in publicly available directories and web sites. They were asked how they decided what device to use in a given patient, sources of information they consulted, and how patients were engaged in decision-making. Sampling was concurrent with data collection and analysis to achieve thematic saturation. Data were analyzed using constant comparative technique by all members of the research team. RESULTS: Twenty-two physicians from five Canadian provinces (10 cardiovascular, 12 orthopedic; 8, 10 and 4 early, mid and late career, respectively) were interviewed. Responses did not differ by specialty, geographic region or career stage. Five major categories of themes emerged that all influence decision-making about a range of devices, and often compromise choice of the most suitable device for a given patient, potentially leading to sub-optimal clinical outcomes: lack of evidence on device performance, patient factors, physician factors, organizational and health system factors, and device and device market factors. In the absence of evidence from research or device registries, tacit knowledge from trusted colleagues and less-trusted industry representatives informed device choice. Patients were rarely engaged in decision-making. Physician preference for particular devices was a barrier to acquiring competency in devices potentially more suitable for patients. Access to suitable devices was further limited to the number of comparable devices on the market, local inventory and purchasing contract specifications. CONCLUSIONS: This study revealed that decision-making about devices is complex, cognitively challenging and constrained by several factors limiting access to and use of devices that could optimize patient outcomes. Further research is needed to assess the impact of these constraints on clinical outcomes, and develop interventions that optimize decision-making about device choice for treating given patients.

Effects of Nested Interruptions on Task Resumption: A Laboratory Study With Intensive Care Nurses.

OBJECTIVE: Interruptions to secondary tasks resulting in multiple tasks to resume may tax working memory. The objective of this research is to study such interruptions experienced by intensive care unit (ICU) nurses. BACKGROUND: ICU nurses are frequently interrupted, resulting in a switch from primary to secondary tasks. In two recent studies, we observed that some of these secondary tasks also get interrupted, resulting in multiple tasks that have to be resumed, a phenomenon we refer to as nested interruptions. Although completing multiple secondary tasks in a serial fashion during an interruption period can create context-switching costs, we hypothesize that nested interruptions tax the working memory even more than just performing multiple secondary tasks sequentially because the nurse would have to encode in working memory the resumption goals for both the primary and the interrupted secondary tasks. METHOD: We conducted a laboratory study with 30 ICU nurses, who performed an electronic order-entry task under three interruption conditions: (a) baseline-no secondary task during the interruption period; (2) serial-performance of two tasks one after the other during the interruption period; and (3) nested-performance of two tasks during the interruption period, one of which was also interrupted. RESULTS: Nested interruptions resulted in significantly longer primary-task resumption lag and less accurate task resumption compared with both the serial interruption and baseline conditions. CONCLUSION: The nested nature of interruptions adds to the resumption lag and diminishes resumption accuracy by likely populating the working memory with goals associated with interrupted secondary tasks.

Grand rounds in methodology: key considerations for implementing machine learning solutions in quality improvement initiatives.

Machine learning (ML) solutions are increasingly entering healthcare. They are complex, sociotechnical systems that include data inputs, ML models, technical infrastructure and human interactions. They have promise for improving care across a wide range of clinical applications but if poorly implemented, they may disrupt clinical workflows, exacerbate inequities in care and harm patients. Many aspects of ML solutions are similar to other digital technologies, which have well-established approaches to implementation. However, ML applications present distinct implementation challenges, given that their predictions are often complex and difficult to understand, they can be influenced by biases in the data sets used to develop them, and their impacts on human behaviour are poorly understood. This manuscript summarises the current state of knowledge about implementing ML solutions in clinical care and offers practical guidance for implementation. We propose three overarching questions for potential users to consider when deploying ML solutions in clinical care: (1) Is a clinical or operational problem likely to be addressed by an ML solution? (2) How can an ML solution be evaluated to determine its readiness for deployment? (3) How can an ML solution be deployed and maintained optimally? The Quality Improvement community has an essential role to play in ensuring that ML solutions are translated into clinical practice safely, effectively, and ethically.

Quality Indicators for Transition from Paediatric to Adult Care for Youth with Chronic Conditions: Protocol for an Online Modified Delphi Study.

BACKGROUND: The transition from paediatric to adult care poses a significant health system-level challenge impeding the delivery of quality health services for youth with chronic health conditions. In Canada and globally, the transition to adult care is regarded as a top priority in adolescent health in need of readily applicable, adaptable, and relevant national metrics to evaluate and benchmark transition success across disease populations and clinical care settings. Unfortunately, existing literature fails to account for the lack of engagement from youth and caregivers in developing indicators, and its applicability across chronic conditions, primary care involvement, and health equity considerations. OBJECTIVE: Our proposed study aims to establish a consensus-driven set of quality indicators for the transition to adult care that are universally applicable across physical, developmental, and/or mental health conditions, clinical care settings, and health jurisdictions. METHODS: Using an integrated knowledge translation (iKT) approach, a panel comprised of youth, caregivers, interdisciplinary healthcare providers, and health system leaders will be established to collaborate with our research team to ensure the study methodology, materials, and knowledge dissemination are suitable and reflect the perspectives of youth and their families. We will then conduct an iterative 3-round Online Modified Delphi (OMD) study (n= 160) to (1) compare and contrast the perspectives of youth, caregivers, health care providers, and health system leaders on quality indicators for transition, and (2) prioritize a key set of quality indicators for transition applicable across disease populations that are the most important, useful, and feasible in the Canadian context. Using the RAND/UCLA Appropriateness Method (RAM) multi-stage analytic approach, data from each panel and stakeholder group will be examined separately and compared to establish a key set of indicators endorsed by both panels. RESULTS: The study is funded by the Canadian Institutes of Health Research and Physicians Services Incorporated. CONCLUSIONS: This study will produce quality indicators to evaluate and inform action equitably to improve transition from paediatric to adult care for youth and their families in Canada.

Patient, caregiver and other knowledge user engagement in consensus-building healthcare initiatives: a scoping review protocol.

INTRODUCTION: Patient engagement and integrated knowledge translation (iKT) processes improve health outcomes and care experiences through meaningful partnerships in consensus-building initiatives and research. Consensus-building is essential for engaging a diverse group of experienced knowledge users in co-developing and supporting a solution where none readily exists or is less optimal. Patients and caregivers provide invaluable insights for building consensus in decision-making around healthcare, policy and research. However, despite emerging evidence, patient engagement remains sparse within consensus-building initiatives. Specifically, our research has identified a lack of opportunity for youth living with chronic health conditions and their caregivers to participate in developing consensus on indicators/benchmarks for transition into adult care. To bridge this gap and inform our consensus-building approach with youth/caregivers, this scoping review will synthesise the extent of the literature on patient and other knowledge user engagement in consensus-building healthcare initiatives. METHODS AND ANALYSIS: Following the scoping review methodology from Joanna Briggs Institute, published literature will be searched in MEDLINE, EMBASE, CINAHL and PsycINFO databases from inception to July 2023. Grey literature will be hand-searched. Two independent reviewers will determine the eligibility of articles in a two-stage process, with disagreements resolved by a third reviewer. Included studies must be consensus-building studies within the healthcare context that involve patient engagement strategies. Data from eligible studies will be extracted and charted on a standardised form. Abstracted data will be analysed quantitatively and descriptively, according to specific consensus methodologies, and patient engagement models and/or strategies. ETHICS AND DISSEMINATION: Ethics approval is not required for this scoping review protocol. The review process and findings will be shared with and informed by relevant knowledge users. Dissemination of findings will also include peer-reviewed publications and conference presentations. The results will offer new insights for supporting patient engagement in consensus-building healthcare initiatives. PROTOCOL REGISTRATION: https://osf.io/beqjr.

Applying an equity lens to hospital safety monitoring: a critical interpretive synthesis protocol.

INTRODUCTION: Hospital safety monitoring systems are foundational to how adverse events are identified and addressed. They are well positioned to bring equity-related safety issues to the forefront for action. However, there is uncertainty about how they have been, and can be, used to achieve this goal. We will undertake a critical interpretive synthesis (CIS) to examine how equity is integrated into hospital safety monitoring systems. METHODS AND ANALYSIS: This review will follow CIS principles. Our initial compass question is: How is equity integrated into safety monitoring systems? We will begin with a structured search strategy of hospital safety monitoring systems in CINAHL, EMBASE, MEDLINE and PsycINFO for up to May 2023 to identify papers on safety monitoring systems generally and those linked to equity (eg, racism, social determinants of health). We will also review reference lists of selected papers, contact experts and draw on team expertise. For subsequent literature searching stages, we will use team expertise and expert contacts to purposively search the social science, humanities and health services research literature to support the development of a theoretical understanding of our topic. Following data extraction, we will use interpretive processes to develop themes and a critique of the literature. The above processes of question formulation, article search and selection, data extraction, and critique and synthesis will be iterative and interactive with the goal to develop a theoretical understanding of equity in hospital monitoring systems that will have practice-based implications. ETHICS AND DISSEMINATION: This review does not require ethical approval because we are reviewing published literature. We aim to publish findings in a peer-reviewed journal and present at conferences.