Human growth hormone for poor responders: a randomized placebo-controlled trial provides no evidence for improved live birth rate.

RESEARCH QUESTION: Does the addition of human growth hormone (HGH) to an IVF cycle improve the live birth rate in previously documented poor responders to FSH? DESIGN: Double-blind, placebo-controlled, randomized clinical trial comparing HGH to placebo in maximal stimulation in an IVF cycle. The study was stopped after 4 years. Women receiving ovarian stimulation in one IVF cycle, having failed to produce more than 5 eggs in a previous cycle with more than 250 IU/day of FSH were included. Basal FSH was ≤15 IU/l, body mass index <33 kg/m2, age <41 years. HGH or placebo were added from the start of the cycle in a double-blinded manner. The primary outcome was live birth rate. MAIN RESULTS: The live birth rates following an IVF cycle were 9/62 (14.5%) for growth hormone and 7/51 (13.7%) for the placebo group (risk difference 0.8%, 95% confidence interval [CI] -12.1 to 13.7%; odds ratio [OR] 1.07, 95% CI 0.37-3.10). There was a greater odds of oocyte retrieval with growth hormone (OR 5.67, 95% CI 1.54-20.80) but no better chance of embryo transfer (OR 1.42, 95% CI 0.50-4.00). Birth weights were comparable. CONCLUSIONS: Planned participant numbers were not reached. It was not possible to demonstrate an increase in live birth rate from the addition of growth hormone in women with a previous poor ovarian response to IVF.

Implementation Science: A Neglected Opportunity to Accelerate Improvements in the Safety and Quality of Surgical Care.

OBJECTIVE: The aim of this review was to emphasize the importance of implementation science in understanding why efforts to integrate evidence-based interventions into surgical practice frequently fail to replicate the improvements reported in early research studies. SUMMARY OF BACKGROUND DATA: Over the past 2 decades, numerous patient safety initiatives have been developed to improve the quality and safety of surgical care. The surgical community is now faced with translating "promising" initiatives from the research environment into clinical practice-the World Health Organization (WHO) has described this task as one of the greatest challenges facing the global health community and has identified the importance of implementation science in scaling up evidence-based interventions. METHODS: Using the WHO surgical safety checklist, a prominent example of a rapidly and widely implemented surgical safety intervention of the past decade, a review of literature, spanning surgery, and implementation science, was conducted to identify and describe a broad range of factors affecting implementation success, including contextual factors, implementation strategies, and implementation outcomes. RESULTS: Our current approach to conceptualizing and measuring the "effectiveness" of interventions has resulted in factors critical to implementing surgical safety interventions successfully being neglected. CONCLUSION: Improvements in the safety and quality of surgical care can be accelerated by drawing more heavily upon implementation science and that until this rapidly evolving field becomes more firmly embedded into surgical research and implementation efforts, our understanding of why interventions such as the checklist "work" in some settings and appear "not to work" in other settings will be limited.

Frailty identification in the emergency department-a systematic review focussing on feasibility.

Introduction: risk-stratifying older people accessing urgent care is a potentially useful first step to ensuring that the most vulnerable are able to access optimal care from the start of the episode. While there are many risk-stratification tools reported in the literature, few have addressed the practical issues of implementation. This review sought evidence about the feasibility of risk stratification for older people with urgent care needs. Methods: medline was searched for papers addressing risk stratification and implementation (feasibility or evaluation or clinician acceptability). All search stages were conducted by two reviewers, and selected papers were graded for quality using the CASP tool for cohort studies. Data were summarised using descriptive statistics only. Results: about 1872 titles of potential interest were identified, of which 1827 were excluded on title/abstract review, and a further 43 after full-text review, leaving four papers for analysis. These papers described nine tools, which took between 1 and 10 minutes to complete for most participants. No more than 52% of potentially eligible older people were actually screened using any of the tools. Little detail was reported on the clinical acceptability of the tools tested. Discussion: the existing literature indicates that commonly used risk-stratification tools are relatively quick to use, but do not cover much more than 50% of the potential population eligible for screening in practice. Additional work is required to appreciate how tools are likely to be used, by whom, and when in order to ensure that they are acceptable to urgent care teams.

Improving Escalation of Care: Development and Validation of the Quality of Information Transfer Tool.

OBJECTIVE: To develop and provide validity and feasibility evidence for the QUality of Information Transfer (QUIT) tool. BACKGROUND: Prompt escalation of care in the setting of patient deterioration can prevent further harm. Escalation and information transfer skills are not currently measured in surgery. METHODS: This study comprised 3 phases: the development (phase 1), validation (phase 2), and feasibility analysis (phase 3) of the QUIT tool. Phase 1 involved identification of core skills needed for successful escalation of care through literature review and 33 semistructured interviews with stakeholders. Phase 2 involved the generation of validity evidence for the tool using a simulated setting. Thirty surgeons assessed a deteriorating postoperative patient in a simulated ward and escalated their care to a senior colleague. The face and content validity were assessed using a survey. Construct and concurrent validity of the tool were determined by comparing performance scores using the QUIT tool with those measured using the Situation-Background-Assessment-Recommendation (SBAR) tool. Phase 3 was conducted using direct observation of escalation scenarios on surgical wards in 2 hospitals. RESULTS: A 7-category assessment tool was developed from phase 1 consisting of 24 items. Twenty-one of 24 items had excellent content validity (content validity index >0.8). All 7 categories and 18 of 24 (P < 0.05) items demonstrated construct validity. The correlation between the QUIT and SBAR tools used was strong indicating concurrent validity (r = 0.694, P < 0.001). Real-time scoring of escalation referrals was feasible and indicated that doctors currently have better information transfer skills than nurses when faced with a deteriorating patient. CONCLUSIONS: A validated tool to assess information transfer for deteriorating surgical patients was developed and tested using simulation and real-time clinical scenarios. It may improve the quality and safety of patient care on the surgical ward.

Mentoring perception, scientific collaboration and research performance: is there a 'gender gap' in academic medicine? An Academic Health Science Centre perspective.

OBJECTIVES: The 'gender gap' in academic medicine remains significant and predominantly favours males. This study investigates gender disparities in research performance in an Academic Health Science Centre, while considering factors such as mentoring and scientific collaboration. MATERIALS AND METHODS: Professorial registry-based electronic survey (n=215) using bibliometric data, a mentoring perception survey and social network analysis. Survey outcomes were aggregated with measures of research performance (publications, citations and h-index) and measures of scientific collaboration (authorship position, centrality and social capital). Univariate and multivariate regression models were constructed to evaluate inter-relationships and identify gender differences. RESULTS: One hundred and four professors responded (48% response rate). Males had a significantly higher number of previous publications than females (mean 131.07 (111.13) vs 79.60 (66.52), p=0.049). The distribution of mentoring survey scores between males and females was similar for the quality and frequency of shared core, mentor-specific and mentee-specific skills. In multivariate analysis including gender as a variable, the quality of managing the relationship, frequency of providing corrective feedback and frequency of building trust had a statistically significant positive influence on number of publications (all p<0.05). CONCLUSIONS: This is the first study in healthcare research to investigate the relationship between mentoring perception, scientific collaboration and research performance in the context of gender. It presents a series of initiatives that proved effective in marginalising the gender gap. These include the Athena Scientific Women's Academic Network charter, new recruitment and advertisement strategies, setting up a 'Research and Family Life' forum, establishing mentoring circles for women and projecting female role models.

Crisis management on surgical wards: a simulation-based approach to enhancing technical, teamwork, and patient interaction skills.

OBJECTIVE: To establish the efficacy of simulation-based training for improving residents' management of postoperative complications on a surgical ward. BACKGROUND: Effective postoperative care is a crucial determinant of patient outcome, yet trainees learn this through the Halstedian approach. Little evidence exists on the efficacy of simulation in this safety-critical environment. METHODS: A pre-/postintervention design was employed with 185 residents from 5 hospitals. Residents participated in 2 simulated ward-based scenarios consisting of a deteriorating postoperative patient. A debriefing intervention was implemented between scenarios. Resident performance was evaluated by calibrated, blinded assessors using the validated Global Assessment Toolkit for Ward Care. This included an assessment of clinical skills (checklist of 35 tasks), team-working skills (score range 1-6 per skill), and physician-patient interaction skills. RESULTS: Excellent interrater reliability was achieved in all assessments (reliability 0.89-0.99, P < 0.001). Clinically, improvements were obtained posttraining in residents' ability to recognize/respond to falling saturations (pre = 73.7% vs post = 94.8%, P < 0.01), check circulatory status (pre = 21.1% vs post = 84.2% P < 0.001), continuously reassess patient (pre = 42.1% vs post = 100%, P < 0.001), and call for help (pre = 36.8% vs post = 89.8%, P < 0.001). Regarding teamwork, there was a significant improvement in residents' communication (pre = 1.75 vs post = 3.43), leadership (pre = 2.43 vs post = 4.20), and decision-making skills (pre = 2.20 vs post = 3.81, P < 0.001). Finally, residents improved in all elements of interaction with patients: empathy, organization, and verbal and nonverbal expression (Ps < 0.001). CONCLUSIONS: The study provides evidence for the efficacy of ward-based team training using simulation. Such exercises should be formally incorporated into training curricula to enhance patient safety in the high-risk surgical ward environment.

The Impact of Operating Room Distractions on Stress, Workload, and Teamwork.

OBJECTIVE: To investigate whether distractions in the operating room (OR) are associated with higher mental workload and stress, and poorer teamwork among OR personnel. BACKGROUND: Engaging in multiple tasks can affect performance. There is little research on the effect of distractions on surgical team members' behavior and cognitive processes. METHODS: Ninety general surgery cases were observed in real time. Cases were assessed by a surgeon and a behavioral scientist using 4 validated tools: OR Distractions Assessment Form, the Observational Teamwork Assessment for Surgery tool, NASA-Task Load Index, and short form of the State Trait Anxiety Inventory. Analysis of variance was performed to evaluate significant differences between teamwork, workload, and stress level among team members. Correlations (Pearson r) were computed to evaluate associations between variables. RESULTS: The most prevalent distractions were those initiated by external staff, followed by case-irrelevant conversations. Case-irrelevant conversations were associated with poorer team performance. Irrelevant conversations initiated by surgeons were associated with lower teamwork in surgeons (across team skills: r = -0.44 to -0.58, P < 0.05 to 0.01) and anesthesiologists (r = -0.38 and r = -0.40, for coordination and leadership; P < 0.05). Equipment-related distractions correlated with higher stress (r = 0.48, P < 0.05) and lower teamwork (across team skills: r = -0.42 to -0.50, P < 0.05) in nurses. Acoustic distractions correlated with higher stress in surgeons (r = 0.32, P < 0.05) and higher workload in anesthesiologists (r = 0.30, P < 0.05). CONCLUSIONS: Although some distractions may be inevitable in the OR, they can also be detrimental to the team. A deeper understanding of the effect of distractions on teams and their outcomes can lead to targeted quality improvement.

Improving surgical ward care: development and psychometric properties of a global assessment toolkit.

OBJECTIVE: To develop a toolkit that covers the clinical, nontechnical, and empathic skills required for effective, safe surgical ward care. BACKGROUND: Despite the explosion of interest in patient safety, little attention has been placed on the skill set required for safe and effective surgical ward care. Currently, there is a lack of a systematic approach to improving ward care via assessing and improving residents' ward care skills. METHODS: A comprehensive evidence-based and expert-derived toolkit was developed, including a novel clinical checklist for ward care (Clinical Skills Assessment for Ward Care: C-SAW-C); a novel team assessment scale for wards rounds (Teamwork Skills Assessment for Ward Care: T-SAW-C); and a revised version of a physician-patient interaction scale (Physician-Patient Interaction Global Rating Scale: PP-GIS). Interassessor reliability (κ, intraclass correlation), internal consistency (Cronbach α), and convergent validity (Pearson r correlations) were evaluated statistically in 38 simulated scenarios (during which a patient rapidly deteriorated) involving 185 residents. RESULTS: Excellent interassessor reliability was obtained for C-SAW-C [median κ = 0.89; median intraclass correlation coefficient (ICC) = 0.94], T-SAW-C (median ICC = 0.99), and the revised PP-GIs (κ = 1.00; ICC = 0.98 or higher). Internal consistency was also very high for all team skills assessed by T-SAW-C (Cronbach α range 0.87-0.94 across 6 skills) and the revised PP-GIS (Cronbach α = 0.96)-all P's < 0.001. Significant positive correlations were obtained between the 3 assessments (r = 0.73-0.92, P < 0.001) thus showing evidence for convergent validity. CONCLUSIONS: We developed a toolkit that captures comprehensively the skills that are required for safe and effective ward care, including the high-risk situation where a patient decompensates. The toolkit offers a systematic evaluation of the quality and safety of surgical ward care and can be used to train and debrief residents' skills and performance.

Re-validating the Observational Teamwork Assessment for Surgery tool (OTAS-D): cultural adaptation, refinement, and psychometric evaluation.

BACKGROUND: The nontechnical and team skills of surgical teams are critical for safety and efficiency in the operating room. Assessment of nontechnical and team skills can facilitate improvement by encouraging both self-reflection and team reflection, identifying training needs, and informing operating room (OR) team training approaches. The observational teamwork assessment for surgery (OTAS) tool is a well-validated and robust tool for capturing teamwork in the operating room. The aims of the present study were to systematically adapt and refine the OTAS for German-speaking OR staff and to test the adapted assessment tool (OTAS-D) for psychometric properties and metric equivalence. METHODS: The study was carried out in three stages: at stage 1, OTAS was translated into German. At stage 2, experienced German OR experts (surgeons, OR nurses, anesthetists) were interviewed. At stage 3, two blinded assessors observed 11 general surgical operations (general surgical and vascular procedures) and interrater reliability was tested for refined OTAS-D behavioral exemplars and scorings. RESULTS: The German OR experts confirmed the applicability and content validity of the vast majority of translated behavioral exemplars. After their evaluation, 32 items were changed slightly, six were changed substantially, and one item was added. During observations, perfect and substantial interobserver agreement was found for 77 behavioral exemplars (67.1 % of the items, kappa coefficient >0.60). Rating at all OTAS behaviors showed acceptable levels of reliability (intraclass correlation coefficients >0.72). CONCLUSIONS: The OTAS-D is a tool for valid and reliable assessment of nontechnical skills that contribute to safe and effective surgical performance in ORs staffed by German-speaking professionals. Furthermore, our study serves as an example for systematically adapting and customizing well-established observational tools across different healthcare environments.

Culture-Fair Cognitive Screening Tools for Assessment of Cognitive Impairment: A Systematic Review.

Background: Cognitive screening tools are important in the detection of dementia, including Alzheimer's disease; however, they may contain cultural biases. Objective: This review examines culture-fair cognitive screening tools and evaluates their screening accuracy, strengths, and limitations. Methods: Medline, Embase, PsychINFO and CINAHL were searched. The protocol was registered on PROSPERO (CRD42021288776). Included studies used a culture-fair tool to assess cognition in older adults from varying ethnicities. Narrative synthesis was conducted. Results: 28 studies were included assessing eleven different tools. The Rowland Universal Dementia Assessment Scale (RUDAS) was as accurate as the Mini-Mental State Examination (MMSE) (AUC 0.62-0.93), with a similar sensitivity (52-94%) and better specificity (70-98%), and the Multicultural Cognitive Examination (MCE) had improved screening accuracy (AUC 0.99) compared to RUDAS (AUC 0.92). The Visual Cognitive Assessment Test (VCAT) was equivalent to MMSE (AUC 0.84-0.91). The Kimberley Indigenous Cognitive Assessment tool (KICA) had AUC of 0.93-0.95; sensitivity of 90.6%, specificity 92.6%. Conclusions: The RUDAS, KICA and VCAT were superior to MMSE for screening dementia in ethnic minorities. Other tools also showed good screening accuracy. Further research should be done to validate tools in different populations.

Training faculty in nontechnical skill assessment: national guidelines on program requirements.

OBJECTIVE: To develop guidelines for a faculty training program in nontechnical skill assessment in surgery. BACKGROUND: Nontechnical skills in the operating room are critical for patient safety. The successful integration of these skills into workplace-based assessment is dependent upon the availability of faculty who are able to teach and assess them. At present, no guidelines exist regarding the training requirements for such faculty in surgical contexts. METHODS: The development of the guidelines was carried out in several stages: stage 1-a detailed literature review on current training for nontechnical skill assessors; stage 2-semistructured interviews with a multidisciplinary panel (consisting of clinicians and psychologists/human factors specialists) of experts in surgical nontechnical skills; and stage 3-interview findings fed into an Expert Consensus Panel (ECP) Delphi approach to establish consensus regarding training requirements for faculty assessing nontechnical skills in surgery. RESULTS: The ECP agreed that training in nontechnical skill assessment should be delivered by a multidisciplinary team consisting of clinicians and psychologists/human factors specialists. The ECP reached consensus regarding who should be targeted to be trained as faculty (including proficiency and revalidation requirements). Consensus was reached on 7 essential training program content elements (including training in providing feedback/debriefing) and 8 essential methods of evaluating the effectiveness of a "train-the-trainers" program. CONCLUSIONS: This study provides evidence-based guidelines that can be used to guide the development and evaluation of programs to educate faculty in the training and assessment of nontechnical skills. Uptake of these guidelines could accelerate the development of surgical expertise required for safe and high-quality patient care.

Evaluation of 28-day estradiol and progesterone vaginal rings in a phase 1 clinical pharmacokinetic study.

OBJECTIVE: The aim of this work is to develop a combination of 17ß-estradiol (E2) and progesterone (P4) in a single-dose intravaginal ring (IVR) for the treatment of vasomotor symptoms (VMS) and genitourinary syndrome of menopause while providing endometrial protection. The objective of this study was to evaluate DARE-HRT1, a 28-day IVR that continuously delivers E2 and P4, in a phase 1 clinical trial to assess its pharmacokinetics. METHODS: This was an open-label, three-arm (group) study. Thirty-two (32) healthy postmenopausal women were recruited at two Australian sites. The average age was 57.2 years (47-69 y). The first arm received one ring for 28 days designed to release E2 at a rate of 80 µg/d and P4 at 4 mg/d (80/4 IVR); the second arm received a ring releasing E2 at 160 µg/d and P4 at 8 mg/d (160/8 IVR). The third arm received oral Estrofem (1 mg E2) and Prometrium (100 mg P4) both daily for 29 days. Blood samples were taken predose then intensively over the first day (day 1) and periodically thereafter over the remaining 27 days. After removal of the rings on the morning of day 29, intensive samples were collected. Similar procedures were conducted with women enrolled in the oral group. The plasma samples were analyzed for E2, estrone (E1), and P4 using validated bioanalytical methods. RESULTS: The baseline-adjusted steady-state plasma levels of E2 and P4 from 80/4 IVR were 20.4 ± 17.1 pg/mL and 1.32 ± 0.19 ng/mL (n = 10), respectively. The baseline-adjusted steady-state plasma levels of E2 and P4 from 160/8 IVR were 30.9 ± 8.7 pg/mL and 2.08 ± 0.50 ng/mL (n = 10), respectively. The baseline-adjusted average plasma concentrations of E2 and P4 at day 29 of the oral group were 35.4 ± 11.2 pg/mL and 0.79 ± 0.72 ng/mL (n = 11), respectively. The baseline-adjusted steady state of E1 from the 80/4 IVR and the 160/8 IVR were 22.1 ± 16.6 pg/mL (n = 10) and 25.2 ± 12.3 pg/mL (n = 10), respectively. The baseline-adjusted concentration of E1 in the oral arm was 209 ± 67.7 ng/mL (n = 11). The IVR were well tolerated, and no serious adverse events were reported. CONCLUSIONS: The 80/4 IVR and 160/8 IVR gave similar steady-state concentrations of E2 as seen with drug products approved by the US Food and Drug Administration for treatment of VMS and genitourinary symptoms of menopause. The E2 concentrations of this study support the potential of DARE-HRT1, a promising new option for hormone therapy for treatment of VMS and vaginal symptoms associated with menopause.

A continuity of care programme for women at risk of preterm birth in the UK: Process evaluation of a hybrid randomised controlled pilot trial.

BACKGROUND: The development and evaluation of specific maternity care packages designed to address preterm birth remains a public health priority. We aim to evaluate the implementation, context, and potential mechanisms of action, of a new care pathway that combined midwifery continuity of care with a specialist obstetric clinic for women at risk of preterm birth (POPPIE) in London (UK). METHODS: We did a multiphase mixed method triangulation evaluation nested within a hybrid type 2, randomised controlled trial in London (United Kingdom). Pregnant women with identified risk factors for preterm birth were eligible for trial participation and randomly assigned (1:1) to either midwifery continuity of care linked to a specialist obstetric clinic (POPPIE group) or standard maternity care. The primary outcome was a composite of appropriate and timely interventions for the prevention and/or management of preterm labour and birth, analysed according to intention to treat. Clinical and process outcome data were abstracted from medical records and electronic data systems, and coded by study team members, who were masked to study group allocation. Implementation data were collected from meeting records and key documents, postnatal surveys (n = 164), semi-structured interviews with women (n = 30), healthcare providers and stakeholders (n = 24) pre-, mid and post implementation. Qualitative and quantitative data from meeting records and key documents were examined narratively. Qualitative data from interviews were analysed using three thematic frameworks: Proctor's (for implementation outcomes: appropriateness, adoption, feasibility, acceptability, fidelity, penetration, sustainability), the Consolidated Framework for Implementation Research (for determinants of implementation), and published program theories of continuity models (for potential mechanisms). Data triangulation followed a convergent parallel and pragmatic approach which brought quantitative and qualitative data together at the interpretation stage. We averaged individual implementation measures across all domains to give a single composite implementation strength score which was compared to the primary outcome. RESULTS: Between May 9, 2017, and Sep 30, 2018, 553 women were assessed for eligibility and 334 were enrolled with less than 6% of loss to follow up (169 were assigned to the POPPIE group; 165 were to the standard group). There was no difference in the primary outcome (POPPIE group 83·3% versus standard group 84·7%; risk ratio 0·98 [95% CI 0·90 to 1·08]). Appropriateness and adoption: The introduction of the POPPIE model was perceived as a positive fundamental change for local maternity services. Partnership working and additional funding were crucial for adoption. Fidelity: More than 75% of antenatal and postnatal visits were provided by a named or partner midwife, and a POPPIE midwife was present in more than 80% of births. Acceptability: Nearly 98% of women who responded to the postnatal survey were very satisfied with POPPIE model. Quantitative fidelity and acceptability results were supported by the qualitative findings. Penetration and sustainability: Despite delays (likely associated with lack of existing continuity models at the hospital), the model was embedded within established services and a joint decision was made to sustain and adapt the model after the trial (strongly facilitated by national maternal policy on continuity pathways). Potential mechanisms of impact identified included e.g. access to care, advocacy and perceptions of safety and trust. There was no association between implementation measures and the primary outcome. CONCLUSIONS: The POPPIE model of care was a feasible and acceptable model of care that was implemented with high fidelity and sustained in maternity services. Larger powered trials are feasible and needed in other settings, to evaluate the impact and implementation of continuity programmes in other communities affected by preterm birth and women who experience social disadvantage and vulnerability. TRIAL REGISTRATION: UKCRN Portfolio Database (prospectively registered, 24 April 2017): 31951. ISRCTN registry (retrospectively registered, 21 August 2017): ISRCTN37733900.

Observational teamwork assessment for surgery: feasibility of clinical and nonclinical assessor calibration with short-term training.

OBJECTIVES: To assess the feasibility of training clinical and nonclinical novice assessors to rate teamwork behavior in the operating room with short-term structured training using the observational teamwork assessment for surgery (OTAS) tool. BACKGROUND: Effective teamwork is fundamental to the delivery of optimal patient care in the operating room (OR). OTAS provides a comprehensive and robust measure of teamwork in surgery. To date, assessors with a background in psychology/human factors have been shown to be able to use OTAS reliably after training. However, the feasibility of observer training over a short timescale and accessibility to the wider clinical community (ie, OTAS use by clinicians) are yet to be empirically demonstrated. METHODS: Ten general surgery cases were observed and assessed using OTAS in real-time by an expert in rating OTAS behaviors (100+ cases rated) and 4 novices: 2 psychologists and 2 surgeons. Assessors were blinded to each other's scores during observations. After each observation, scores were compared and discussed between expert and novice assessors in a debriefing session. RESULTS: All novices were reliable with the expert to a acceptable degree at rating all OTAS behaviors by the end of training (intraclass correlation coefficients ≥0.68). For 3 of the 5 behaviors (communication, cooperation, and leadership), calibration improved most rapidly across the first 7 observed cases. For monitoring/situational awareness, calibration improved steadily across the 10 observed cases. For coordination, no significant improvement in calibration over time was observed because of high interrater reliability from the outset (ie, a ceiling effect). There was no significant difference between surgeons and psychologists in their calibration with the expert. CONCLUSIONS: It is feasible to train both clinicians and nonclinicians to use OTAS to assess teamwork behaviors in ORs over a short structured training period. OTAS is an accessible tool that can be used robustly (ie, reliably) by assessors from both clinical and nonclinical backgrounds.

Objective structured assessment of debriefing: bringing science to the art of debriefing in surgery.

OBJECTIVE: : To identify the features of effective debriefing and to use this to develop and validate a tool for assessing such debriefings. INTRODUCTION: : Simulation-based training has become an accepted means of surgical skill acquisition. A key component of this is debriefing-yet there is a paucity of research to guide best practice. METHODS: : Phase 1-Identification of best practice and tool development. A search of the Medline, Embase, PsycINFO, and ERIC databases identified current evidence on debriefing. End-user input was obtained through 33 semistructured interviews conducted with surgeons (n = 18) and other operating room personnel (n = 15) from 3 continents (UK, USA, Australia) using standardized qualitative methodology. An expert panel (n = 7) combined the data to create the Objective Structured Assessment of Debriefing (OSAD) tool. Phase 2-Psychometric testing. OSAD was tested for feasibility, reliability, and validity by 2 independent assessors who rated 20 debriefings following high-fidelity simulations. RESULTS: : Phase 1: 28 reports on debriefing were retrieved from the literature. Key components of an effective debriefing identified from these reports and the 33 interviews included: approach to debriefing, learning environment, learner engagement, reaction, reflection, analysis, diagnosis of strengths and areas for improvement, and application to clinical practice. Phase 2: OSAD was feasible, reliable [inter-rater ICC (intraclass correlation coefficient) = 0.88, test-retest ICC = 0.90], and face and content valid (content validity index = 0.94). CONCLUSIONS: : OSAD provides an evidence-based, end-user informed approach to debriefing in surgery. By quantifying the quality of a debriefing, OSAD has the potential to identify areas for improving practice and to optimize learning during simulation-based training.

Defining, conceptualizing and evaluating pragmatic qualities of quantitative instruments measuring implementation determinants and outcomes: a scoping and critical review of the literature and recommendations for future research.

The pragmatic (i.e., practical) quality of quantitative implementation measures has received increased attention in the implementation science literature in recent years. Implementation measures that are judged to be pragmatic by implementation stakeholders are thought to be more likely to be applied in research and practice. Despite the need for pragmatic implementation measures, ambiguity and uncertainty regarding what constitutes a pragmatic measure remains. This study sought to identify and critically appraise the published literature to understand (i) how pragmatism is defined as a measurement construct/quality of implementation determinants and outcome instruments; (ii) how pragmatic qualities of instruments are evaluated; (iii) identify key gaps and limitations of the current evidence-base and (iv) identify recommendations for future research. We conducted a scoping review of the literature also employing methods of critical review. PubMed and PsycINFO databases, using the OVID interface, were searched for relevant articles published between January 2010 and September 2020. Articles that contained a definition and/or described characteristics of "pragmatism" as a measurement construct of quantitative implementation outcomes (as defined by Proctor's Implementation Outcomes taxonomy) and/or implementation determinants were eligible for inclusion. Nine articles met inclusion criteria. A degree of overlap in definitions and terms used to describe the pragmatic qualities of quantitative implementation determinant and outcome instruments were found. The most frequently cited descriptors of pragmatism were "not burdensome", "brief", "reliable", "valid" and "sensitive to change". 3 of the 9 included articles involved international implementation stakeholders in defining and conceptualizing pragmatism and employed specific methods to do so, including a systematic literature review, stakeholder interviews, concept mapping, and a Delphi process. All other articles defined pragmatism, with or without citing relevant literature. One article objectively assessed the pragmatic qualities, above and beyond the psychometric qualities, of implementation measures, using the Psychometric and Pragmatic Evidence Rating Scale (PAPERS). The evidence base within the implementation instrumentation literature on what pragmatism is and how it might be assessed is limited. Some of the research identified in the review provides a strong foundation to build upon, by testing its applicability in other settings (including healthcare areas and countries) and among a more diverse group of stakeholders. We discuss directions for further development of the concept of pragmatism relating to the measurement of implementation determinants and outcomes.

The need for pragmatic (i.e., practical) measures to evaluate implementation efforts has been repeatedly called for in the implementation science literature. This literature review focuses on understanding how pragmatism, as a measurement construct and quality of implementation measures, is defined, conceptualized and evaluated. We identified few articles (n = 9) that contribute to our understanding of how pragmatism is defined and evaluated. We found that the most frequently used terms to describe pragmatic qualities of implementation measures include "not burdensome", "brief", "reliable", "valid" and "sensitive to change". We identified one scale, the Psychometric and Pragmatic Evidence Rating Scale (PAPERS), developed to measure the pragmatic quality, as well as psychometric quality, of implementation measures. We identified several gaps and limitations of the current evidence-base and offer directions to further develop the concept and appraisal of pragmatism. Specifically, we recommend that future research focus on engaging and involving a wider diversity of implementation stakeholders in defining and conceptualizing pragmatism as well as subjecting existing pragmatic assessment measures to more rigorous and extensive evaluation.

Development and psychometric evaluation of the Implementation Science Research Project Appraisal Criteria (ImpResPAC) tool: a study protocol.

INTRODUCTION: The need for quantitative criteria to appraise the quality of implementation research has recently been highlighted to improve methodological rigour. The Implementation Science Research development (ImpRes) tool and supplementary guide provide methodological guidance and recommendations on how to design high-quality implementation research. This protocol reports on the development of the Implementation Science Research Project Appraisal Criteria (ImpResPAC) tool, a quantitative appraisal tool, developed based on the structure and content of the ImpRes tool and supplementary guide, to evaluate the conceptual and methodological quality of implementation research. METHODS AND ANALYSIS: This study employs a three-stage sequential mixed-methods design. During stage 1, the research team will map core domains of the ImpRes tool, guidance and recommendations contained in the supplementary guide and within the literature, to ImpResPAC. In stage 2, an international multidisciplinary expert group, recruited through purposive sampling, will inform the refinement of ImpResPAC, including content, scoring system and user instructions. In stage 3, an extensive psychometric evaluation of ImpResPAC, that was created in stage 1 and refined in stage 2, will be conducted. The scaling assumptions (inter-item and item-total correlations), reliability (internal consistency, inter-rater) and validity (construct and convergent validity) will be investigated by applying ImpResPAC to 50 protocols published in Implementation Science. We envisage developing ImpResPAC in this way will provide implementation research stakeholders, primarily grant reviewers and educators, a comprehensive, transparent and fair appraisal of the conceptual and methodological quality of implementation research, increasing the likelihood of funding research that will generate knowledge and contribute to the advancement of the field. ETHICS AND DISSEMINATION: This study will involve human participants. This study has been registered and minimal risk ethical clearance granted by The Research Ethics Office, King's College London (reference number MRA-20/21-20807). Participants will receive written information on the study via email and will provide e-consent if they wish to participate. We will use traditional academic modalities of dissemination (eg, conferences and publications).

A systematic review and meta-analysis of digital application use in clinical research in pain medicine.

Importance: Pain is a silent global epidemic impacting approximately a third of the population. Pharmacological and surgical interventions are primary modes of treatment. Cognitive/behavioural management approaches and interventional pain management strategies are approaches that have been used to assist with the management of chronic pain. Accurate data collection and reporting treatment outcomes are vital to addressing the challenges faced. In light of this, we conducted a systematic evaluation of the current digital application landscape within chronic pain medicine. Objective: The primary objective was to consider the prevalence of digital application usage for chronic pain management. These digital applications included mobile apps, web apps, and chatbots. Data sources: We conducted searches on PubMed and ScienceDirect for studies that were published between 1st January 1990 and 1st January 2021. Study selection: Our review included studies that involved the use of digital applications for chronic pain conditions. There were no restrictions on the country in which the study was conducted. Only studies that were peer-reviewed and published in English were included. Four reviewers had assessed the eligibility of each study against the inclusion/exclusion criteria. Out of the 84 studies that were initially identified, 38 were included in the systematic review. Data extraction and synthesis: The AMSTAR guidelines were used to assess data quality. This assessment was carried out by 3 reviewers. The data were pooled using a random-effects model. Main outcomes and measures: Before data collection began, the primary outcome was to report on the standard mean difference of digital application usage for chronic pain conditions. We also recorded the type of digital application studied (e.g., mobile application, web application) and, where the data was available, the standard mean difference of pain intensity, pain inferences, depression, anxiety, and fatigue. Results: 38 studies were included in the systematic review and 22 studies were included in the meta-analysis. The digital interventions were categorised to web and mobile applications and chatbots, with pooled standard mean difference of 0.22 (95% CI: -0.16, 0.60), 0.30 (95% CI: 0.00, 0.60) and -0.02 (95% CI: -0.47, 0.42) respectively. Pooled standard mean differences for symptomatologies of pain intensity, depression, and anxiety symptoms were 0.25 (95% CI: 0.03, 0.46), 0.30 (95% CI: 0.17, 0.43) and 0.37 (95% CI: 0.05, 0.69), respectively. A sub-group analysis was conducted on pain intensity due to the heterogeneity of the results (I 2 = 82.86%; p = 0.02). After stratifying by country, we found that digital applications were more likely to be effective in some countries (e.g., United States, China) than others (e.g., Ireland, Norway). Conclusions and relevance: The use of digital applications in improving pain-related symptoms shows promise, but further clinical studies would be needed to develop more robust applications. Systematic Review Registration: https://www.crd.york.ac.uk/prospero/, identifier: CRD42021228343.

Evaluating the Application of the RE-AIM Planning and Evaluation Framework: An Updated Systematic Review and Exploration of Pragmatic Application.

BACKGROUND: RE-AIM is one of the most widely applied frameworks to plan and evaluate the implementation of public health and health behavior change interventions. The objective of this review is to provide an updated synthesis of use of the RE-AIM (Reach Effectiveness Adoption Implementation and Maintenance) planning and evaluation framework and explore pragmatic use (i.e., partial application of the framework) and how this is reported. METHODS: Systematic review. MEDLINE (R) and PsycINFO were searched, via the Ovid interface, between January 2011 and December 2017. Studies that applied RE-AIM as a planning and/or evaluation framework were included. RESULTS: One hundred fifty-seven articles met inclusion criteria. One hundred forty-nine reported using RE-AIM for evaluation, three for planning and five for planning and evaluation. Reach was the most frequently reported dimension (92.9%), followed by implementation (90.3%), adoption (89.7%), effectiveness (84.5%), and maintenance (77.4%). One hundred forty-seven/one hundred fifty-seven articles originated from high-income economy countries. Within a sub-set analysis (10% of included articles), 9/15 articles evaluated all dimensions. Of the 6/15 articles that did not evaluate all dimensions, five provided no justification for pragmatic application. CONCLUSIONS: RE-AIM has gained increased use in recent years and there is evidence that it is being applied pragmatically. However, the rationale for pragmatic use is often not reported. SYSTEMATIC REVIEW REGISTRATION: PROSPERO (CRD42017054616).

Adaptation of the World Health Organization Electronic Mental Health Gap Action Programme Intervention Guide App for Mobile Devices in Nepal and Nigeria: Protocol for a Feasibility Cluster Randomized Controlled Trial.

BACKGROUND: There is a growing global need for scalable approaches to training and supervising primary care workers (PCWs) to deliver mental health services. Over the past decade, the World Health Organization Mental Health Gap Action Programme Intervention Guide (mhGAP-IG) and associated training and implementation guidance have been disseminated to more than 100 countries. On the basis of the opportunities provided by mobile technology, an updated electronic Mental Health Gap Action Programme Intervention Guide (e-mhGAP-IG) is now being developed along with a clinical dashboard and guidance for the use of mobile technology in supervision. OBJECTIVE: This study aims to assess the feasibility, acceptability, adoption, and other implementation parameters of the e-mhGAP-IG for diagnosis and management of depression in 2 lower-middle-income countries (Nepal and Nigeria) and to conduct a feasibility cluster randomized controlled trial (cRCT) to evaluate trial procedures for a subsequent fully powered trial comparing the clinical effectiveness and cost-effectiveness of the e-mhGAP-IG and remote supervision with standard mhGAP-IG implementation. METHODS: A feasibility cRCT will be conducted in Nepal and Nigeria to evaluate the feasibility of the e-mhGAP-IG for use in depression diagnosis and treatment. In each country, an estimated 20 primary health clinics (PHCs) in Nepal and 6 PHCs in Nigeria will be randomized to have their staff trained in e-mhGAP-IG or the paper version of mhGAP-IG v2.0. The PHC will be the unit of clustering. All PCWs within a facility will receive the same training (e-mhGAP-IG vs paper mhGAP-IG). Approximately 2-5 PCWs, depending on staffing, will be recruited per clinic (estimated 20 health workers per arm in Nepal and 15 per arm in Nigeria). The primary outcomes of interest will be the feasibility and acceptability of training, supervision, and care delivery using the e-mhGAP-IG. Secondary implementation outcomes include the adoption of the e-mhGAP-IG and feasibility of trial procedures. The secondary intervention outcome-and the primary outcome for a subsequent fully powered trial-will be the accurate identification of depression by PCWs. Detection rates before and after training will be compared in each arm. RESULTS: To date, qualitative formative work has been conducted at both sites to prepare for the pilot feasibility cRCT, and the e-mhGAP-IG and remote supervision guidelines have been developed. CONCLUSIONS: The incorporation of mobile digital technology has the potential to improve the scalability of mental health services in primary care and enhance the quality and accuracy of care. TRIAL REGISTRATION: ClinicalTrials.gov NCT04522453; https://clinicaltrials.gov/ct2/show/NCT04522453. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/24115.