When words fail us: An integrative review of innovative elicitation techniques for qualitative interviews.

INTRODUCTION: Interviews are central to many qualitative studies in health professions education (HPE). However, researchers often struggle to elicit rich data and engage diverse participants who may find this strategy exclusionary. Elicitation techniques are strategies tailored to address these challenges, enhancing oral conversations through other forms of interaction-for example, participant photography and neighbourhood walks. These strategies are tailored to elicit the rich data needed to address complex problems and meaningfully engage participants. Unfortunately, guidance on these techniques is scattered across literatures from diverse fields. In this synthesis, we offer an overview of the elicitation techniques available and advice about how to choose between them. METHODS: We conducted an integrative review, drawing on methodological literature from across the health and social sciences. Our interdisciplinary searches yielded 3056 citations. We included 293 citations that were methodologically focused and discussed elicitation techniques used in interviews with adults. We then extracted specific elicitation techniques, summarising each technique to capture key features, as well as strengths and weaknesses. From this, we developed a framework to help researchers identify challenges in their interview-based research and to select elicitation techniques that address their challenges. RESULTS: To enrich data, researchers might seek to shift conversations away from participants' entrenched narratives, to externalise conversations on sensitive topics, or to elicit affect, tacit knowledge or contextual details. When empowering participants, researchers might seek to increase equity between the researcher and participant or foster interview accessibility across diverse participant populations. DISCUSSION: When chosen with study goals in mind, elicitation techniques can enrich interview data. To harness this potential, we need to re-conceptualise interviews as co-production of knowledge by researcher(s) and participant(s). To make interviews more equitable and accessible, we need to consider flexibility so that each participant can engage in ways that best suit their needs and preferences.

CONFERD-HP: recommendations for reporting COmpeteNcy FramEwoRk Development in health professions.

BACKGROUND: Competency frameworks outline the perceived knowledge, skills, attitudes, and other attributes required for professional practice. These frameworks have gained in popularity, in part for their ability to inform health professions education, assessment, professional mobility, and other activities. Previous research has highlighted inadequate reporting related to their development which may then jeopardize their defensibility and utility. METHODS: This study aimed to develop a set of minimum reporting criteria for developers and authors of competency frameworks in an effort to improve transparency, clarity, interpretability and appraisal of the developmental process, and its outputs. Following guidance from the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network, an expert panel was assembled, and a knowledge synthesis, a Delphi study, and workshops were conducted using individuals with experience developing competency frameworks, to identify and achieve consensus on the essential items for a competency framework development reporting guideline. RESULTS: An initial checklist was developed by the 35-member expert panel and the research team. Following the steps listed above, a final reporting guideline including 20 essential items across five sections (title and abstract; framework development; development process; testing; and funding/conflicts of interest) was developed. CONCLUSION: The COmpeteNcy FramEwoRk Development in Health Professions (CONFERD-HP) reporting guideline permits a greater understanding of relevant terminology, core concepts, and key items to report for competency framework development in the health professions.

Improving social justice in observational studies: protocol for the development of a global and Indigenous STROBE-equity reporting guideline.

BACKGROUND: Addressing persistent and pervasive health inequities is a global moral imperative, which has been highlighted and magnified by the societal and health impacts of the COVID-19 pandemic. Observational studies can aid our understanding of the impact of health and structural oppression based on the intersection of gender, race, ethnicity, age and other factors, as they frequently collect this data. However, the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline, does not provide guidance related to reporting of health equity. The goal of this project is to develop a STROBE-Equity reporting guideline extension. METHODS: We assembled a diverse team across multiple domains, including gender, age, ethnicity, Indigenous background, disciplines, geographies, lived experience of health inequity and decision-making organizations. Using an inclusive, integrated knowledge translation approach, we will implement a five-phase plan which will include: (1) assessing the reporting of health equity in published observational studies, (2) seeking wide international feedback on items to improve reporting of health equity, (3) establishing consensus amongst knowledge users and researchers, (4) evaluating in partnership with Indigenous contributors the relevance to Indigenous peoples who have globally experienced the oppressive legacy of colonization, and (5) widely disseminating and seeking endorsement from relevant knowledge users. We will seek input from external collaborators using social media, mailing lists and other communication channels. DISCUSSION: Achieving global imperatives such as the Sustainable Development Goals (e.g., SDG 10 Reduced inequalities, SDG 3 Good health and wellbeing) requires advancing health equity in research. The implementation of the STROBE-Equity guidelines will enable a better awareness and understanding of health inequities through better reporting. We will broadly disseminate the reporting guideline with tools to enable adoption and use by journal editors, authors, and funding agencies, using diverse strategies tailored to specific audiences.

Inappropriate use of clinical practices in Canada: a systematic review.

BACKGROUND: Inappropriate health care leads to negative patient experiences, poor health outcomes and inefficient use of resources. We aimed to conduct a systematic review of inappropriately used clinical practices in Canada. METHODS: We searched multiple bibliometric databases and grey literature to identify inappropriately used clinical practices in Canada between 2007 and 2021. Two team members independently screened citations, extracted data and assessed methodological quality. Findings were synthesized in 2 categories: diagnostics and therapeutics. We reported ranges of proportions of inappropriate use for all practices. Medians and interquartile ranges (IQRs), based on the percentage of patients not receiving recommended practices (underuse) or receiving practices not recommended (overuse), were calculated. All statistics are at the study summary level. RESULTS: We included 174 studies, representing 228 clinical practices and 28 900 762 patients. The median proportion of inappropriate care, as assessed in the studies, was 30.0% (IQR 12.0%-56.6%). Underuse (median 43.9%, IQR 23.8%-66.3%) was more frequent than overuse (median 13.6%, IQR 3.2%-30.7%). The most frequently investigated diagnostics were glycated hemoglobin (underused, range 18.0%-85.7%, n = 9) and thyroid-stimulating hormone (overused, range 3.0%-35.1%, n = 5). The most frequently investigated therapeutics were statin medications (underused, range 18.5%-71.0%, n = 6) and potentially inappropriate medications (overused, range 13.5%-97.3%, n = 9). INTERPRETATION: We have provided a summary of inappropriately used clinical practices in Canadian health care systems. Our findings can be used to support health care professionals and quality agencies to improve patient care and safety in Canada.

Patient involvement in assessment of postgraduate medical learners: A scoping review.

CONTEXT: Competency-based assessment of learners may benefit from a more holistic, inclusive, approach for determining readiness for unsupervised practice. However, despite movements towards greater patient partnership in health care generally, inclusion of patients in postgraduate medical learners' assessment is largely absent. METHODS: We conducted a scoping review to map the nature, extent and range of literature examining the inclusion (or exclusion) of patients within the assessment of postgraduate medical learners. Guided by Arskey and O'Malley's framework and informed by Levac et al. and Thomas et al., we searched two databases (MEDLINE® and Embase®) from inception until February 2021 using subheadings related to assessment, patients and postgraduate learners. Data analysis examined characteristics regarding the nature and factor influencing patient involvement in assessment. RESULTS: We identified 41 papers spanning four decades. Some literature suggests patients are willing to be engaged in assessment, however choose not to engage when, for example, language barriers may exist. When stratified by specialty or clinical setting, the influence of factors such as gender, race, ethnicity or medical condition seems to remain consistent. Patients may participate in assessment as a stand-alone group or part of a multi-source feedback process. Patients generally provided high ratings but commented on the observed professional behaviours and communication skills in comparison with physicians who focused on medical expertise. CONCLUSION: Factors that influence patient involvement in assessment are multifactorial including patients' willingness themselves, language and reading-comprehension challenges and available resources for training programmes to facilitate the integration of patient assessments. These barriers however are not insurmountable. While understudied, research examining patient involvement in assessment is increasing; however, our review suggests that the extent which the unique insights will be taken up in postgraduate medical education may be dependent on assessment systems readiness and, in particular, physician readiness to partner with patients in this way.

Improving WHO's understanding of WHO guideline uptake and use in Member States: a scoping review.

BACKGROUND: WHO publishes public health and clinical guidelines to guide Member States in achieving better health outcomes. Furthermore, WHO's Thirteenth General Programme of Work for 2019-2023 prioritizes strengthening its normative functional role and uptake of normative and standard-setting products, including guidelines at the country level. Therefore, understanding WHO guideline uptake by the Member States, particularly the low- and middle-income countries (LMICs), is of utmost importance for the organization and scholarship. METHODS: We conducted a scoping review using a comprehensive search strategy to include published literature in English between 2007 and 2020. The review was conducted between May and June 2021. We searched five electronic databases including CINAHL, the Cochrane Library, PubMed, Embase and Scopus. We also searched Google Scholar as a supplementary source. The review adhered to the PRISMA-ScR (PRISMA extension for scoping reviews) guidelines for reporting the searches, screening and identification of evaluation studies from the literature. A narrative synthesis of the evidence around key barriers and challenges for WHO guideline uptake in LMICs is thematically presented. RESULTS: The scoping review included 48 studies, and the findings were categorized into four themes: (1) lack of national legislation, regulations and policy coherence, (2) inadequate experience, expertise and training of healthcare providers for guideline uptake, (3) funding limitations for guideline uptake and use, and (4) inadequate healthcare infrastructure for guideline compliance. These challenges were situated in the Member States' health systems. The findings suggest that governance was often weak within the existing health systems amongst most of the LMICs studied, as was the guidance provided by WHO's guidelines on governance requirements. This challenge was further exacerbated by a lack of accountability and transparency mechanisms for uptake and implementation of guidelines. In addition, the WHO guidelines themselves were either unclear and were technically challenging for some health conditions; however, WHO guidelines were primarily used as a reference by Member States when they developed their national guidelines. CONCLUSIONS: The challenges identified reflect the national health systems' (in)ability to allocate, implement and monitor the guidelines. Historically this is beyond the remit of WHO, but Member States could benefit from WHO implementation guidance on requirements and needs for successful uptake and use of WHO guidelines.

Continuing education meetings and workshops: effects on professional practice and healthcare outcomes.

BACKGROUND: Educational meetings are used widely by health personnel to provide continuing medical education and to promote implementation of innovations or translate new knowledge to change practice within healthcare systems. Previous reviews have concluded that educational meetings can result in small changes in behaviour, but that effects vary considerably. Investigations into which characteristics of educational meetings might lead to greater impact have yielded varying results, and factors that might explain heterogeneity in effects remain unclear. This is the second update of this Cochrane Review. OBJECTIVES: • To assess the effects of educational meetings on professional practice and healthcare outcomes • To investigate factors that might explain the heterogeneity of these effects SEARCH METHODS: We searched CENTRAL, MEDLINE, Embase, ERIC, Science Citation Index Expanded (ISI Web of Knowledge), and Social Sciences Citation Index (last search in November 2016). SELECTION CRITERIA: We sought randomised trials examining the effects of educational meetings on professional practice and patient outcomes. DATA COLLECTION AND ANALYSIS: Two review authors independently extracted data and assessed risk of bias. One review author assessed the certainty of evidence (GRADE) and discussed with a second review author. We included studies in the primary analysis that reported baseline data and that we judged to be at low or unclear risk of bias. For each comparison of dichotomous outcomes, we measured treatment effect as risk difference adjusted for baseline compliance. We expressed adjusted risk difference values as percentages, and we noted that values greater than zero favour educational meetings. For continuous outcomes, we measured treatment effect as per cent change relative to the control group mean post test, adjusted for baseline performance; we expressed values as percentages and noted that values greater than zero favour educational meetings. We report means and 95% confidence intervals (CIs) and, when appropriate, medians and interquartile ranges to facilitate comparisons to previous versions of this review. We analysed professional and patient outcomes separately and analysed 22 variables that were hypothesised a priori to explain heterogeneity. We explored heterogeneity by using univariate meta-regression and by inspecting violin plots. MAIN RESULTS: We included 215 studies involving more than 28,167 health professionals, including 142 new studies for this update. Educational meetings as the single intervention or the main component of a multi-faceted intervention compared with no intervention • Probably slightly improve compliance with desired practice when compared with no intervention (65 comparisons, 7868 health professionals for dichotomous outcomes (adjusted risk difference 6.79%, 95% CI 6.62% to 6.97%; median 4.00%; interquartile range 0.29% to 13.00%); 28 comparisons, 2577 health professionals for continuous outcomes (adjusted relative percentage change 44.36%, 95% CI 41.98% to 46.75%; median 20.00%; interquartile range 6.00% to 65.00%)) • Probably slightly improve patient outcomes compared with no intervention (15 comparisons, 2530 health professionals for dichotomous outcomes (adjusted risk difference 3.30%, 95% CI 3.10% to 3.51%; median 0.10%; interquartile range 0.00% to 4.00%); 28 comparisons, 2294 health professionals for continuous outcomes (adjusted relative percentage change 8.35%, 95% CI 7.46% to 9.24%; median 2.00%; interquartile range -1.00% to 21.00%)) The certainty of evidence for this comparison is moderate. Educational meetings alone compared with other interventions • May improve compliance with desired practice when compared with other interventions (6 studies, 1402 health professionals for dichotomous outcomes (adjusted risk difference 9.99%, 95% CI 9.47% to 10.52%; median 16.5%; interquartile range 0.80% to 16.50%); 2 studies, 72 health professionals for continuous outcomes (adjusted relative percentage change 12.00%, 95% CI 9.16% to 14.84%; median 12.00%; interquartile range 0.00% to 24.00%)) No studies met the inclusion criteria for patient outcome measurements. The certainty of evidence for this comparison is low. Interactive educational meetings compared with didactic (lecture-based) educational meetings • We are uncertain of effects on compliance with desired practice (3 studies, 370 health professionals for dichotomous outcomes; 1 study, 192 health professionals for continuous outcomes) or on patient outcomes (1 study, 54 health professionals for continuous outcomes), as the certainty of evidence is very low Any other comparison of different formats and durations of educational meetings • We are uncertain of effects on compliance with desired practice (1 study, 19 health professionals for dichotomous outcomes; 1 study, 20 health professionals for continuous outcomes) or on patient outcomes (1 study, 113 health professionals for continuous outcomes), as the certainty of evidence is very low. Factors that might explain heterogeneity of effects Meta-regression suggests that larger estimates of effect are associated with studies judged to be at high risk of bias, with studies that had unit of analysis errors, and with studies in which the unit of analysis was the provider rather than the patient. Improved compliance with desired practice may be associated with: shorter meetings; poor baseline compliance; better attendance; shorter follow-up; professionals provided with additional take-home material; explicit building of educational meetings on theory; targeting of low- versus high-complexity behaviours; targeting of outcomes with high versus low importance; goal of increasing rather than decreasing behaviour; teaching by opinion leaders; and use of didactic versus interactive teaching methods. Pre-specified exploratory analyses of behaviour change techniques suggest that improved compliance with desired practice may be associated with use of a greater number of behaviour change techniques; goal-setting; provision of feedback; provision for social comparison; and provision for social support. Compliance may be decreased by the use of follow-up prompts, skills training, and barrier identification techniques. AUTHORS' CONCLUSIONS: Compared with no intervention, educational meetings as the main component of an intervention probably slightly improve professional practice and, to a lesser extent, patient outcomes. Educational meetings may improve compliance with desired practice to a greater extent than other kinds of behaviour change interventions, such as text messages, fees, or office systems. Our findings suggest that multi-strategy approaches might positively influence the effects of educational meetings. Additional trials of educational meetings compared with no intervention are unlikely to change the review findings; therefore we will not further update this review comparison in the future. However, we note that randomised trials comparing different types of education are needed.

The relevance of the LEADS framework during the COVID-19 pandemic.

COVID-19 has created a unique context for the practice of leadership in healthcare. Given the significant use of the LEADS in a Caring Environment capabilities framework (LEADS) in Canada's health system, it is important to document the relevancy of LEADS. The authors reviewed literature, conducted research, and reflected on their own experience to identify leadership practices during the pandemic and related them to LEADS. Findings are presented in three sections: Hindsight (before), Insight (during), and Foresight (post). We profile the issue of improving long-term Care to provide an example of how LEADS can be applied in crisis times. Our analysis suggests that while LEADS appears to specify the leadership capabilities needed, it requires adaptation to context. The vision Canada has for healthcare will dictate how LEADS will be used as a guide to leadership practice in the current context or to shape a bolder vision of healthcare's future.

Key factors for national spread and scale-up of an eConsult innovation.

BACKGROUND: Expanding healthcare innovations from the local to national level is a complex pursuit requiring careful assessment of all relevant factors. In this study (a component of a larger eConsult programme of research), we aimed to identify the key factors involved in the spread and scale-up of a successful regional eConsult model across Canada. METHODS: We conducted a constant comparative thematic analysis of stakeholder discussions captured during a full-day National eConsult Forum meeting held in Ottawa, Canada, on 11 December 2017. Sixty-four participants attended, representing provincial and territorial governments, national organisations, healthcare providers, researchers and patients. Proceedings were recorded, transcribed and underwent qualitative analysis using the Framework for Applied Policy Research. RESULTS: This study identified four main themes that were critical to support the intentional efforts to spread and scale-up eConsult across Canada, namely (1) identifying population care needs and access problems, (2) engaging stakeholders who were willing to roll up their sleeves and take action, (3) building on current strategies and policies, and (4) measuring and communicating outcomes. CONCLUSIONS: Efforts to promote innovation in healthcare are more likely to succeed if they are based on an understanding of the forces that drive the spread and scale-up of innovation. Further research is needed to develop and strengthen the conceptual and applied foundations of the spread and scale-up of healthcare innovations, especially in the context of emergent learning health systems across Canada and beyond.

Fundamentals of randomized designs: AMEE Guide No. 128.

This AMEE Guide summarizes fundamentals of a major experimental design option for medical education researchers: the randomised study. Medical education researchers face an overwhelming taxonomy of study design options; given the breadth of information on experimental design, the purpose of this Guide is to offer a resource for medical education researchers wishing to equip themselves with helpful information for when to match a study's objective and the use of randomised designs. Once a research question has been formulated study design is the cornerstone of the intricate, nested activities of any research project. Researchers negotiate many decisions in the pursuit of choosing an appropriate design approach; failure to do so can undermine a project's capacity to, for example, sufficiently test a hypothesis or theory. Written as an introduction, this Guide is intended for medical education researchers seeking to build on and synthesise the existing corpus of literature on experimental and quasi-experimental design approaches. While not comprehensive, presented are key concepts alongside relevant examples from the field of health professions education.

The use of BEME reviews in the medical education literature.

INTRODUCTION: Knowledge syntheses in medical education are intended to promote the translation to, and mobilization of, research knowledge into practice. Despite the effort invested in conducting them, how these knowledge syntheses are used is unclear. This study aimed to explore how knowledge syntheses published by the Best Evidence Medical Education Collaboration (BEME) have been used in a cross-section of published literature. METHODS: Citation patterns for BEME reviews were explored using data drawn from Web of Science and Scopus, and a sub-sample of citing papers. RESULTS: Bibliometric data on 3419 papers citing 29 BEME reviews were analysed. More detailed data were extracted from a random sample of 629 full-text papers. DISCUSSION: BEME reviews were most often positioned to consolidate and summarize the current state of knowledge on a particular topic and to identify gaps in the literature; they were also used to justify current research, and less frequently to contextualize and explain results, or direct future areas of research. Their use to identify instruments or methodological approaches was relatively absent. CONCLUSION: While BEME reviews are primarily used to justify and support other studies, the current literature does not demonstrate their translation to educational practice.

PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation.

Scoping reviews, a type of knowledge synthesis, follow a systematic approach to map evidence on a topic and identify main concepts, theories, sources, and knowledge gaps. Although more scoping reviews are being done, their methodological and reporting quality need improvement. This document presents the PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews) checklist and explanation. The checklist was developed by a 24-member expert panel and 2 research leads following published guidance from the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network. The final checklist contains 20 essential reporting items and 2 optional items. The authors provide a rationale and an example of good reporting for each item. The intent of the PRISMA-ScR is to help readers (including researchers, publishers, commissioners, policymakers, health care providers, guideline developers, and patients or consumers) develop a greater understanding of relevant terminology, core concepts, and key items to report for scoping reviews.

Digital storytelling in health professions education: a systematic review.

BACKGROUND: Digital stories are short videos that combine stand-alone and first-person narratives with multimedia. This systematic review examined the contexts and purposes for using digital storytelling in health professions education (HPE) as well as its impact on health professionals' learning and behaviours. METHODS: We focused on the results of HPE studies gleaned from a larger systematic review that explored digital storytelling in healthcare and HPE. In December 2016, we searched MEDLINE, EMBASE, PsycINFO, CINAHL, and ERIC. We included all English-language studies on digital storytelling that reported at least one outcome from Levels 2 (learning) or 3 (behaviour) of The New World Kirkpatrick Model. Two reviewers independently screened articles for inclusion and extracted data. RESULTS: The comprehensive search (i.e., digital storytelling in healthcare and HPE) resulted in 1486 unique titles/abstracts. Of these, 153 were eligible for full review and 42 pertained to HPE. Sixteen HPE articles were suitable for data extraction; 14 focused on health professionals' learning and two investigated health professionals' learning as well as their behaviour changes. Half represented the undergraduate nursing context. The purposes for using digital storytelling were eclectic. The co-creation of patients' digital stories with health professionals as well as the creation and use of health professionals' own digital stories enhanced learning. Patients' digital stories alone had minimal impact on health professionals' learning. CONCLUSIONS: This review highlights the need for high-quality research on the impact of digital storytelling in HPE, especially on health professionals' behaviours. PROSPERO REGISTRATION NUMBER: CRD42016050271 .

Collaborative writing applications in healthcare: effects on professional practice and healthcare outcomes.

BACKGROUND: Collaborative writing applications (CWAs), such as wikis and Google Documents, hold the potential to improve the use of evidence in both public health and healthcare. Although a growing body of literature indicates that CWAs could have positive effects on healthcare, such as improved collaboration, behavioural change, learning, knowledge management, and adaptation of knowledge to local context, this has never been assessed systematically. Moreover, several questions regarding safety, reliability, and legal aspects exist. OBJECTIVES: The objectives of this review were to (1) assess the effects of the use of CWAs on process (including the behaviour of healthcare professionals) and patient outcomes, (2) critically appraise and summarise current evidence on the use of resources, costs, and cost-effectiveness associated with CWAs to improve professional practices and patient outcomes, and (3) explore the effects of different CWA features (e.g. open versus closed) and different implementation factors (e.g. the presence of a moderator) on process and patient outcomes. SEARCH METHODS: We searched CENTRAL, MEDLINE, Embase, and 11 other electronic databases. We searched the grey literature, two trial registries, CWA websites, individual journals, and conference proceedings. We also contacted authors and experts in the field. We did not apply date or language limits. We searched for published literature to August 2016, and grey literature to September 2015. SELECTION CRITERIA: We included randomised controlled trials (RCTs), non-randomised controlled trials (NRCTs), controlled before-and-after (CBA) studies, interrupted time series (ITS) studies, and repeated measures studies (RMS), in which CWAs were used as an intervention to improve the process of care, patient outcomes, or healthcare costs. DATA COLLECTION AND ANALYSIS: Teams of two review authors independently assessed the eligibility of studies. Disagreements were resolved by discussion, and when consensus was not reached, a third review author was consulted. MAIN RESULTS: We screened 11,993 studies identified from the electronic database searches and 346 studies from grey literature sources. We analysed the full text of 99 studies. None of the studies met the eligibility criteria; two potentially relevant studies are ongoing. AUTHORS' CONCLUSIONS: While there is a high number of published studies about CWAs, indicating that this is an active field of research, additional studies using rigorous experimental designs are needed to assess their impact and cost-effectiveness on process and patient outcomes.

Reporting quality and risk of bias in randomised trials in health professions education.

CONTEXT: Complete reporting of research is essential to enable consumers to accurately appraise, interpret and apply findings. Quality appraisal checklists are giving way to tools that judge the risk for bias. OBJECTIVES: We sought to determine the prevalence of these complementary aspects of research reports (completeness of reporting and perceived risk for bias) of randomised studies in health professions education. METHODS: We searched bibliographic databases for randomised studies of health professions education. We appraised two cohorts representing different time periods (2008-2010 and 2014, respectively) and worked in duplicate to apply the CONSORT guidelines and Cochrane Risk of Bias tool. We explored differences between time periods using independent-samples t-tests or the chi-squared test, as appropriate. RESULTS: We systematically identified 180 randomised studies (2008-2010, n = 150; 2014, n = 30). Frequencies of reporting of CONSORT elements within full-text reports were highly variable and most elements were reported in fewer than 50% of studies. We found a statistically significant difference in the CONSORT reporting index (maximum score: 500) between the 2008-2010 (mean ± standard deviation [SD]: 242.7 ± 55.6) and 2014 (mean ± SD: 311.6 ± 53.2) cohorts (p < 0.001). High or unclear risk for bias was most common for allocation concealment (157, 87%) and blinding of participants (147, 82%), personnel (152, 84%) and outcome assessors (112, 62%). Most risk for bias elements were judged to be unclear (range: 51-84%). Risk for bias elements significantly improved over time for blinding of participants (p = 0.007), incomplete data (p < 0.001) and the presence of other sources of bias (p < 0.001). CONCLUSIONS: Reports of randomised studies in health professions education frequently omit elements recommended by the CONSORT statement. Most reports were assessed as having a high or unclear risk for bias. Greater attention to how studies are reported at study outset and in manuscript preparation could improve levels of complete transparent reporting.

A BEME (Best Evidence in Medical Education) review of the use of workplace-based assessment in identifying and remediating underperformance among postgraduate medical trainees: BEME Guide No. 43.

INTRODUCTION: The extent to which workplace-based assessment (WBA) can be used as a facilitator of change among trainee doctors has not been established; this is particularly important in the case of underperforming trainees. The aim of this review is to examine the use of WBA in identifying and remediating performance among this cohort. METHODS: Following publication of a review protocol a comprehensive search of eight databases took place to identify relevant articles published prior to November 2015. All screening, data extraction and analysis procedures were performed in duplicate or with quality checks and necessary consensus methods throughout. Given the study-level heterogeneity, a descriptive synthesis approach informed the study analysis. RESULTS: Twenty studies met the inclusion criteria. The use of WBA within the context of remediation is not supported within the existing literature. The identification of underperformance is not supported by the use of stand-alone, single-assessor WBA events although specific areas of underperformance may be identified. Multisource feedback (MSF) tools may facilitate identification of underperformance. CONCLUSION: The extent to which WBA can be used to detect and manage underperformance in postgraduate trainees is unclear although evidence to date suggests that multirater assessments (i.e. MSF) may be of more use than single-rater judgments (e.g. mini-clinical evaluation exercise).

Productivity in medical education research: an examination of countries of origin.

BACKGROUND: Productivity and countries of origin of publications within the field of medical education research have not been explored. Using bibliometric techniques we conducted an analysis of studies evaluating medical education interventions, examining the country where research originated as well as networks of authors within countries identified as 'most productive'. METHODS: PubMed was used to search for evaluative studies of medical education. We then examined relative productivity of countries with >100 publications in our sample (number of publications/number of medical schools in country). Author networks from the top 2 countries with the highest relative productivity were constructed. RESULTS: 6874 publications from 18,883 different authors were included. The countries with the highest relative publication productivity were Canada (37.1), Netherlands (28.3), New Zealand (27), the UK (23), and the U.S.A (17.1). Author collaboration networks differed in both numbers of authors and intensity of collaborations in the countries with highest relative productivity. CONCLUSIONS: In terms of the number of publications of evaluative studies in medical education, Canada.

The h-index in medical education: an analysis of medical education journal editorial boards.

BACKGROUND: Disciplines differ in their authorship and citation practices, thus discipline-specific h-index norms are desirable. Thus the goal of this study was to examine the relationship between the h-index and academic rank in the field of medical education, and the differences in the h-index between MD's and PhD's in this field. METHODS: Due to the absence of a formalized registry of medical educators, we sampled available editorial board membership (considered a proxy for identifying 'career' medical educators) to establish h-index values. These were determined using Web of Science (WoS) and Google Scholar (GS), and internet searching was used to determine their academic rank. The correlation between authors' h-indices derived from WoS and GS was also determined. RESULTS: 130 editors were identified (95 full professors, 21 associate professors, 14 assistant professors). A significant difference was noted between the h-indices of full professors and associate/assistant professors (p < .001). Median h-indices equaled 14 for full professors (Interquartile range [IQR] =11); 7 for associate professors (IQR =7) and 6.5 for assistant professors (IQR = 8). h-indices of MD's and PhD's did not differ significantly. Moderate correlation between GS and WOS h-indices was noted R = 0.46, p < .001. CONCLUSIONS: The results provide some guidance as to the expected h-indices of a select group of medical educators. No differences appear to exist between assistant professor and associate professor ranks or between MD's and PhD's.

The use of virtual nominal groups in healthcare research: An extended scoping review.

INTRODUCTION: The Nominal Group Technique (NGT) is a consensus group method used to synthesize expert opinions. Given the global shift to virtual meetings, the extent to which researchers leveraged virtual platforms is unclear. This scoping review explores the use of the vNGT in healthcare research during the COVID-19 pandemic. METHODS: Following the Arksey and O'Malley's framework, eight cross-disciplinary databases were searched (January 2020-July 2022). Research articles that reported all four vNGT stages (idea generation, round robin sharing, clarification, voting) were included. Media Synchronicity Theory informed analysis. Corresponding authors were surveyed for additional information. RESULTS: Of 2,589 citations, 32 references were included. Articles covered healthcare (27/32) and healthcare education (4/32). Platforms used most were Zoom, MS Teams and GoTo but was not reported in 44% of studies. Only 22% commented on the benefits/challenges of moving the NGT virtually. Among authors who responded to our survey (16/32), 80% felt that the vNGT was comparable or superior. CONCLUSIONS: The vNGT provides several advantages such as the inclusion of geographically dispersed participants, scheduling flexibility and cost savings. It is a promising alternative to the traditional in-person meeting, but researchers should carefully describe modifications, potential limitations, and impact on results.

Motivations for investigating health inequities in observational epidemiology: a content analysis of 320 studies.

OBJECTIVES: To enhance equity in clinical and epidemiological research, it is crucial to understand researcher motivations for conducting equity-relevant studies. Therefore, we evaluated author motivations in a randomly selected sample of equity-relevant observational studies published during the COVID-19 pandemic. STUDY DESIGN AND SETTING: We searched MEDLINE for studies from 2020 to 2022, resulting in 16,828 references. We randomly selected 320 studies purposefully sampled across income setting (high vs low-middle-income), COVID-19 topic (vs non-COVID-19), and focus on populations experiencing inequities. Of those, 206 explicitly mentioned motivations which we analyzed thematically. We used discourse analysis to investigate the reasons behind emerging motivations. RESULTS: We identified the following motivations: (1) examining health disparities, (2) tackling social determinants to improve access, and (3) addressing knowledge gaps in health equity. Discourse analysis showed motivations stem from commitments to social justice and recognizing the importance of highlighting it in research. Other discourses included aspiring to improve health-care efficiency, wanting to understand cause-effect relationships, and seeking to contribute to an equitable evidence base. CONCLUSION: Understanding researchers' motivations for assessing health equity can aid in developing guidance that tailors to their needs. We will consider these motivations in developing and sharing equity guidance to better meet researchers' needs.