Data sharing and big data in health professions education: Ottawa consensus statement and recommendations for scholarship.

Changes in digital technology, increasing volume of data collection, and advances in methods have the potential to unleash the value of big data generated through the education of health professionals. Coupled with this potential are legitimate concerns about how data can be used or misused in ways that limit autonomy, equity, or harm stakeholders. This consensus statement is intended to address these issues by foregrounding the ethical imperatives for engaging with big data as well as the potential risks and challenges. Recognizing the wide and ever evolving scope of big data scholarship, we focus on foundational issues for framing and engaging in research. We ground our recommendations in the context of big data created through data sharing across and within the stages of the continuum of the education and training of health professionals. Ultimately, the goal of this statement is to support a culture of trust and quality for big data research to deliver on its promises for health professions education (HPE) and the health of society. Based on expert consensus and review of the literature, we report 19 recommendations in (1) framing scholarship and research through research, (2) considering unique ethical practices, (3) governance of data sharing collaborations that engage stakeholders, (4) data sharing processes best practices, (5) the importance of knowledge translation, and (6) advancing the quality of scholarship through multidisciplinary collaboration. The recommendations were modified and refined based on feedback from the 2022 Ottawa Conference attendees and subsequent public engagement. Adoption of these recommendations can help HPE scholars share data ethically and engage in high impact big data scholarship, which in turn can help the field meet the ultimate goal: high-quality education that leads to high-quality healthcare.

Validity evidence supporting clinical skills assessment by artificial intelligence compared with trained clinician raters.

BACKGROUND: Artificial intelligence (AI) is becoming increasingly used in medical education, but our understanding of the validity of AI-based assessments (AIBA) as compared with traditional clinical expert-based assessments (EBA) is limited. In this study, the authors aimed to compare and contrast the validity evidence for the assessment of a complex clinical skill based on scores generated from an AI and trained clinical experts, respectively. METHODS: The study was conducted between September 2020 to October 2022. The authors used Kane's validity framework to prioritise and organise their evidence according to the four inferences: scoring, generalisation, extrapolation and implications. The context of the study was chorionic villus sampling performed within the simulated setting. AIBA and EBA were used to evaluate performances of experts, intermediates and novice based on video recordings. The clinical experts used a scoring instrument developed in a previous international consensus study. The AI used convolutional neural networks for capturing features on video recordings, motion tracking and eye movements to arrive at a final composite score. RESULTS: A total of 45 individuals participated in the study (22 novices, 12 intermediates and 11 experts). The authors demonstrated validity evidence for scoring, generalisation, extrapolation and implications for both EBA and AIBA. The plausibility of assumptions related to scoring, evidence of reproducibility and relation to different training levels was examined. Issues relating to construct underrepresentation, lack of explainability, and threats to robustness were identified as potential weak links in the AIBA validity argument compared with the EBA validity argument. CONCLUSION: There were weak links in the use of AIBA compared with EBA, mainly in their representation of the underlying construct but also regarding their explainability and ability to transfer to other datasets. However, combining AI and clinical expert-based assessments may offer complementary benefits, which is a promising subject for future research.

Patient Engagement in Medical Trainee Selection: A Scoping Review.

PURPOSE: The stakes of medical trainee selection are high, making it ironic and somewhat paradoxical that patients and the public often get little say in selection practices. The authors sought to undertake a knowledge synthesis to uncover what is known about patient engagement across the medical trainee selection continuum. METHOD: The authors conducted a scoping review aimed at exploring the current state of practice and research on patient engagement in medical trainee selection in 2017-2021. MeSH headings and keywords were used to capture patient, community, and standardized patient engagement in selection processes across multiple health professions. The authors employed broad inclusion criteria and iteratively refined the corpus, ultimately, limiting study selection to those reporting engagement of actual patients in selection within medicine, but maintaining a broad focus on any patient contributions across the entire selection continuum. The Cambridge Framework was adapted and used to organize the included studies. RESULTS: In total, 2,858 abstracts were reviewed, and ultimately, 28 papers were included in the final corpus. The included studies were global but nascent. Most of the literature on this topic appears in the form of individual projects advocating for patient engagement in selection rather than cohesive programs with empirical exploration of patient engagement in selection. Job analysis methodology was particularly prominent for incorporating the patient voice into identifying competencies of relevance to selection. Direct patient engagement in early selection activities allowed the patient voice to assist candidates in determining their fit for medicine. CONCLUSIONS: Patient engagement has not been made a specific focus of study in its own right, leading the authors to encourage researchers to turn their lens more directly on patient engagement to explore how it complements the professional voice in medical trainee selection.

Inter-institutional data-driven education research: consensus values, principles, and recommendations to guide the ethical sharing of administrative education data in the Canadian medical education research context.

Background: Administrative data are generated when educating, licensing, and regulating future physicians but these data are rarely used beyond their pre-specified purposes. The capacity necessary for sensitive and responsive oversight that supports the sharing of administrative medical education data across institutions for research purposes needs to be developed. Method: A pan-Canadian consensus-building project was undertaken to develop agreement on the goals, benefits, risks, values, and principles that should underpin inter-institutional data-driven medical education research in Canada. A survey of key literature, consultations with various stakeholders and five successive knowledge synthesis workshops informed this project. Propositions were developed, driving subsequent discussions until collective agreement was distilled. Results: Consensus coalesced around six key principles: establishing clear purposes, rationale, and methodology for inter-institutional data-driven research a priori; informed consent from data generators in education systems is non-negotiable; multi-institutional data sharing requires special governance; data governance should be guided by data sovereignty; data use should be guided by an identified set of shared values; and best practices in research data-management should be applied. Conclusion: We recommend establishing a representative governance body, engaging trusted data facility, and adherence to extant data management policies when sharing administrative medical education data for research purposes in Canada.

Contexte: Des données administratives sont générées dans le cadre de la formation des médecins, d'octroi de permis d'exercice et de réglementation des activités professionnelles, mais ces données sont rarement utilisées au-delà de leurs objectifs prédéfinis. Il convient de créer un système de supervision réactif et sensible aux risques pour permettre le partage de données relatives à l'enseignement médical entre établissements à des fins de recherche. Méthode: Une initiative pancanadienne de recherche de consensus a été réalisée pour parvenir à un accord sur les objectifs, les avantages, les risques, les valeurs et les principes qui devraient sous-tendre la recherche interinstitutionnelle sur l'enseignement médical à l'aide des données existantes. Ce projet s'est appuyé sur une analyse de la littérature scientifique, sur des consultations avec diverses parties prenantes et sur cinq ateliers successifs de synthèse des connaissances. Des discussions ont été menées sur la base de propositions formulées préalablement jusqu'à la cristallisation d'un accord collectif. Résultats: Un consensus s'est dégagé autour de six principes clés : la création a priori d'objectifs, d'une logique et d'une méthodologie clairs pour la recherche interinstitutionnelle fondée sur les données; l'obtention, sans exception, du consentement éclairé des personnes concernées par la collecte de données dans les systèmes d'éducation; la création d'un cadre de gouvernance visant spécifiquement le partage des données entre établissements; le respect, dans ce cadre, de la souveraineté des données; l'utilisation des données fondée sur un ensemble de valeurs partagées; et l'application des meilleures pratiques en matière de gestion des données de recherche. Conclusion: En vue du partage des données administratives relatives à l'enseignement médical à des fins de recherche au Canada, nous recommandons la création d'une instance de gouvernance représentative ainsi que l'utilisation d'infrastructures fiables et le respect des politiques existantes régissant la gestion des données.

Timing's not everything: Immediate and delayed feedback are equally beneficial for performance in formative multiple-choice testing.

INTRODUCTION: Test-enhanced learning (TEL) is an impactful teaching and learning strategy that prioritises active learner engagement through the process of regular testing and reviewing. While it is clear that meaningful feedback optimises the effects of TEL, the ideal timing of this feedback (i.e. immediate or delayed) in a medical education setting is unclear. METHOD: Forty-one second-year medical students were recruited from the University of Melbourne. Participants were given a multiple-choice question test with a mix of immediate (i.e. post-item) and delayed (i.e. post-item-block) conceptual feedback. Students were then tested on near and far transfer items during an immediate post-test, and at a one-week follow-up. RESULTS: A logistic mixed effects model was used to predict the probability of successful near and far transfer. As expected, participants in our study tended to score lower on far transfer items than they did on near transfer items. In addition, correct initial response on a parent question predicted subsequent correct responding. Contrary to our hypotheses, the feedback timing effect was non-significant-there was no discernible difference between feedback delivered immediately versus delayed feedback. DISCUSSION: The findings of this study suggest that the timing of feedback delivery (post-item versus post-item-block) does not influence the efficacy of TEL in this medical education setting. We therefore suggest that educators may consider practical factors when determining appropriate TEL feedback timing in their setting.

Detour or New Direction: The Impact of the COVID-19 Pandemic on the Professional Identity Formation of Postgraduate Residents.

PURPOSE: The COVID-19 pandemic has resulted in numerous disruptions to health professions education training programs. Much attention has been given to the impact of these disruptions on formal learning opportunities in training; however, little attention has been given to the impact on professional socialization and professional identity formation. This study explored the impact of the pandemic and resultant curricular changes on the professional identity of family medicine residents. METHOD: 23 family medicine residents at the University of Toronto were interviewed between September 2020 and September 2022. Using symbolic interactionism as a theoretical framework, thematic analysis explored the meanings residents attributed to both experiences that were disrupted due to the pandemic, and new experiences that resulted from these disruptions. RESULTS: Participant responses reflected that disruptions in training did not always align with their expectations for family medicine and plans for future practice; however, these new experiences also reinforced their understanding of what it means to be a family physician. While participants felt the pandemic represented a loss of agency and negatively impacted relationships in their training program, it also provided a sense of belonging and membership in their profession. Finally, these new experiences continually blurred the line between professional and personal identities through the impact of the pandemic on participants' sense of well-being and safety. CONCLUSIONS: The impact of the pandemic on training experiences extends beyond the loss of formal learning opportunities. Participant responses reflect the collective influence of the formal, informal, and hidden curriculum on the professional socialization and professional identity formation of residents-and how these different curricular influences were disrupted due to the pandemic. These training experiences have important implications for the future practice of residents who completed their training during the pandemic and highlight the role of training programs in supporting the professional identity formation of residents.

Effectiveness of Life Goal Framing to Motivate Medical Students During Online Learning: A Randomized Controlled Trial.

Introduction: Educators need design strategies to support medical students' motivation in online environments. Prompting students to frame a learning activity as preparing them to attain their life goals (e.g., helping others) via their clinical practice, a strategy called 'life goal framing', may enhance their autonomous motivation, learning strategy use, and knowledge retention. However, for students with low perceived competence for learning (PCL), life goal framing may have an adverse effect. A randomized controlled trial was conducted to test the effectiveness of life goal framing and the moderating effect of students' PCL. Methods: First- and second-year medical students across four Canadian universities (n = 128) were randomized to receive a version of an online module with an embedded prompt for life goal framing, or one without. Students' motivation, learning strategy use, and knowledge retention were assessed. Differences between conditions on each outcome were estimated using Bayesian regression. Results: Students' PCL was a moderator for autonomous motivation but no other outcomes. The prompt did not have a statistically significant effect on any outcome, even for learners with high PCL, except for a small effect on link-clicking behaviour. Discussion: The results of this study suggest that learners' autonomous motivation is influenced by how they make meaning of instruction in terms of their future life goals and their present confidence. We cannot recommend life goal framing as an effective design strategy at this point, but we point to future work to increase the benefit of life goal framing for learners with high confidence.

Defining the capabilities and competencies of high-performing family physicians: a mixed methods study.

INTRODUCTION: High-performing primary care is recognised as the foundation of an effective and efficient healthcare system. Many medical graduates report they are not prepared for independent practice. To date, no research has been conducted to identify the key capabilities and competencies of high-performing family medicine graduates in Canada. This pilot project aims to identify the capabilities and competencies of high-performing early-career family physicians in Ontario, Canada, and explore opportunities for enhancing learning, teaching and assessment within family medicine residency programmes. METHODS AND ANALYSIS: Employing a mixed-methods explanatory sequential study design, this research will use a theory-driven Professional Capability Framework, previously validated in studies across nine professions, to guide the investigation. The first (quantitative) phase involves surveying ~50 high-performing early-career family physicians identified as high performing by educators, colleagues and leaders. The objective of the survey is to identify the key competencies and personal, interpersonal and cognitive capabilities of high-performing family physicians. The second (qualitative) phase involves conducting workshops with stakeholders, including educators, professional associations, regulators and colleges, to test the veracity of the results. Quantitative data will be analysed using descriptive statistics, and qualitative data will be analysed using Braun and Clarke's thematic analysis. The first and second phases will identify the key capabilities and competencies required to confidently adapt to the independent practice of comprehensive family medicine and inform fit-for-purpose educational strategies for teaching, learning and assessment. ETHICS AND DISSEMINATION: The study is approved by the University of Toronto's Health Sciences Research Ethics Board (#41799). Research findings will be discussed with professional bodies, educators responsible for family medicine curricula and universities. Study findings will also be disseminated through academic conferences and academic publications in peer-reviewed journals. Project summaries and infographics will be developed and disseminated to key stakeholders.

Six ways to maximize survey response rates: lessons from a medical school accreditation survey in a Canadian setting.

Background: Surveys are being increasingly used to gather feedback and study data in healthcare professions. However, it may be challenging to achieve high response rates in surveys administered to healthcare professionals. The aim of this paper is to report six strategies that contributed to a high response rate on the Independent Student Analysis at the University of Toronto (U of T), which can be applied to other surveys to achieve strong response rates amongst healthcare professionals. Methods: In 2019, as part of accreditation for the U of T MD Program, we conducted the Independent Student Analysis, a student-led survey examining a medical student's experience. We review and critically evaluate the factors that contributed to a robust response rate amongst one of the largest cohorts of medical students in Canada. Results: Among 1080 students in the MD program, we achieved an unprecedented response rate of 87.2%. Six factors were identified that most contributed to our high response rate, including: faculty support, student representation, eliciting participant feedback, creating protected time for completion, offering incentives, and generating awareness. Conclusions: Eliciting high survey response rates from medical learners can be challenging. However, with careful consideration of learner feedback and effective employment of the strategies discussed in this paper, medical school faculty may better engage students in survey completion, achieving higher response rates and gathering richer insight, which can be used to more effectively enact meaningful change amongst healthcare professionals.

Contexte: Les enquêtes auprès de professionnels de la santé sont de plus en plus utilisées pour recueillir et étudier des perspectives et des données, mais il peut s'avérer difficile d'obtenir des taux de réponse élevés. Cet article vise à présenter six stratégies qui ont permis de susciter une forte participation à l'enquête de l'Analyse indépendante des étudiants à l'Université de Toronto et qui peuvent être transposées à d'autres contextes de sondage auprès des professionnels du domaine.En 2019, dans le cadre du processus d'agrément du programme de doctorat en médecine de l'Université de Toronto, nous avons réalisé l'Analyse indépendante des étudiants, une enquête dirigée par les étudiants en médecine visant à examiner leur expérience. Nous passons en revue et évaluons de manière critique les facteurs qui ont contribué à l'atteinte d'un taux de réponse élevé auprès d'une des plus grandes cohortes d'étudiants en médecine au Canada. Résultats: Nous avons obtenu un taux de réponse sans précédent de 87,2 % parmi les 1 080 étudiants inscrits au programme de médecine. Les six facteurs qui ont le plus contribué à ce taux de réponse élevé sont : le soutien du corps professoral, la représentation des étudiants, leur participation à la conception de l'enquête, la création de plages horaires réservées pour remplir le questionnaire, l'offre d'incitatifs et la sensibilisation. Conclusions: Il peut être difficile de susciter une forte participation aux questionnaires chez les étudiants en médecine. Cependant, en considérant attentivement les commentaires des apprenants et en utilisant efficacement les stratégies présentées dans cet article, le corps professoral seront en mesure de mobiliser les étudiants à participer davantage aux enquêtes et de tirer profit de leur apport pour éclairer la promotion de changements pertinents chez les professionnels de la santé.

Impact of early waves of the COVID-19 pandemic on family medicine residency training: Analysis of survey data.

OBJECTIVE: To identify how graduating and incoming family medicine residents (FMR) experienced changes to their education during the early waves of the COVID-19 pandemic. DESIGN: The Family Medicine Longitudinal Survey was modified with questions related to the impact of COVID-19 on FMR and their training. Short-answer responses underwent thematic analysis. Responses to Likert scale and multiple-choice questions were reported as summary statistics. SETTING: Department of Family and Community Medicine at the University of Toronto in Ontario. PARTICIPANTS: Graduating FMR in spring 2020 and incoming FMR in fall 2020. MAIN OUTCOME MEASURES: Residents' perceptions of the impact of COVID-19 on clinical skills acquisition and preparedness for practice. RESULTS: Surveys response rates were 124 of 167 (74%) and 142 of 162 (88%) for graduating and incoming residents, respectively. Important themes for both cohorts included reduced access to clinical environments, reduced patient volumes, and lack of exposure to procedural skills. While the graduating cohort indicated they felt confident to begin practising family medicine, they described being impacted by the loss of a tailored learning environment, including canceled or altered electives. In contrast, incoming residents reported the loss of core skills, such as physical examination competency, as well as the loss of face-to-face communication, rapport, and relationship-building opportunities. However, both cohorts endorsed gaining new skills during the pandemic, including conducting telemedicine appointments, pandemic planning, and interfacing with public health. CONCLUSION: Based on these results, residency programs can specifically tailor solutions and modifications to address common themes across cohorts to facilitate optimal learning environments in pandemic times.

Admitting privileges: A construction ecology perspective on the unintended consequences of medical school admissions.

Medical-school applicants learn from many sources that they must stand out to fit in. Many construct self-presentations intended to appeal to medical-school admissions committees from the raw materials of work and volunteer experiences, in order to demonstrate that they will succeed in a demanding profession to which access is tightly controlled. Borrowing from the field of architecture the lens of construction ecology, which considers buildings in relation to the global effects of the resources required for their construction, we reframe medical-school admissions as a social phenomenon that has far-reaching harmful unintended consequences, not just for medicine but for the broader world. Illustrating with discussion of three common pathways to experiences that applicants widely believe will help them gain admission, we describe how the construction ecology of medical school admissions can recast privilege as merit, reinforce colonizing narratives, and lead to exploitation of people who are already disadvantaged.

Clinical Objectives of the Canadian Licensing Examination: Exploring the Representation of Oncology.

Every physician should be comfortable caring for cancer patients, yet medical education in oncology is known as inadequate. Our study explored this issue from an assessment perspective, by determining the representation of oncology and other health priorities in the clinical learning objectives for the Medical Council of Canada Qualifying Examination Part 1 (MCCQE Part 1). The Medical Council of Canada lists objectives for the MCCQE Part 1, based on CanMEDS roles for medical graduate competency. These objectives guide exam question development and can be used as a study guide for learners. Our study focused on the Medical Expert role and mapped the clinical objectives for oncologic, cardiovascular, cerebrovascular and chronic lower respiratory disease. The clinical objectives contained 190 topics under the Medical Expert role. Oncology content was found in 57 (30%), cardiovascular disease in 56 (29.5%), cerebrovascular disease in 21 (11%) and chronic lower respiratory disease in 7 (3.7%). In objectives with oncology content, gastrointestinal cancer (16, 28%), non-specific indicators of cancer (7, 12%) and genitourinary/musculoskeletal cancers (6, 10.5%) were most frequent. Content coding had inter-rater agreement greater than 99%, with kappas from 0.73 to 1.00. Oncology was highly represented in the clinical objectives listed for the MCCQE Part 1. With existing research showing limited oncology content on examinations and in medical curricula, our results were unexpected. This finding could signal the efficacy of initiatives seeking to improve oncology education and emphasizes the importance of continuing improvements in oncology education to meet patient and healthcare system needs.

Motivational Design for Web-Based Instruction in Health Professions Education: Protocol for a Systematic Review and Directed Content Analysis.

BACKGROUND: Web-based instruction plays an essential role in health professions education (HPE) by facilitating learners' interactions with educational content, teachers, peers, and patients when they would not be feasible in person. Within the unsupervised settings where web-based instruction is often delivered, learners must effectively self-regulate their learning to be successful. Effective self-regulation places heavy demands on learners' motivation, so effective web-based instruction must be designed to instigate and maintain learners' motivation to learn. Models of motivational design integrate theories of motivation with design strategies intended to create the conditions for motivated engagement. Teachers can use such models to develop their procedural and conceptual knowledge in ways that help them design motivating instruction in messy real-world contexts. Studies such as randomized controlled trials (RCTs) and other quasi-experimental designs that compare different motivational design strategies play a critical role in advancing models of motivational design. Synthesizing the evidence from those studies can identify effective strategies and help teachers and researchers understand the mechanisms governing why strategies work, for whom, and under what circumstances. OBJECTIVE: The planned review aims to analyze how studies comparing motivational design strategies for web-based instruction in HPE support and advance models of motivational design by (1) controlling for established risks to internal validity, (2) leveraging authentic educational contexts to afford ecological validity, (3) drawing on established theories of motivation, (4) investigating a wide breadth of motivational constructs, and (5) analyzing mediators and moderators of strategy effects. METHODS: The planned review will use database searching, registry searching, and hand searching to identify studies comparing motivational design strategies for web-based instruction, delivered to learners in HPE. Studies will be considered from 1990 onward. Two team members will independently screen studies and extract data from the included studies. During extraction, we will record information on the design characteristics of the studies, the theories of motivation they are informed by, the motivational constructs they target, and the mediators and moderators they consider. RESULTS: We have executed our database and registry searches and have begun screening titles and abstracts. CONCLUSIONS: By appraising the characteristics of studies that have focused on the motivational design of web-based instruction in HPE, the planned review will produce recommendations that will ensure impactful programs of future research in this crucial educational space. TRIAL REGISTRATION: PROSPERO CRD42022359521; https://tinyurl.com/57chuzf6. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/42681.

Beyond the tensions within transfer theories: implications for adaptive expertise in the health professions.

Ensuring trainees develop the flexibility with their knowledge to address novel problems, and to efficiently build upon prior knowledge to learn new knowledge is a common goal in health profession education. How trainees come to develop this capacity to transfer and transform knowledge across contexts can be described by adaptive expertise, which focuses on the ability of some experts to innovate upon their existing knowledge to develop novel solutions to novel problems. While adaptive expertise is often presented as an alternative framework to more traditional cognitivist and constructivist expertise models, it is unclear whether the non-routine and routine forms of transfer it describes are distinct from those described by other accounts of transfer. Furthermore, whether what (e.g., knowledge) is transferred and how (e.g., cognitive processes) differs between these views is still debated. In this review, we describe various theories of transfer and present a synthesis clarifying the relationship between transfer and adaptive expertise. Informed by our analysis, we argue that the mechanisms of transfer in adaptive expertise share important commonalities with traditional accounts of transfer, which when understood, can complement efforts by educators and researchers to foster and study adaptive expertise. We present three instructional principles that may better support transfer and adaptive expertise in trainees: i) identifying and incorporating meaningful variability in practice, ii) integrating conceptual knowledge during practice iii) using assessments of trainees' transfer. Taken together, we offer an integrative perspective to how educational systems and experiences can be designed to develop and encourage adaptive expertise and transfer.

Can I Ask a Question About URiM Awards That I Don't Know the Answer to? Designing an Award for Underrepresented Medical Education Researchers.

Meaningful Equity, Diversity, and Inclusion (EDI) efforts may be stymied by concerns about whether proposed initiatives are performative or tokenistic. The purpose of this project was to analyze discussions by the Research in Medical Education (RIME) Program Planning committee about how best to recognize and support underrepresented in medicine (URiM) researchers in medical education to generate lessons learned that might inform local, national, and international actions to implement meaningful EDI initiatives. Ten RIME Program Planning Committee members and administrative staff participated in a focus group held virtually in August 2021. Focus group questions elicited opinions about "if and how" to establish a URiM research award. The focus group was recorded, transcribed, and thematically analyzed. Recognition of privilege, including who has it and who doesn't, underpinned the focus group discussion, which revolved around 2 themes: (1) tensions between optics and semantics, and (2) potential unintended consequences of trying to level the medical education playing field. The overarching storyline threaded throughout the focus group discussion was intentionality. Focus group participants sought to avoid performativity by creating an award that would be meaningful to recipients and to career gatekeepers such as department chairs and promotion and tenure committees. Ultimately, participants decided to create an award that focused on exemplary Equity, Diversity, and Inclusion (EDI) scholarship, which was eventually named the "RIME URiM Research Award." Difficult but productive conversations about EDI initiatives are necessary to advance underrepresented in medicine (URiM) scholarship. This transparent commentary may trigger further critical conversations.

Primary Care Severe Asthma Registry and Education Project (PCSAR-EDU): Phase 1 - an e-Delphi for registry definitions and indices of clinician behaviour.

INTRODUCTION: Although most asthma is mild to moderate, severe asthma accounts for disproportionate personal and societal costs. Poor co-ordination of care between primary care and specialist settings is recognised as a barrier to achieving optimal outcomes. The Primary Care Severe Asthma Registry and Education (PCSAR-EDU) project aims to address these gaps through the interdisciplinary development and evaluation of both a 'real-world' severe asthma registry and an educational programme for primary care providers. This manuscript describes phase 1 of PCSAR-EDU which involves establishing interdisciplinary consensus on criteria for the: (1) definition of severe asthma; (2) generation of a severe asthma registry and (3) definition of an electronic-medical record data-based Clinician Behaviour Index (CBI). METHODS AND ANALYSIS: In phase 1, a modified e-Delphi activity will be conducted. Delphi panellists (n≥13) will be invited to complete a 30 min online survey on three separate occasions (i.e., three separate e-Delphi 'rounds') over a 3-month period. Expert opinion will be collected via an open-ended survey ('Open' round 1) and 5-point Likert scale and ranking surveys ('Closed' round 2 and 3). A fourth and final Delphi round will occur via synchronous meeting, whereby panellists approve a finalised ideal 'core criteria list', CBI and corresponding item weighting. ETHICS AND DISSEMINATION: Ethical approval has been obtained for the activities involved in phase 1 from the University of Toronto's Human Research Ethics Programme (approval number 39695). Future ethics approvals will depend on information gathered in the proceeding phase; thus, ethical approval for phase 2 and 3 of this study will be sought sequentially. Findings will be disseminated through conference presentations, peer-reviewed publications and knowledge translation tools.

Does group size matter during collaborative skills learning? A randomised study.

BACKGROUND: Collaborative skills learning in the form of dyad learning compared with individual learning has been shown to lead to non-inferior skills retention and transfer. However, we have limited knowledge on which learning activities improve collaborative skills training and how the number of collaborators may impact skills transfer. We explored the effects of skills training individually, in dyads, triads or tetrads on learning activities during training and on subsequent skills transfer. METHODS: In a randomised, controlled study, participants completed a pre-post-transfer-test set-up in groups of one to four. Participants completed 2 hours of obstetric ultrasound training. In the dyad, triad and tetrad group participants took turns actively handling the ultrasound probe. All performances were rated by two blinded experts using the Objective Structured Assessment of Ultrasound Skills (OSAUS) scale and a Global Rating Scale (GRS). All training was video recorded, and learning activities were analysed using the Interactive-Constructive-Active-Passive (ICAP) framework. RESULTS: One hundred one participants completed the simulation-based training, and ninety-seven completed the transfer test. Performance scores improved significantly from pre- to post-test for all groups (p < 0.001, ηp2 = 0.55). However, group size did not affect transfer test performance on OSAUS scores (p = 0.13, ηp2 = 0.06) or GRS scores (p = 0.23, ηp2 = 0.05). ICAP analyses of training activities showed that time spent on non-learning and passive learning activities increased with group size (p < 0.001, ηp2 = 0.31), whereas time spent on constructive and interactive learning activities was constant between groups compared with singles (p < 0.001, ηp2 = 0.72). CONCLUSION: Collaborative skills learning in groups of up to four did not impair skills transfer despite less hands-on time. This may be explained by a compensatory shift towards constructive and interactive learning activities that outweigh the effect of shorter hands-on time.

Toward 'seeing' critically: a Bayesian analysis of the impacts of a critical pedagogy.

Critical reflection supports enactment of the social roles of care, like collaboration and advocacy. We require evidence that links critical teaching approaches to future critically reflective practice. We thus asked: does a theory-informed approach to teaching critical reflection influence what learners talk about (i.e. topics of discussion) and how they talk (i.e. whether they talk in critically reflective ways) during subsequent learning experiences? Pre-clinical students (n = 75) were randomized into control and intervention conditions (8 groups each, of up to 5 interprofessional students). Participants completed an online Social Determinants of Health (SDoH) module, followed by either: a SDoH discussion (control) or critically reflective dialogue (intervention). Participants then experienced a common learning session (homecare curriculum and debrief) as outcome assessment, and another similar session one-week later. Blinded coders coded transcripts for what (topics) was said and how (critically reflective or not). We constructed Bayesian regression models for the probability of meaning units (unique utterances) being coded as particular what codes and as critically reflective or not (how). Groups exposed to the intervention were more likely, in a subsequent learning experience, to talk in a critically reflective manner (how) (0.096 [0.04, 0.15]) about similar content (no meaningful differences in what was said). This difference waned at one-week follow up. We showed experimentally that a particular critical pedagogical approach can make learners' subsequent talk, ways of seeing, more critically reflective even when talking about similar topics. This study offers the field important new options for studying historically challenging-to-evaluate impacts and supports theoretical assertions about the potential of critical pedagogies.

Beyond the ratings: gender effects in written comments from clinical teaching assessments.

Assessment of clinical teachers by learners is problematic. Construct-irrelevant factors influence ratings, and women teachers often receive lower ratings than men. However, most studies focus only on numeric scores. Therefore, the authors analyzed written comments on 4032 teacher assessments, representing 282 women and 448 men teachers in one Department of Medicine, to explore for gender differences. NVivo was used to search for 61 evidence- and theoretically-based terms purported to reflect teaching excellence, which were analyzed using 2 × 2 chi-squared tests. The Linguistic Index and Word Count (LIWC) was used to categorize comment data, which were analyzed using linear regressions. The only significant difference in NVivo was that men were more likely than women to have the word "available" in a comment (OR 1.4, p < .05). A subset of LIWC variables showed significant gender differences, but all effects were modest. Men teachers had more positive emotion words written about them, while negative emotion words appeared equally. Significant differences occurred more often between the men and women residents who wrote the comments, rather than those attributed to the gender of the teachers. For example, women residents used more social and gender-related words (ß 1.87, p < 0.001) and fewer words related to power or achievement (ß -3.78, p < 0.001) than men residents. Profound gender differences were not found in teacher assessment comments in this large, diverse academic department of medicine, which differs from other studies. The authors explore possible reasons including differences in departmental culture and issues related to the methods used.

The system, the resident, and the preceptor: a curricular approach to continuity of care training.

BACKGROUND: Continuity of care (CoC) is integral to the practice of comprehensive primary care, yet research in the area of CoC training in residency programs is limited. In light of distributed medical education and evolving accreditation standards, a rigorous understanding of the context and enablers contributing to CoC education must be considered in the design and delivery of residency training programs. APPROACH: At our preceptor-based community academic site, we developed a system-resident-preceptor (SRP) framework to explore factors that influence a resident's perception regarding CoC, and established variables in each area to enhance learning. We then implemented a two-year educational SRP intervention (SRPI) to one cohort of residents and their preceptors to integrate critical education factors and align teaching of continuity of care within curricular goals. EVALUATION: Evaluation of the intervention was based on resident interviews and faculty focus groups, and a qualitative phenomenological approach was used to analyze the data. While some factors identified are inherent to family medicine, the opportunity for reflection is a unique component to inculcate CoC learning. REFLECTION: The SRP innovation provides a unique framework to facilitate residents' understanding and development of CoC competency. Our model can be applied to all residency programs, including traditional academic sites as well as distributed training sites, to enhance CoC education.