Synthesis and perspectives from the Ottawa 2022 conference on the assessment of competence.

INTRODUCTION: The Ottawa Conference on the Assessment of Competence in Medicine and the Healthcare Professions was first convened in 1985 in Ottawa. Since then, what has become known as the Ottawa conference has been held in various locations around the world every 2 years. It has become an important conference for the community of assessment - including researchers, educators, administrators and leaders - to share contemporary knowledge and develop international standards for assessment in medical and health professions education. METHODS: The Ottawa 2022 conference was held in Lyon, France, in conjunction with the AMEE 2022 conference. A diverse group of international assessment experts were invited to present a symposium at the AMEE conference to summarise key concepts from the Ottawa conference. This paper was developed from that symposium. RESULTS AND DISCUSSION: This paper summarises key themes and issues that emerged from the Ottawa 2022 conference. It highlights the importance of the consensus statements and discusses challenges for assessment such as issues of equity, diversity, and inclusion, shifts in emphasis to systems of assessment, implications of 'big data' and analytics, and challenges to ensure published research and practice are based on contemporary theories and concepts.

Early stages of learning in interprofessional education: stepping towards collective competence for healthcare teams.

BACKGROUND: Interprofessional education (IPE) is a core element of many health professional education curricula. To date the focus of much research has been on student perceptions of, and attitudes towards, the learning experience. Little is known about the impact of early IPE experience on how students understand and learn about effective interprofessional teamwork. METHODS: This qualitative study involved first year university students enrolled in health professions degrees and investigated their descriptions of interprofessional teamwork through graphic elicitation and interviews. Participants were enrolled in a large-scale interprofessional unit (subject) in the university. RESULTS: The data were analysed through the lens of a tool that classifies dimensions of interprofessional activity. The findings indicated the majority of students had what was classified as a Stage 1 (or 'nascent') understanding of integration between work practices and a Stage 2 (or 'emerging') understanding of the dimensions of interprofessional teamwork which were commitment, identity, goals, roles and responsibilities, and interdependence. CONCLUSIONS: Based on the findings, the stages for a learning trajectory for interprofessional education are proposed and each stage is mapped to dimensions of interprofessional activity. A number of pedagogical strategies are suggested in order to move students through this two-stage model of learning and ensure their readiness for interprofessional teamwork as health professionals.

Performance assessment: Consensus statement and recommendations from the 2020 Ottawa Conference.

INTRODUCTION: In 2011 the Consensus Statement on Performance Assessment was published in Medical Teacher. That paper was commissioned by AMEE (Association for Medical Education in Europe) as part of the series of Consensus Statements following the 2010 Ottawa Conference. In 2019, it was recommended that a working group be reconvened to review and consider developments in performance assessment since the 2011 publication. METHODS: Following review of the original recommendations in the 2011 paper and shifts in the field across the past 10 years, the group identified areas of consensus and yet to be resolved issues for performance assessment. RESULTS AND DISCUSSION: This paper addresses developments in performance assessment since 2011, reiterates relevant aspects of the 2011 paper, and summarises contemporary best practice recommendations for OSCEs and WBAs, fit-for-purpose methods for performance assessment in the health professions.

Medical student wellbeing - a consensus statement from Australia and New Zealand.

BACKGROUND: Medical student wellbeing - a consensus statement from Australia and New Zealand outlines recommendations for optimising medical student wellbeing within medical schools in our region. Worldwide, medical schools have responsibilities to respond to concerns about student psychological, social and physical wellbeing, but guidance for medical schools is limited. To address this gap, this statement clarifies key concepts and issues related to wellbeing and provides recommendations for educational program design to promote both learning and student wellbeing. The recommendations focus on student selection; learning, teaching and assessment; learning environment; and staff development. Examples of educational initiatives from the evidence-base are provided, emphasising proactive and preventive approaches to student wellbeing. MAIN RECOMMENDATIONS: The consensus statement provides specific recommendations for medical schools to consider at all stages of program design and implementation. These are: Design curricula that promote peer support and progressive levels of challenge to students. Employ strategies to promote positive outcomes from stress and to help others in need. Design assessment tasks to foster wellbeing as well as learning. Provide mental health promotion and suicide prevention initiatives. Provide physical health promotion initiatives. Ensure safe and health-promoting cultures for learning in on-campus and clinical settings. Train staff on student wellbeing and how to manage wellbeing concerns. CONCLUSION: A broad integrated approach to improving student wellbeing within medical school programs is recommended. Medical schools should work cooperatively with student and trainee groups, and partner with clinical services and other training bodies to foster safe practices and cultures. Initiatives should aim to assist students to develop adaptive responses to stressful situations so that graduates are prepared for the realities of the workplace. Multi-institutional, longitudinal collaborative research in Australia and New Zealand is needed to close critical gaps in the evidence needed by medical schools in our region.

Teaching medical humanities in the digital world: affordances of technology-enhanced learning.

Medical humanities courses are typically taught in face-to-face teaching environments, but now medical humanities educators, alongside educators from other disciplines, are facing shifts in higher education towards online (and sometimes open) courses. For the medical humanities educator, there is limited guidance regarding how technology-enhanced learning design can support the learning outcomes associated with medical humanities. This article aims to provide useful direction for such educators on how digital technologies can be used through learner-focused pedagogies. Specific examples are provided as to how the affordances of Web 2.0 and other tools can be realised in innovative ways to help achieve skills development within the medical humanities. The guidance, alongside the practical suggestions for implementation, can provide important conceptual background for medical humanities educators who wish to embrace technology-enhanced learning, and reconceptualise or redesign medical humanities for an online or blended teaching environment.

Supporting validity: steps to contextualise applications for technology and assessment, for learning.

In the paper, the authors offer perspectives on the uses of technology and assessment, that support learning. The perspectives are viewed through validity (from the field of assessment) as a framework and they discuss four aspects of an interconnected technology, learning and assessment space that represent theory informed, authentic practice. The four are: 1) integrated coherence for learning, assessment and technology; 2) responsibilities for equity, diversity, inclusion and wellbeing; 3) sustainability; and 4) balancing resources in global contexts. The authors propose steps and considerations for medical and health professions educators who need to contextualise applications for technology, learning and assessment, for positive impact for learners, faculty, institutions and patient care.

Outcomes, Measurement Instruments, and Their Validity Evidence in Randomized Controlled Trials on Virtual, Augmented, and Mixed Reality in Undergraduate Medical Education: Systematic Mapping Review.

BACKGROUND: Extended reality, which encompasses virtual reality (VR), augmented reality (AR), and mixed reality (MR), is increasingly used in medical education. Studies assessing the effectiveness of these new educational modalities should measure relevant outcomes using outcome measurement tools with validity evidence. OBJECTIVE: Our aim is to determine the choice of outcomes, measurement instruments, and the use of measurement instruments with validity evidence in randomized controlled trials (RCTs) on the effectiveness of VR, AR, and MR in medical student education. METHODS: We conducted a systematic mapping review. We searched 7 major bibliographic databases from January 1990 to April 2020, and 2 reviewers screened the citations and extracted data independently from the included studies. We report our findings in line with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. RESULTS: Of the 126 retrieved RCTs, 115 (91.3%) were on VR and 11 (8.7%) were on AR. No RCT on MR in medical student education was found. Of the 115 studies on VR, 64 (55.6%) were on VR simulators, 30 (26.1%) on screen-based VR, 9 (7.8%) on VR patient simulations, and 12 (10.4%) on VR serious games. Most studies reported only a single outcome and immediate postintervention assessment data. Skills outcome was the most common outcome reported in studies on VR simulators (97%), VR patient simulations (100%), and AR (73%). Knowledge was the most common outcome reported in studies on screen-based VR (80%) and VR serious games (58%). Less common outcomes included participants' attitudes, satisfaction, cognitive or mental load, learning efficacy, engagement or self-efficacy beliefs, emotional state, competency developed, and patient outcomes. At least one form of validity evidence was found in approximately half of the studies on VR simulators (55%), VR patient simulations (56%), VR serious games (58%), and AR (55%) and in a quarter of the studies on screen-based VR (27%). Most studies used assessment methods that were implemented in a nondigital format, such as paper-based written exercises or in-person assessments where examiners observed performance (72%). CONCLUSIONS: RCTs on VR and AR in medical education report a restricted range of outcomes, mostly skills and knowledge. The studies largely report immediate postintervention outcome data and use assessment methods that are in a nondigital format. Future RCTs should include a broader set of outcomes, report on the validity evidence of the measurement instruments used, and explore the use of assessments that are implemented digitally.

Conducting a high-stakes OSCE in a COVID-19 environment.

This article was migrated. The article was marked as recommended. The COVID-19 pandemic has presented significant challenges for medical schools. It is critical to ensure final year medical school students are not delayed in their entry to the clinical workforce in times of healthcare crisis. However, proceeding with assessment to determine competency for graduation from medical school, and maintaining performance standards for graduating doctors is an unprecedented challenge under pandemic conditions. This challenge is hitherto uncharted territory for medical schools and there is scant guidance for medical educators. In early March 2020, Duke-National University Singapore Medical School embraced the challenge for ensuring competent final year medical students could complete their final year of studies and graduate on time, to enter the medical workforce in Singapore without delay. This paper provides details of how the final year clinical performance examinations were planned and conducted during the COVID-19 pandemic. The aim of the paper is to provide guidance to other medical schools in similar circumstances who need to plan and make suitable adjustments to clinical skills examinations under current pandemic conditions. The paper illustrates how it is possible to design and implement clinical skills examinations (OSCEs) to ensure the validity and reliability of high-stakes performance assessments whilst protecting the safety of all participants, minimising risk and maintaining defensibility to key stakeholders.

Choice of outcomes and measurement instruments in randomised trials on eLearning in medical education: a systematic mapping review protocol.

BACKGROUND: There will be a lack of 18 million healthcare workers by 2030. Multiplying the number of well-trained healthcare workers through innovative ways such as eLearning is highly recommended in solving this shortage. However, high heterogeneity of learning outcomes in eLearning systematic reviews reveals a lack of consistency and agreement on core learning outcomes in eLearning for medical education. In addition, there seems to be a lack of validity evidence for measurement instruments used in these trials. This undermines the credibility of these outcome measures and affects the ability to draw accurate and meaningful conclusions. The aim of this research is to address this issue by determining the choice of outcomes, measurement instruments and the prevalence of measurement instruments with validity evidence in randomised trials on eLearning for pre-registration medical education. METHODS: We will conduct a systematic mapping and review to identify the types of outcomes, the kinds of measurement instruments and the prevalence of validity evidence among measurement instruments in eLearning randomised controlled trials (RCTs) in pre-registration medical education. The search period will be from January 1990 until August 2017. We will consider studies on eLearning for health professionals' education. Two reviewers will extract and manage data independently from the included studies. Data will be analysed and synthesised according to the aim of the review. DISCUSSION: Appropriate choice of outcomes and measurement tools is essential for ensuring high-quality research in the field of eLearning and eHealth. The results of this study could have positive implications for other eHealth interventions, including (1) improving quality and credibility of eLearning research, (2) enhancing the quality of digital medical education and (3) informing researchers, academics and curriculum developers about the types of outcomes and validity evidence for measurement instruments used in eLearning studies. The protocol aspires to assist in the advancement of the eLearning research field as well as in the development of high-quality healthcare professionals' digital education. SYSTEMATIC REVIEW REGISTRATION: PROSPERO CRD42017068427.

Continuing Professional Development - reflections from nursing and education.

The tradition of engaging in reflective practice for learning is firmly entrenched in the discourse of nursing. This is also the situation for the field of education. This commonality provided the stimulus for this paper. The purpose is to provide an account of the value of reflection as a tool for critical thought in relation to learning through a case study of two Continuing Professional Development (CPD) programmes in different professional contexts (nursing and teaching) and geographical locations (the United Kingdom and Singapore). The authors, who are educators of teaching and nursing practitioners respectively, exchanged narratives of professional experience in order to frame reflections on practice of CPD programmes. They problematised what it means for nurses and teachers to learn, and how this learning might be judged by both the participants and the facilitators of CPD programmes. Adopting a critical and reflexive approach to practice supported the authors' professional learning. One common issue evident from the reflections of practice raised questions in relation to how participants might provide evaluative feedback of CPD programmes in socially desirable ways. This was foregrounded as an important issue to be considered by the authors when designing and facilitating CPD programmes and suggests implications for other educators of nursing and teaching practitioners.