Managing assessment during curriculum change: Ottawa Consensus Statement.

Curriculum change is relatively frequent in health professional education. Formal, planned curriculum review must be conducted periodically to incorporate new knowledge and skills, changing teaching and learning methods or changing roles and expectations of graduates. Unplanned curriculum evolution arguably happens continually, usually taking the form of "minor" changes that in combination over time may produce a substantially different programme. However, reviewing assessment practices is less likely to be a major consideration during curriculum change, overlooking the potential for unintended consequences for learning. This includes potentially undermining or negating the impact of even well-designed and important curriculum changes. Changes to any component of the curriculum "ecosystem "- graduate outcomes, content, delivery or assessment of learning - should trigger an automatic review of the whole ecosystem to maintain constructive alignment. Consideration of potential impact on assessment is essential to support curriculum change. Powerful contextual drivers of a curriculum include national examinations and programme accreditation, so each assessment programme sits within its own external context. Internal drivers are also important, such as adoption of new learning technologies and learning preferences of students and faculty. Achieving optimal and sustainable outcomes from a curriculum review requires strong governance and support, stakeholder engagement, curriculum and assessment expertise and internal quality assurance processes. This consensus paper provides guidance on managing assessment during curriculum change, building on evidence and the contributions of previous consensus papers.

Is ChatGPT 'ready' to be a learning tool for medical undergraduates and will it perform equally in different subjects? Comparative study of ChatGPT performance in tutorial and case-based learning questions in physiology and biochemistry.

PURPOSE: Generative AI will become an integral part of education in future. The potential of this technology in different disciplines should be identified to promote effective adoption. This study evaluated the performance of ChatGPT in tutorial and case-based learning questions in physiology and biochemistry for medical undergraduates. Our study mainly focused on the performance of GPT-3.5 version while a subgroup was comparatively assessed on GPT-3.5 and GPT-4 performances. MATERIALS AND METHODS: Answers were generated in GPT-3.5 for 44 modified essay questions (MEQs) in physiology and 43 MEQs in biochemistry. Each answer was graded by two independent examiners. Subsequently, a subset of 15 questions from each subject were selected to represent different score categories of the GPT-3.5 answers; responses were generated in GPT-4, and graded. RESULTS: The mean score for physiology answers was 74.7 (SD 25.96). GPT-3.5 demonstrated a statistically significant (p = .009) superior performance in lower-order questions of Bloom's taxonomy in comparison to higher-order questions. Deficiencies in the application of physiological principles in clinical context were noted as a drawback. Scores in biochemistry were relatively lower with a mean score of 59.3 (SD 26.9) for GPT-3.5. There was no statistically significant difference in the scores for higher and lower-order questions of Bloom's taxonomy. The deficiencies highlighted were lack of in-depth explanations and precision. The subset of questions where the GPT-4 and GPT-3.5 were compared demonstrated a better overall performance in GPT-4 responses in both subjects. This difference between the GPT-3.5 and GPT-4 performance was statistically significant in biochemistry but not in physiology. CONCLUSIONS: The differences in performance across the two versions, GPT-3.5 and GPT-4 across the disciplines are noteworthy. Educators and students should understand the strengths and limitations of this technology in different fields to effectively integrate this technology into teaching and learning.

Empathy in health professions education: What works, gaps and areas for improvement.

INTRODUCTION: Developing a physician equipped with both technical and affective skills is crucial in ensuring quality patient care. Of these skills, nurturing empathy is a key skill that has been studied in great detail, particularly among medical undergraduates. Despite numerous studies trending the changes in empathy, the results are often contradictory or confusing. Our study aims to find what interventions are effective to inculcate empathy in both undergraduate and postgraduate medical education and suggest areas for improvement. METHODS: A narrative review was conducted on the interventions in nurturing empathy in undergraduate and postgraduate medical education. Original research articles and systematic reviews with clear interventions and outcomes were included in the study. RESULTS: A total of 44 articles were reviewed. About 44% (n = 18) of the studies used a mixture of different approaches as their interventions. Some interventions were anchored by specific topics: Seven papers focusing on communication skills, three papers on humanities, and three on arts. A majority of the interventions (60%; n = 26) were implemented over a span of time as compared to studies which suggested a one-off intervention (30%; n = 12). Of the 26 papers in which the interventions were enforced over a period of time, 62% (n = 16) indicated an increase in student empathy whereas 16% (n = 4) indicated no changes in empathy post-intervention. On the contrary, 50% (n = 6) of the one-off interventions revealed no significant change in student empathy. Jefferson Scale of Empathy (JSE) is widely used in measuring student empathy postintervention, but approximately 41% of the studies included measuring tools other than JSE. CONCLUSIONS: Pedagogical methods that invoke thought processes related to the affective domain of learning and experiential learning are more effective than the didactic methods of teaching and learning. Multimodal mixed-methods approach that combine different pedagogical interventions is more likely to bring about the desired results.

Training infectious diseases senior residents during COVID-19: The impact and the lessons learnt.

BACKGROUND: The COVID-19 pandemic had major impact on the training of Infectious Diseases (ID) residents across the globe. They were part of the frontline staff, while at the same time training to be ID physicians. This study focused on identifying their capability, i.e. the ability to adapt existing competencies to new situations, which is now recognised as an essential element of professional practice. AIM: This study explored what ID residents learnt and how they learnt as they adapted to working in this unpredictable and challenging COVID-19 pandemic. METHODS: This qualitative explorative study was based in the Infectious Diseases Senior Residency Programme across three training institutions in Singapore. Individual semi-structured interviews were conducted. Data were analysed using a template analysis technique. RESULTS: Nine ID residents participated in this study. They learnt to engage with uncertainty in a meaningful way by relying on prior training and rapidly learning how to most effectively learn (metacognition). Learning was enhanced by collaboration between multidisciplinary health professionals, strong leadership and intrinsic motivation from personal interest in ID. They learnt through observing how senior faculty approached and managed the COVID-19 situation. CONCLUSION: When learning for future capability in a rapidly evolving situation, role-modelling and mentoring are essential as available information resources may still not provide the learning from skilled doctors with actual experiences managing complex, uncertain situations.

Mentorship in health professions education - an AMEE guide for mentors and mentees: AMEE Guide No. 167.

This AMEE guide discusses theoretical principles and practical strategies for health professions educators to promote impactful mentoring relationships. Traditional definitions are challenged, distinctions are made between roles such as mentor, advisor, coach and sponsor. As educational environments change and options for professional development expand, we argue that the traditional dyadic format of mentoring alone will not help mentees to maximise their professional growth. Newer formats of mentoring are discussed in detail and their advantages and disadvantages compared. We use a variety of theoretical concepts to anchor the practice of mentorship: self-focussed and other-focussed motives; psychological safety; personal interpretive framework; Daloz model for balancing support and challenge; zone of proximal development; communities of practice; and development along multiple layers of competence. Recommended strategies for effective mentoring are based on extensive review of literature, as well as combined professional mentoring experiences of the authors. We use key principles from the theories described and phases of mentoring relationships as foundations for the suggested best practices of mentorship. Finally, we emphasise the role of mentees in their own professional development and provide tips for them on seeking mentors, expanding their mentoring network and taking the lead in setting the agenda during mentoring meetings and formulating action plans for their own advancement.

Entrustable professional activities in undergraduate dental education: A practical model for development and validation.

OBJECTIVE: Entrustable professional activities (EPAs) are tasks that a person who is qualified or is in the process to be credentialed are allowed to engage. There are several levels of entrustment based on degrees of supervision assigned to each EPA. This paper aims to describe the process and outcome of creating EPAs; validate EPAs relevant to undergraduate dental training. METHODS: A draft set of EPA statements was developed based on the consensus of an expert panel. These were then mapped to the nationally determined minimum experience thresholds (clinical and procedural experiences/competencies) and aligned to task-based instructional strategy. The EPAs were validated to improve the relevance by using a criterion-based rubric. RESULTS: An end-to-end process workflow led to the development of an EPA-based educational framework to bridge the gaps in the curriculum. The process identified a total of 41 EPAs and out of which, 10 EPAs were notated as core EPAs and will be subjected to structured workplace-based assessment complying to the national standards. The validation exercise rated core EPAs with an overall score matching close to the cut-off of 4.07 (Equal rubric). CONCLUSION: The end-to-end process workflow provided the opportunity to elaborate a structured process for the development of EPAs for undergraduate dental education. As validation is a continuous process, feedback from implementation will inform the next steps.

Professional identity formation of medical teachers in a non-Western setting.

INTRODUCTION: Understanding and supporting professional identity formation (PIF) among medical teachers has become increasingly important in faculty development programs. In this study, we explored medical teachers' PIF in Indonesia. METHODS: We conducted a qualitative descriptive study using focus group discussions (FGDs) at four medical schools in Indonesia. Basic science and clinical teachers from four different schools were selected to participate. Data were transcribed, coded, and analysed to develop themes and subthemes. RESULTS: Seventeen FGDs were completed, involving 60 basic science and 59 clinical teachers. Four major themes regarding the formation of medical teachers' professional identity emerged: an internal dialogue between intrinsic values and external influences, empowerment through early socialization, experiential workplace learning, and envisioning the future. The PIF process was similar for basic science and clinical teachers. CONCLUSION: Our findings suggested that PIF among medical teachers in a non-Western setting is a continuous and dynamic process that is shaped by key socialization factors (e.g. role models, workplace learning, peer support), with significant influences from religious beliefs, family values, and societal recognition. Faculty development programs should consider the dynamic and continuous nature of PIF among medical teachers and encourage clinicians and basic scientists to explore their values and beliefs, realize their goals, and envision their future.

A professionalism program in medical education and training - From broad values to specific applications: YLL School of Medicine, Singapore.

The process for introducing and developing a program for teaching medical professionalism at the National University of Singapore, School of Medicine is outlined. Professionalism was recognised as embracing 'honesty and integrity,' 'responsibility and participation,' 'respect and sensitivity,' and 'compassion and empathy.' Those broad values are expressed as specific attitudes and behaviours that are taught and assessed throughout the course. Honesty and integrity, for example, are demonstrated by 'presenting original, authentic assignments' (in medical education); and 'accepting personal mistakes and honestly acknowledging them' (in clinical training and practice). Values and items of behaviour were drawn from the literature, and reviewed and refined to address needs identified within the Medical School. A broad spectrum of pre-clinical and clinical teachers contributed to this development. The program was reassessed to determine the extent to which it has been implemented and has evolved following its adoption. The results are confirming in that: the majority of recommendations have been implemented; the program has developed further; and is supported by ancillary student enrichment activities. Medical professionalism has been given prominence through all phases of the course. Nevertheless, challenges remain and particularly in the extent to which medical professionalism is taught and assessed in various clinical postings.

A BEME systematic review of teaching interventions to equip medical students and residents in early recognition and prompt escalation of acute clinical deteriorations: BEME Guide No. 62.

Background: Current educational interventions and teaching for acute deteriorations seem to address acute care learning in discreet segments. Technology enhanced and team training methodologies are in vogue though well studied in the nursing profession, teaching avenues for junior 'doctors in training' seem to be a lacuna.Aims: The BEME systematic review was designed to (1) appraise the existing published evidence on educational interventions that are intended for 'doctors in training' to teach early recognition and prompt escalation in acute clinical deteriorations (2) to synthesise evidence & to evaluate educational effectiveness.Methodology: The method applied was a descriptive, justification & clarification review. Databases searched included PubMed, PsycINFO, Science Direct and Scopus for original research and grey literature with no restrictions to year or language. Abstract review, full text decisions and data extraction were completed by two primary coders with final consensus by a third reviewer.Results: 5592 titles and abstracts were chosen after removal of 905 duplications. After exclusion of 5555 studies, 37 full text articles were chosen for coding. 22 studies met final criteria of educational effectiveness, relevance to acute care. Educational platforms varied from didactics to blended learning approaches, small group teaching sessions, simulations, live & cadaveric tissue training, virtual environments and insitu team-based training. Translational outcomes with reduction in long term (up to 3-6 years) morbidity & mortality with financial savings were reported by 18% (4/22) studies. Interprofessional training were reported in 41% (9/22) of studies. Recent evidence demonstrated effectiveness of virtual environment and mobile game-based learning.Conclusions: There were significant improvements in teaching initiatives with focus on observable behaviours and translational real patient outcomes. Serious game-based learning and virtual multi-user collaborative environments might enhance individual learners' cognitive deliberate practice. Acute care learning continuum with programmatic acute care portfolios could be a promise of the future.

Coordinated responses of academic medical centres to pandemics: Sustaining medical education during COVID-19.

Background: The Corona Virus Disease-19 (COVID-19) has been declared a pandemic by the World Health Organization (WHO). We state the consolidated and systematic approach for academic medical centres in response to the evolving pandemic outbreaks for sustaining medical education.Discussion: Academic medical centres need to establish a 'COVID-19 response team' in order to make time-sensitive decisions while managing pandemic threats. Major themes of medical education management include leveraging on remote or decentralised modes of medical education delivery, maintaining the integrity of formative and summative assessments while restructuring patient-contact components, and developing action plans for maintenance of essential activities based on pandemic risk alert levels. These core principles must be applied seamlessly across the various fraternities of academic centres: undergraduate education, residency training, continuous professional development and research. Key decisions from the pandemic response teams that help to minimise major disruptions in medical education and to control disease transmissions include: minimising inter-cluster cross contaminations and plans for segregation within and among cohorts; reshuffling academic calendars; postponing or restructuring assessments.Conclusions: While minimising the transmission of the pandemic outbreak within the healthcare establishments is paramount, medical education and research activities cannot come to a standstill each time there is a threat of one.

Is empathy change in medical school geo-socioculturally influenced?

CONTEXT: There is convincing evidence that physician empathy leads to better patient care. As a result, there has been considerable research interest in investigating how empathy changes during undergraduate medical studies. Early (generally North American) studies raised concerns that medical training causes a decline in empathy. More recent studies (conducted around the world) have begun to suggest that either a slight increase or decrease in empathy occurs during undergraduate medical training, which has led some to argue that empathy changes indiscriminately (with no discernible pattern). This paper explores whether there is evidence to suggest that empathy changes indiscriminately or with a discernible geo-sociocultural pattern during undergraduate medical training. METHODS: Literature that investigated change in empathy during undergraduate medical training was reviewed. Cross-sectional and longitudinal studies were tabulated separately according to their respective geographical locations. The tabulated results were analysed to investigate whether empathy changed similarly or differently within different geographical locations. RESULTS: The studies reviewed indicate similar patterns of empathy change within approximate geo-sociocultural clusters. Whereas US studies predominantly show small but significant decreases in empathy, Far Eastern studies mostly show small but significant increases in empathy as undergraduates progress through the medical course. CONCLUSIONS: These results suggest that change in empathy during undergraduate medical education is not as indiscriminate (patternless) as once thought. Additionally, these results support the notion that empathy is a locally construed global construct.

The clarion call for a third wave in medical education to optimise healthcare in the twenty-first century.

During the years preceding 1910, the education and training of physicians (doctors) -to-be was based mainly on a master-apprentice model; the primary focus then was on the teaching and development of clinical skills. In 1910, however, Abraham Flexner submitted a highly influential report to the American medical authorities: in it, he recommended that all medical schools should be university-based and that, importantly, medical practice should have a scientific basis strongly underpinned by the basic medical sciences. The recommendation provided the impetus for the design of medical education that begins with a pre-clinical phase to provide the strong scientific foundation for the clinical phase that follows. During the clinical phase, student learning will focus primarily on the clinical sciences relating to the diagnosis, treatment and management of patient care. Thus, two key 'pillars' (the basic sciences and the clinical sciences) of medical education were established; this two pillar model of medical education persisted for many decades thereafter and remained so till today. However, in order to optimise delivery of health care this must be viewed as an 'eco-system' taking into account the practice setting both present and future. The authors will attempt to provide a background to the changing trends in medical education and the changing practice environment, due primarily to the disruptive forces of change in this article.

International consensus statement on the assessment of interprofessional learning outcomes.

Regulatory frameworks around the world mandate that health and social care professional education programs graduate practitioners who have the competence and capability to practice effectively in interprofessional collaborative teams. Academic institutions are responding by offering interprofessional education (IPE); however, there is as yet no consensus regarding optimal strategies for the assessment of interprofessional learning (IPL). The Program Committee for the 17th Ottawa Conference in Perth, Australia in March, 2016, invited IPE champions to debate and discuss the current status of the assessment of IPL. A draft statement from this workshop was further discussed at the global All Together Better Health VIII conference in Oxford, UK in September, 2016. The outcomes of these deliberations and a final round of electronic consultation informed the work of a core group of international IPE leaders to develop this document. The consensus statement we present here is the result of the synthesized views of experts and global colleagues. It outlines the challenges and difficulties but endorses a set of desired learning outcome categories and methods of assessment that can be adapted to individual contexts and resources. The points of consensus focus on pre-qualification (pre-licensure) health professional students but may be transferable into post-qualification arenas.

Adoption and correlates of Postgraduate Hospital Educational Environment Measure (PHEEM) in the evaluation of learning environments - A systematic review.

BACKGROUND: The Postgraduate Hospital Educational Environment Measure (PHEEM) is a highly reliable and valid instrument to measure the educational environment during post graduate medical training. This review extends earlier reports by evaluating the extant adoption of PHEEM in various international clinical training sites, and its significant correlations in order to expand our understanding on the use of PHEEM and facilitate future applications and research. METHOD: A systematic literature review was conducted on all articles between 2005 and October 2015 that adopted and reported data using the PHEEM. RESULTS: Overall 30 studies were included, encompassing data from 14 countries internationally. Notable differences in the PHEEM scores were found between different levels of training, disciplines, and clinical training sites. Common strengths and weaknesses in learning environments were observed and there were significant correlations between PHEEM scores and In-Training Exam (ITE) performance (positive correlation) and level of burnout (negative correlation), respectively. CONCLUSIONS: PHEEM is widely adopted in different learning settings, and is a useful tool to identify the strengths and weaknesses of an educational environment. Future research can examine other correlates of PHEEM and longitudinal changes in interventional studies.

Medical education in Singapore.

Allopathic medical education in Singapore extends for more than a century from its simple beginnings. In recent times, changes have been rapid, both in undergraduate and postgraduate specialty medical training. Over the last decade, undergraduate medical education has increased from a single to three medical schools and the postgraduate training has expanded further by incorporating the Accreditation Council for Graduate Medical Education International framework. With these changes, the curricula, assessment systems, as well as teaching and learning approaches, with the use of technology-enhanced learning and program evaluation processes have expanded, largely based on best evidence medical education. To support these initiatives and the recent rapid expansion, most training institutions have incorporated faculty development programs, such as the Centre for Medical Education at the National University of Singapore.

Encouraging an environment to nurture lifelong learning: an Asian experience.

INTRODUCTION: Within an Asian context, this study examines the effect of changing from traditional course grades to a distinction/pass/fail (D/P/F) grading system on medical student self-perceived stress levels and on student exam performance. METHODS: At the end of the 2010-2011 academic year, the Perceived Stress Scale-10 (PSS-10) was administered to the cohort of students finishing their first year of medical studies. For the academic year 2011-2012, the grading system was changed to D/P/F for the first year of medical school. The PSS-10 was also administered to the subsequent cohort of first-year medical students at the same point in the academic year as previous. Qualitative comments were collected for both cohorts. RESULTS: Stress as measured by the PSS-10 was significantly lower in the cohort that went through the year with the D/P/F grading system in place. Thematic analysis of qualitative responses showed a shift in sources of student stress away from peer-competition. There were no significant differences in overall exam performance. DISCUSSION: Within an Asian context, switching to a D/P/F grading system can alleviate stress and peer competition without compromising knowledge. This may help foster a "learning orientation" rather than an "exam orientation," and contribute to inculcating lifelong learning skills.

Building a successful platform for interprofessional education for health professions in an Asian university.

Implementing Interprofessional Education (IPE) across health professions schools is challenging. Within an Asian context, academic staff at the National University of Singapore designed a platform to create a sustainable IPE effort. A two-pronged approach was developed to ensure adequate coverage of key concepts relating to IPE within each involved faculty. The Interprofessional Core Curricula (ICC) component ensures that each health profession student will be exposed to IPE concepts in their required curriculum. Interprofessional Enrichment Activities (IEA) incentivize further cross-faculty participation and progress within the IPE competency framework. Best practices and success factors were identified, while lessons learned led to further improvements. Adoption of this approach can help circumvent well-known barriers to implementation.