Using MCQ response certainty to determine how aspects of self-monitoring develop through a medical course.

INTRODUCTION: Self-monitoring of clinical-decision-making is essential for health care professional practice. Using certainty in responses to assessment items could allow self-monitoring of clinical-decision-making by medical students to be tracked over time. This research introduces how aspects of insightfulness, safety and efficiency could be based on certainty in, and correctness of, multiple-choice question (MCQ) responses. We also show how these measures change over time. METHODS: With each answer on twice yearly MCQ progress tests, medical students provided their certainty of correctness. An insightful student would be more likely to be correct for those answers given with increasing certainty. A safe student would be expected to have a high probability of being correct for answers given with a high certainty. An efficient student would be expected to have a sufficiently low probability of being correct when they have no certainty. The system was developed using first principles and data from one cohort of students. A dataset from a second cohort was then used as an independent validation sample. RESULTS: The patterns of aspects of self-monitoring were similar for both cohorts. Almost all the students met the criteria for insightfulness on all tests. Most students had an undetermined outcome for the safety aspect. When a definitive result for safety was obtained, absence of safety was most prevalent in the middle of the course, while the presence of safety increased later. Most of the students met the criteria for efficiency, with the highest prevalence mid-course, but efficiency was more likely to be absent later. DISCUSSION: Throughout the course, students showed reassuring levels of insightfulness. The results suggest that students may balance safety with efficiency. This may be explained by students learning the positive implications of decisions before the negative implications, making them initially more efficient, but later being more cautious and safer.

Cross-cultural code-switching - the impact on international medical graduates in New Zealand.

BACKGROUND: New Zealand relies on International Medical Graduates (IMGs); however, the retention of IMGs is not optimal. This research uses a lens of cross-cultural code-switching to explore how professional and cultural differences impact on International Medical Graduates' (IMGs') journeys to practise effectively and remain in New Zealand. METHODS: Utilising theory-informing inductive analysis within a constructivist approach, framework analysis was conducted following 14 face-to-face interviews with IMGs. The analysis then explored the degree to which their experiences could be explained by cross-cultural code-switching's psychological challenges (authenticity, competence, and resentment). RESULTS: Analysis showed there was an expectation for IMGs to code-switch. The greater the cultural and professional difference of IMGs (compared to New Zealand), the greater the intensity of psychological challenges experienced when switching. Moreover, IMGs received minimal support, making it difficult to overcome psychological challenges, especially the competence challenge. This led to feelings of frustration and vulnerability. Code-switching could also explain why complaints about IMGs were more likely when IMGs were stressed or tired. CONCLUSION: Cross-cultural code-switching can be used to explain and identify how cultural differences cause psychological challenges. These findings inform how programmes can better support IMGs in orientation and ongoing training. Additionally, establishing, and allocating IMG cultural mentors would assist in addressing IMGs' vulnerability and isolation. With this support, the journey may prove more manageable and encourage IMGs to continue practising in their adopted country.

Sample representativeness and influence of attrition on longitudinal data collected as part of a national medical career tracking project.

BACKGROUND: The Medical Schools Outcomes Database and Longitudinal Tracking Project (MSOD) in New Zealand is one example of a national survey-based resource of medical student experiences and career outcomes. Longitudinal studies of medical students are valuable for evaluating the outcomes of medical programs against workforce objectives. As a prospective longitudinal multiple-cohort study, survey response rates at each collection point of MSOD vary. This paper assesses the effects of participant non-response rates on MSOD data. METHODS: Demographic variables of MSOD respondents between 2012 and 2018 were compared to the distribution of the demographic variables in the population of all NZ medical graduates to ascertain whether respondent samples at multiple survey collection points were representative of the population. Analysis using logistic regression assessed the impact of participant non-response on variables at collection points throughout MSOD. RESULTS: 2874 out of a total population of 2939 domestic medical students graduating between 2012 and 2018 responded to MSOD surveys. Entry and exit surveys achieved response rates around 80% and were broadly representative of the total population on demographic variables. Post-graduation survey response rates were around 50% of the total population of graduates and underrepresented graduates from the University of Auckland. Between the entry and exit and the exit and postgraduation year three samples, there was a significant impact of non-response on ascribed variables, including age at graduation, university, gender and ethnic identity. Between the exit and postgraduation year one sample, non-response significantly impacted ascribed and non-ascribed variables, including future practice intentions. CONCLUSION: Samples collected from MSOD at entry and exit are representative, and findings from cross-sectional studies using these datasets are likely generalisable to the wider population of NZ medical graduates. Samples collected one and three years post-graduation are less representative. Researchers should be aware of this bias when utilizing these data. When using MSOD data in a longitudinal manner, e.g. comparing the change in career intentions from one collection point to the next, researchers should appropriately control for bias due to non-response between collection points. This study highlights the value of longitudinal career-tracking studies for answering questions relevant to medical education and workforce development.

The consequences of cultural difference: the international medical graduate journey in New Zealand.

Objectives: To identify how differences in cultural and professional values between New Zealand born and trained doctors and International Medical Graduates (IMGs) affect the practice and retention of IMGs in New Zealand. Methods: A mixed-method approach was used. An anonymous 42-item online questionnaire was used to compare participants' cultural and professional values. Participants were 373 New Zealand doctors, 198 IMG, and 25 doctors born and raised elsewhere but who qualified in New Zealand, a group not identified prospectively. The qualitative component used interviews with 14 IMGs to identify cultural challenges faced and with nine New Zealand doctors to identify the challenges they faced working alongside IMGs. Qualitative data were transcribed and analysed thematically. Results: There were differences in power distance, with the medically qualified in New Zealand doctors having the highest power distance, followed by the IMGs, suggesting a preference for a hierarchical environment at odds with the New Zealand culture. Interviews found cultural differences in communication styles and hierarchy contributed to professional challenges. The cultural transition was difficult for IMGs as they received minimal support. One-third of IMGs acknowledged their behaviours did not fit well in New Zealand. Complaints about IMGs increased when they reverted to default behaviours regarded negatively by New Zealand colleagues or patients. Conclusions: IMGs are open to change but face a lack of orientation and cultural education opportunities, hindering integration. Residency programs must recognise this disconnect and incorporate cross-cultural programmes in the curriculum. Such programmes would assist the adaption and retention of IMG doctors.

Supporting Bystanders and Other Staff Exposed to Negative Workplace Behaviour: Developing and Testing the IMO Intervention Framework in the Clinical Workplace.

Background: Medical student and health-care staff bullying remain international concerns. Our understanding about what might solve such problems is still developing. A common approach suggested for bystanders to bullying is that they challenge or "stand up" to the bully. At the same time, the considerable risks should they act (eg, verbal rebuke) are rarely acknowledged, and neither is the potential for considerable cognitive dissonance should they choose not to. Methods: Drawing on a substantial literature review, we developed an interventional program, initially with medical student bullying in mind, that generally employed a discourse for values cultivation. We implemented and evaluated this program through 6 cycles of action research methodology in 6 different departments of one tertiary teaching hospital. Data include individual interviews, discussions with key participants plus freehand comments from wider staff's workshop evaluations. Results: Alongside others, an early serendipitous finding was the "IMO" [Indirect, focus on Me, focus on Outcome] framework, intended for bystanders to student bullying. From using this framework, participants reported developing confidence in tackling negative behaviours, that interactions had better outcomes and that the workplace atmosphere felt more relaxed. It's use also significantly reduced cognitive dissonance. Encouragingly, participants went on to use IMO for staff-staff bullying incidents, and other common negative behaviours. We attribute IMO's utility to its memorability, simplicity and for catering to the many possible causes of bullying, including values issues, without perpetuating or creating conflict. Conclusion: The IMO framework is offered as a widely usable framework for all staff in a clinical environment - a useful addition to popular frameworks used in the clinical workplace, ideas from which might also help these be further refined. Further evaluation would establish whether IMO invites long-term change and its utility in other contexts.

Educators as Judges: Applying Judicial Decision-Making Principles to High-Stakes Education Assessment Decisions.

Phenomenon: Programmatic assessment and competency-based education have highlighted the need to make robust high-stakes assessment decisions on learner performance from evidence of varying types and quality. Without guidance, lengthy deliberations by decision makers and competence committees can end inconclusively with unresolved concerns. These decisional dilemmas are heightened by their potential impacts. For learners, erroneous decisions may lead to an unjustified exit from a long-desired career, or premature promotion to clinical responsibilities. For educators, there is the risk of wrongful decision-making, leading to successful appeals and mistrust. For communities, ill-prepared graduates risk the quality and safety of care. Approaches such as psychometric analyses are limited when decision-makers are faced with seemingly contradictory qualitative and quantitative evidence about the same individual. Expertise in using such evidence to make fair and defensible decisions is well established in judicial practice but is yet to be practically applied to assessment decision-making. Approach: Through interdisciplinary exchange, we investigated medical education and judicial perspectives on decision-making to explore whether principles of decision-making in law could be applied to educational assessment decision-making. Using Dialogic Inquiry, an iterative process of scholarly and mutual critique, we contrasted assessment decision making in medical education with judicial practice to identify key principles in judicial decision-making relevant to educational assessment decisions. We developed vignettes about common but problematic high-stakes decision-making scenarios to test how these principles could apply. Findings: Over 14 sessions, we identified, described, and applied four principles for fair, reasonable, and transparent assessment decision-making. These were: The person whose interests are affected has a right to know the case against them, and to be heard.Reasons for the decision should be given.Rules should be transparent and consistently applied.Like cases should be treated alike and unlike cases treated differently.Reflecting our dialogic process, we report findings by separately presenting the medical educator and judicial perspectives, followed by a synthesis describing a preferred approach to decision-making in three vignettes. Insights: Judicial principles remind educators to consider both sides of arguments, to be consistent, and to demonstrate transparency when making assessment decisions. Dialogic Inquiry is a useful approach for generating interdisciplinary insights on challenges in medical education by critiquing difference (e.g., the meaning of objectivity) and achieving synthesis where possible (e.g., fairness is not equal treatment of all cases). Our principles and exemplars provide groundwork for promoting good practice and furthering assessment research toward fairer and more robust decisions that will assist learning.

Four ways to get a grip on making robust decisions from workplace-based assessments.

Synthesising the results of workplace-based assessments to inform robust decisions is seen as both important and difficult. Concerns about failing to fail the trainee not ready to proceed has drawn disproportionate attention to assessors. This paper proposes a model for a more systems-based view so that the value of the assessor's judgement is incorporated while preserving the value and robustness of collective decision-making. Our experience has shown it can facilitate robust decisions on some of the more difficult areas, such as professionalism.

Il est aussi important que difficile de synthétiser les résultats d'évaluations en milieu de travail pour fonder des décisions solides. Les préoccupations liées à la renonciation à faire échouer les stagiaires qui ne sont pas prêts à poursuivre leur formation ont focalisé l'attention sur les évaluateurs, et ce de manière disproportionnée. Cet article propose un modèle avec une vision plus systémique permettant d'intégrer la valeur du jugement de l'évaluateur tout en préservant la valeur et la solidité de la prise de décision collective. Notre expérience a montré qu'il peut faciliter la prise de décisions robustes en ce qui concerne des domaines plus difficiles comme le professionnalisme.

Learning is what happens between seeing patients: defining clinical access.

The inability to access clinical placements during the COVID-19 pandemic stimulated us to reflect on key elements of the experience, beyond history taking and examination. We were also mindful of concerns about work readiness of new graduates. We identified seven aspects of clinical experience distinct from those requiring direct patient contact. These are: recognise and contribute to the collective competence of multidisciplinary teams; apply project management principles to the complexities of clinical care; integrate personal and team-based clinical reasoning; deliver patient-centred collaborative care; achieve an integrated perspective of clinical care; demonstrate adaptability to health systems; consolidate professional identity formation. We consider that making these aspects explicit in learning objectives and assessments in medical schools is likely to improve the work-readiness of new graduates and should also be reflected in accreditation standards.

Learning, judgement and workforce tensions impact failure to fail in a medical postgraduate setting: A supervisor perspective.

PURPOSE: Medical training organisations have a duty to prepare medical graduates for future safe, competent practice. Decisions about underperformance are high stakes at the postgraduate level and failure to fail can occur. We aimed to explore this concept from a systems and supervisor perspective. METHOD: Supervisors of specialist physician trainees were invited to provide written feedback on failure to fail as part of a broader anonymous supervisor survey. They were provided with a trigger statement and responded in free-text format. A deductive content analysis was undertaken through the lenses of supervisor and institution. RESULTS: Of 663 supervisors who responded to the broader survey, 373 (56%) provided feedback on the failure to fail trigger statement. Analyses indicated an interplay between trainee and supervisor characteristics, and broader system elements. System elements that contributed to failure to fail trainees included lack of longitudinal monitoring and quality of assessment information. Supervisor characteristics included confident, conflicted and avoidant behaviours towards underperforming trainees. CONCLUSIONS: Individual and system challenges that contributed to failure to fail were identified in this study, and we propose a three-way tension among learning, judgement and workforce. Three potential mitigation strategies have been identified to reduce failure to fail, namely a stage-based approach to remediation, faculty development in supervisory skills and improved assessment-for-learning processes.

When the wheels fall off - Medical students' experiences of interrupted academic progression.

INTRODUCTION: There is limited published research on medical students' perspectives of a significant interruption to their academic progression. This study sought to identify the factors that contribute to difficulties with academic progression and to understand how medical students successfully respond. METHODS: This interpretive phenomenological study reports on the findings from in-depth interviews of 38 final year medical students who had experienced a significant academic interruption. RESULTS: The two superordinate themes were: the factors contributing to the interruption and their experience of the interruption. Factors identified as contributing to the interruption were: workload, learning in medicine, motivation for medicine, isolation, adapting to local culture, health and external factors. Their experience of the interruption focused on stages of working through the process: 'what happened,' 'how it felt,' 'managing the failure,' 'accepting the failure' and 'making some changes.' DISCUSSION: Each factor affected how the participants reacted and responded to the interruption. Regardless of the origins of the interruption, most reacted and responded in a comparable process, albeit with varying timespans. These reactions and responses were in a state of fluctuation. In order to succeed many stated they shifted their motivation from external to internal, in direct response to the interruption, resulting in changed learning behaviours. CONCLUSIONS: The process of working through an interruption to academic progression for students may benefit from a model of interval debriefing, restorative academic and personal development support. Facilitation of this process could enable students to face an interruption constructively rather than as an insurmountable emotionally burdensome barrier. Medical schools could utilise these findings to implement further support strategies to reduce the number of significant academic disruptions.

Technology enhanced neuroanatomy teaching techniques: A focused BEME systematic review of current evidence: BEME Guide No. 75.

BACKGROUND: In response to growing curriculum pressures and reduced time dedicated to teaching anatomy, research has been conducted into developing innovative teaching techniques. This raises important questions for neuroanatomy education regarding which teaching techniques are most beneficial for knowledge acquisition and long-term retention, and how they are best implemented. This focused systematic review aims to provide a review of technology-enhanced teaching methods available to neuroanatomy educators, particularly in knowledge acquisition and long-term retention, compared to traditional didactic techniques, and proposes reasons for why they work in some contexts. METHODS: Electronic databases were searched from January 2015 to June 2020 with keywords that included combinations of 'neuroanatomy,' 'technology,' 'teaching,' and 'effectiveness' combined with Boolean phrases 'AND' and 'OR.' The contexts and outcomes for all studies were summarised while coding, and theories for why particular interventions worked were discussed. RESULTS: There were 4287 articles identified for screening, with 13 studies included for final analysis. There were four technologies of interest: stereoscopic views of videos, stereoscopic views of images, augmented reality (AR), and virtual reality (VR). No recommendation for a particular teaching method was made in six studies (46%) while recommendations (from weak to moderate) were made in seven studies (54%). There was weak to moderate evidence for the efficacy of stereoscopic images and AR, and no difference in the use of stereoscopic videos or VR compared to controls. CONCLUSIONS: To date, technology-enhanced teaching is not inferior to teaching by conventional didactic methods. There are promising results for these methods in complex spatial anatomy and reducing cognitive load. Possible reasons for why interventions worked were described including students' engagement with the object, cognitive load theory, complex spatial relationships, and the technology learning curve. Future research may build on the theorised explanations proposed here and develop and test innovative technologies that build on prior research.

Low-impact (compliant) flooring and staff injuries.

INTRODUCTION: Low impact flooring (LIF) has shown potential for reducing fall-related injuries for older people in residential care or hospital environments. However, the increased rolling resistance when moving equipment on these floors has raised concerns that staff injuries may increase. METHODS: LIF was trialled on one Older Persons Health ward for 2.5 years. Reported staff injuries were monitored during and following the trial. Numbers of staff injured on the LIF ward were compared with three other similar and adjacent OPH wards without LIF for the duration of the trial ('concurrent control' evaluation). At the trial conclusion the LIF ward moved to a different hospital that had standard flooring. This enabled a further 'during and after' evaluation where numbers of staff injured from the LIF ward during the trial were compared with those reported afterwards by the same ward staff without LIF. RESULTS: There was no difference in the numbers of staff injured in the LIF ward compared with the concurrent control wards (28 LIF vs 30 control; p = 0.44). The number of staff with injuries in the LIF ward also did not significantly alter when those staff moved to a new ward without LIF (45 after vs 28 before; p = 0.11). CONCLUSION: There was no change in the numbers of staff with injuries during the LIF trial in an Older Persons Health ward. This small study suggests LIF appears safe for both patients and staff.Implications for rehabilitationFalls in hospital are common with patient injuries occurring in approximately 20-30% of falls.Low impact (compliant) flooring may reduce fall-related injuries in hospitals and residential care.Low impact flooring has an increased rolling resistance, which has the potential to increase staff injuries when moving equipment.This study found no change in the number of staff injured during a low impact flooring trial providing some reassurance that these floors are safe for staff.

Widening how we see the impact of culture on learning, practice and identity development in clinical environments.

As part of this State of the Science series on Self, Society and Situation, we focus on how we might see the situation of the workplace as a learning environment in the future. Research to date into how health professionals learn while working in clinical workplace environments has mostly focused on the supervisor-trainee relationship or on the interaction between the affordances of a workplace and the receptiveness of trainees. However, the wider environment has not received as much focus-though frequently mentioned, it is seldom investigated. We suggest there is a need to embrace the wider institution factors, recognise and acknowledge an organisation's values and culture as they impact on clinical learning in order to work with these, not around them or ignore them, to make what may be tacit visible through reflection and observation and to embrace a range of perspectives on culture.

Twelve tips for medical students experiencing an interruption in their academic progress.

For every commencing cohort of medical students, a small but significant number will experience an interruption to their academic progression because of academic difficulties, health concerns or external influences outside of the students' control. During the process of researching the factors surrounding difficulties with academic progression, students told us many ways that they have learned from that experience, which then allowed most of them to graduate. This paper combines the shared experiences of students who have had an interruption, and those of the authors as medical educators.

Risk factors for injuries in New Zealand older adults with complex needs: a national population retrospective study.

BACKGROUND: Falls and falls-related injuries are common among older adults. Injuries in older adults lead to poor outcomes and lower quality of life. The objective of our study was to identify factors associated with fall-related injuries among home care clients in New Zealand. METHODS: The study cohort consisted of 75,484 community-dwelling people aged 65 years or older who underwent an interRAI home care assessment between June 2012 and June 2018 in New Zealand. The injuries included for analysis were fracture of the distal radius, hip fracture, pelvic fracture, proximal humerus fracture, subarachnoid haemorrhage, traumatic subdural haematoma, and vertebral fracture. Unadjusted and adjusted competing risk regression models were used to identify factors associated with fall-related injuries. RESULTS: A total of 7414 (9.8%) people sustained a falls-related injury over the 6-year period, and most injuries sustained were hip fractures (4735 63.9%). The rate of injurious falls was 47 per 1000 person-years. The factors associated with injury were female sex, older age, living alone, Parkinson's disease, stroke/CVA, falls, unsteady gait, tobacco use, and being underweight. Cancer, dyspnoea, high BMI, and a decrease in the amount of food or fluid usually consumed, were associated with a reduced risk of sustaining an injury. After censoring hip fractures the risks associated with other types of injury were sex, age, previous falls, dyspnoea, tobacco use, and BMI. CONCLUSIONS: While it is important to reduce the risk of falls, it is especially important to reduce the risk of falls-related injuries. Knowledge of risk factors associated with these types of injuries can help to develop focused intervention programmes and development of a predictive model to identify those who would benefit from intervention programmes.

Global health classroom: mixed methods evaluation of an interinstitutional model for reciprocal global health learning among Samoan and New Zealand medical students.

BACKGROUND: Global health education partnerships should be collaborative and reciprocal to ensure mutual benefit. Utilisation of digital technologies can overcome geographic boundaries and facilitate collaborative global health learning. Global Health Classroom (GHCR) is a collaborative global health learning model involving medical students from different countries learning about each other's health systems, cultures, and determinants of health via videoconference. Principles of reciprocity and interinstitutional partnership informed the development of the GHCR. This study explores learning outcomes and experiences in the GHCR between students from New Zealand and Samoa. METHODS: This study used a mixed methods approach employing post-GHCR questionnaires and semi-structured face-to-face interviews to explore self-reported learning and experiences among medical students in the GHCR. The GHCR collaboration studied was between the medical schools at the University of Otago, New Zealand and the National University of Samoa, Samoa. RESULTS: Questionnaire response rate was 85% (74/87). Nineteen interviews were conducted among New Zealand and Samoan students. Students reported acquiring the intended learning outcomes relating to patient care, health systems, culture, and determinants of health with regards to their partner country. Interview data was indicative of attitudinal changes in relation to cultural humility and curiosity. Some reported a vision for progress regarding their own health system. Students in the GHCR reported that learning with their international peers in the virtual classroom made learning about global health more real and tangible. The benefits to students from both countries indicated reciprocity. CONCLUSIONS: This study demonstrates GHCR to be a promising model for collaborative and reciprocal global health learning using a student-led format and employing digital technology to create a virtual classroom. The self-reported learning outcomes align favourably with those recommended in the literature. In view of our positive findings, we present GHCR as an adaptable model for equitable, collaborative global health learning between students in internationally partnered institutions.

The Case for Junior Doctor Academic Internships: A Narrative Review of the Published Literature.

INTRODUCTION: There is a growing concern with the dwindling academic workforce especially in medicine. Academic internships refer to a hybrid internship during which an intern spends a period of time (typically 3-6 months) in academic/research endeavours. These may serve as initial research experience for junior doctors. However, the merits of this programme have not been assessed to date. METHODS: Studies on integrated academic internships by junior doctors (ie, interns) were included in the present review. The identified articles were grouped into themes. For each article, the methodological approach (and subsequent implemented methods) was noted. The articles were also critically appraised for methodological soundness (both at the study-level, and the outcome-level). RESULTS: A total of 1621 publication titles were identified and screened, of which 8 publications were included in the final review. The major themes of the identified publications are: overview of the academic internship programme, evaluation of a programme's experience and outcomes, and other miscellaneous publications. DISCUSSION: The studies to date have only reported on "soft outcomes", but overall, interns and supervisors alike appear to be satisfied with the programme. Whether the programme increases the likelihood of future academic careers is difficult to establish at this juncture. The academic internship posts appear to be competitive, which reflect their popularity among graduating medical students.

Ottawa 2020 consensus statement for programmatic assessment - 1. Agreement on the principles.

INTRODUCTION: In the Ottawa 2018 Consensus framework for good assessment, a set of criteria was presented for systems of assessment. Currently, programmatic assessment is being established in an increasing number of programmes. In this Ottawa 2020 consensus statement for programmatic assessment insights from practice and research are used to define the principles of programmatic assessment. METHODS: For fifteen programmes in health professions education affiliated with members of an expert group (n = 20), an inventory was completed for the perceived components, rationale, and importance of a programmatic assessment design. Input from attendees of a programmatic assessment workshop and symposium at the 2020 Ottawa conference was included. The outcome is discussed in concurrence with current theory and research. RESULTS AND DISCUSSION: Twelve principles are presented that are considered as important and recognisable facets of programmatic assessment. Overall these principles were used in the curriculum and assessment design, albeit with a range of approaches and rigor, suggesting that programmatic assessment is an achievable education and assessment model, embedded both in practice and research. Knowledge on and sharing how programmatic assessment is being operationalized may help support educators charting their own implementation journey of programmatic assessment in their respective programmes.

Ottawa 2020 consensus statements for programmatic assessment - 2. Implementation and practice.

INTRODUCTION: Programmatic assessment is a longitudinal, developmental approach that fosters and harnesses the learning function of assessment. Yet the implementation, a critical step to translate theory into practice, can be challenging. As part of the Ottawa 2020 consensus statement on programmatic assessment, we sought to provide descriptions of the implementation of the 12 principles of programmatic assessment and to gain insight into enablers and barriers across different institutions and contexts. METHODS: After the 2020 Ottawa conference, we surveyed 15 Health Profession Education programmes from six different countries about the implementation of the 12 principles of programmatic assessment. Survey responses were analysed using a deductive thematic analysis. RESULTS AND DISCUSSION: A wide range of implementations were reported although the principles remained, for the most part, faithful to the original enunciation and rationale. Enablers included strong leadership support, ongoing faculty development, providing students with clear expectations about assessment, simultaneous curriculum renewal and organisational commitment to change. Most barriers were related to the need for a paradigm shift in the culture of assessment. Descriptions of implementations in relation to the theoretical principles, across multiple educational contexts, coupled with explanations of enablers and barriers, provided new insights and a clearer understanding of the strategic and operational considerations in the implementation of programmatic assessment. Future research is needed to further explore how contextual and cultural factors affect implementation.