Validity evidence and psychometric evaluation of a socially accountable health index for health professions schools.

There is an expectation that health professions schools respond to priority societal health needs. This expectation is largely based on the underlying assumption that schools are aware of the priority needs in their communities. This paper demonstrates how open-access, pan-national health data can be used to create a reliable health index to assist schools in identifying societal needs and advance social accountability in health professions education. Using open-access data, a psychometric evaluation was conducted to examine the reliability and validity of the Canadian Health Indicators Framework (CHIF) conceptual model. A non-linear confirmatory factor analysis (CFA) on 67 health indicators, at the health-region level (n = 97) was used to assess the model fit of the hypothesized 10-factor model. Reliability analysis using McDonald's Omega were conducted, followed by Pearson's correlation coefficients. Findings from the non-linear CFA rejected the original conceptual model structure of the CHIF. Exploratory post hoc analyses were conducted using modification indices and parameter constraints to improve model fit. A final 5-factor multidimensional model demonstrated superior fit, reducing the number of indicators from 67 to 32. The 5-factors included: Health Conditions (8-indicators); Health Functions (6-indicators); Deaths (5-indicators); Non-Medical Health Determinants (7-indicators); and Community & Health System Characteristics (6-indicators). All factor loadings were statistically significant (p < 0.001) and demonstrated excellent internal consistency ( ω >0.95). Many schools struggle to identify and measure socially accountable outcomes. The process highlighted in this paper and the indices developed serve as starting points to allow schools to leverage open-access data as an initial step in identifying societal needs.

Competency-based medical education: The spark to ignite healthcare's escape fire.

High-value care is what patients deserve and what healthcare professionals should deliver. However, it is not what happens much of the time. Quality improvement master Dr. Don Berwick argued more than two decades ago that American healthcare needs an escape fire, which is a new way of seeing and acting in a crisis situation. While coined in the U.S. context, the analogy applies in other Western healthcare contexts as well. Therefore, in this paper, the authors revisit Berwick's analogy, arguing that medical education can, and should, provide the spark for such an escape fire across the globe. They assert that medical education can achieve this by fully embracing competency-based medical education (CBME) as a way to place medicine's focus on the patient. CBME targets training outcomes that prepare graduates to optimize patient care. The authors use the escape fire analogy to argue that medical educators must drop long-held approaches and tools; treat CBME implementation as an adaptive challenge rather than a technical fix; demand genuine, rich discussions and engagement about the path forward; and, above all, center the patient in all they do.

Reliability, validity and acceptability of an online clinical reasoning simulator for medical students: An international pilot.

INTRODUCTION: Clinical reasoning skills are essential for decision-making. Current assessment methods are limited when testing clinical reasoning and management of uncertainty. This study evaluates the reliability, validity and acceptability of Practicum Script, an online simulation-based programme, for developing medical students' clinical reasoning skills using real-life cases. METHODS: In 2020, we conducted an international, multicentre pilot study using 20 clinical cases with 2457 final-year medical students from 21 schools worldwide. Psychometric analysis was performed (n = 1502 students completing at least 80% of cases). Classical estimates of reliability for three test domains (hypothesis generation, hypothesis argumentation and knowledge application) were calculated using Cronbach's alpha and McDonald's omega coefficients. Validity evidence was obtained by confirmatory factor analysis (CFA) and measurement alignment (MA). Items from the knowledge application domain were analysed using cognitive diagnostic modelling (CDM). Acceptability was evaluated by an anonymous student survey. RESULTS: Reliability estimates were high with narrow confidence intervals. CFA revealed acceptable goodness-of-fit indices for the proposed three-factor model. CDM analysis demonstrated good absolute test fit and high classification accuracy estimates. Student survey responses showed high levels of acceptability. CONCLUSION: Our findings suggest that Practicum Script is a useful resource for strengthening students' clinical reasoning skills and ability to manage uncertainty.

Utility of a multimodal computer-based assessment format for assessment with a higher degree of reliability and validity.

Multiple choice questions (MCQs) suffer from cueing, item quality and factual knowledge testing. This study presents a novel multimodal test containing alternative item types in a computer-based assessment (CBA) format, designated as Proxy-CBA. The Proxy-CBA was compared to a standard MCQ-CBA, regarding validity, reliability, standard error of measurement, and cognitive load, using a quasi-experimental crossover design. Biomedical students were randomized into two groups to sit a 65-item formative exam starting with the MCQ-CBA followed by the Proxy-CBA (group 1, n = 38), or the reverse (group 2, n = 35). Subsequently, a questionnaire on perceived cognitive load was taken, answered by 71 participants. Both CBA formats were analyzed according to parameters of the Classical Test Theory and the Rasch model. Compared to the MCQ-CBA, the Proxy-CBA had lower raw scores (p < 0.001, η2 = 0.276), higher reliability estimates (p < 0.001, η2 = 0.498), lower SEM estimates (p < 0.001, η2 = 0.807), and lower theta ability scores (p < 0.001, η2 = 0.288). The questionnaire revealed no significant differences between both CBA tests regarding perceived cognitive load. Compared to the MCQ-CBA, the Proxy-CBA showed increased reliability and a higher degree of validity with similar cognitive load, suggesting its utility as an alternative assessment format.

Global Perceptions on Social Accountability and Outcomes: A Survey of Medical Schools.

Phenomenon: Social accountability has become a universal component in medical education. However, medical schools have little guidance for operationalizing and applying this concept in practice. This study explored institutional practices and administrative perceptions of social accountability in medical education. Approach: An online survey was distributed to a purposeful sample of English-speaking undergraduate medical school deans and program directors/leads from 245 institutions in 14 countries. The survey comprised of 38-items related to program mission statements, admission processes, curricular content, and educational outcomes. Survey items were developed using previous literature and categorized using a context-input-process-products (CIPP) evaluation model. Exploratory Factor Analysis (EFA) was used to assess the inter-relationship among survey items. Reliability and internal consistency of items were evaluated using McDonald's Omega. Findings: Results from 81 medical schools in 14 countries collected between February and June 2020 are presented. Institutional commonalities of social accountability were observed. However, our findings suggest programs focus predominately on educational inputs and processes, and not necessarily on outcomes. Findings from our EFA demonstrated excellent internal consistency and reliability. Four-factors were extracted: (1) selection and recruitment; (2) institutional mandates; (3) institutional activities; and (4) community awareness, accounting for 71% of the variance. McDonald's Omega reliability estimates for subscales ranged from 0.80-0.87. Insights: This study identified common practices of social accountability. While many medical schools expressed an institutional commitment to social accountability, their effects on the community remain unknown and not evaluated. Overall, this paper offers programs and educators a psychometrically supported tool to aid in the operationalization and reliability of evaluating social accountability.

Marking parties for marking written assessments: A spontaneous community of practice.

In programmes of assessment with both high and low-stakes assessments, the inclusion of open-ended long answer questions in the high-stakes examination can contribute to driving deeper learning among students. However, in larger institutions, this would generate a seemingly insurmountable marking workload. In this study, we use a focused ethnographic approach to explore how such a marking endeavour can be tackled efficiently and pragmatically. In marking parties, examiners come together to individually mark student papers. This study focuses on marking parties for two separate tasks assessing written clinical communication in medical school finals at Southampton, UK. Data collected included field notes from 21.3 h of marking parties, details of demographics and clinical and educational experience of examiners, examiners' written answers to an open-ended post-marking party questionnaire, an in-depth interview and details of the actual marks assigned during the marking parties. In a landscape of examiners who are busy clinicians and rarely interact with each other educationally, marking parties represent a spontaneous and sustainable community of practice, with functions extending beyond the mere marking of exams. These include benchmarking, learning, managing biases and exam development. Despite the intensity of the work, marking parties built camaraderie and were considered fun and motivating.

Entrustable Professional Activities and Learning: The Postgraduate Trainee Perspective.

OBJECTIVE: Entrustable professional activities (EPAs) are used as clinical activities in postgraduate psychiatry training in Australasia. This study aimed to explore psychiatry trainees' perceptions of the impact of EPAs on their motivation and learning. METHODS: A constructivist grounded theory approach was used to conceptualize the impact of EPAs on trainees' motivation and learning. A purposive sample of trainees was recruited from across New Zealand. Semi-structured individual interviews were used for data collection and continued until theoretical saturation was reached. RESULTS: The impact of EPAs on learning was mediated by the trainee's appraisals of subjective control, value, and the costs of engaging with EPAs. When appraisals were positive, EPAs encouraged a focus on particular learning needs and structured learning with the supervisor. However, when appraisals were negative, EPAs encouraged a superficial approach to learning. Trainee appraisals and their subsequent impact on motivation and learning were most affected by EPA granularity, alignment of EPAs with clinical practice, and the supervisor's conscientiousness in their approach to EPAs. CONCLUSIONS: To stimulate learning, EPAs must be valued by both trainees and supervisors as constituting a coherent work-based curriculum that encompasses the key fellowship competencies. If EPAs are to be effective as clinical tasks for learning, ongoing faculty development must be the leading priority.

An international study on the implementation of programmatic assessment: Understanding challenges and exploring solutions.

INTRODUCTION: Programmatic assessment is an approach to assessment aimed at optimizing the learning and decision function of assessment. It involves a set of key principles and ground rules that are important for its design and implementation. However, despite its intuitive appeal, its implementation remains a challenge. The purpose of this paper is to gain a better understanding of the factors that affect the implementation process of programmatic assessment and how specific implementation challenges are managed across different programs. METHODS: An explanatory multiple case (collective) approach was used for this study. We identified 6 medical programs that had implemented programmatic assessment with variation regarding health profession disciplines, level of education and geographic location. We conducted interviews with a key faculty member from each of the programs and analyzed the data using inductive thematic analysis. RESULTS: We identified two major factors in managing the challenges and complexity of the implementation process: knowledge brokers and a strategic opportunistic approach. Knowledge brokers were the people who drove and designed the implementation process acting by translating evidence into practice allowing for real-time management of the complex processes of implementation. These knowledge brokers used a 'strategic opportunistic' or agile approach to recognize new opportunities, secure leadership support, adapt to the context and take advantage of the unexpected. Engaging in an overall curriculum reform process was a critical factor for a successful implementation of programmatic assessment. DISCUSSION: The study contributes to the understanding of the intricacies of implementation processes of programmatic assessment across different institutions. Managing opportunities, adaptive planning, awareness of context, were all critical aspects of thinking strategically and opportunistically in the implementation of programmatic assessment. Future research is needed to provide a more in-depth understanding of values and beliefs that underpin the assessment culture of an organization, and how such values may affect implementation.

The pursuit of fairness in assessment: Looking beyond the objective.

Health professions education has undergone significant changes over the last few decades, including the rise of competency-based medical education, a shift to authentic workplace-based assessments, and increased emphasis on programmes of assessment. Despite these changes, there is still a commonly held assumption that objectivity always leads to and is the only way to achieve fairness in assessment. However, there are well-documented limitations to using objectivity as the 'gold standard' to which assessments are judged. Fairness, on the other hand, is a fundamental quality of assessment and a principle that almost no one contests. Taking a step back and changing perspectives to focus on fairness in assessment may help re-set a traditional objective approach and identify an equal role for subjective human judgement in assessment alongside objective methods. This paper explores fairness as a fundamental quality of assessments. This approach legitimises human judgement and shared subjectivity in assessment decisions alongside objective methods. Widening the answer to the question: 'What is fair assessment' to include not only objectivity but also expert human judgement and shared subjectivity can add significant value in ensuring learners are better equipped to be the health professionals required of the 21st century.

Locally adapting generic rubrics for the implementation of outcome-based medical education: a mixed-methods approach.

BACKGROUND: Rubrics are frequently used to assess competencies in outcome-based medical education (OBE). The implementation of assessment systems using rubrics is usually realised through years of involvement in projects with various stakeholders. However, for countries or specialities new to OBE, faster and more simplified processes are required. In March 2019, Japan introduced nine competencies and generic rubrics of competencies for medical residents. We explored the local adaptation of these generic rubrics and its consequences for assessors. METHODS: The study followed three steps. First, we locally adapted the generic rubrics. This was followed by conducting mixed-method research to explore the effect of the local adaptation. In step two, we examined the correlations between the scores in the locally adapted assessment sheets for supervising doctors and generic rubrics. In step three, we conducted interviews with supervising doctors. The study was conducted in the General Internal Medicine Department of Nagoya University, Japan. In the first step, doctors in the Medical Education Center and other medical departments, clerks, and residents participated. Supervising doctors in the General Internal Medicine Department participated in the second and third steps. RESULTS: A locally adapted assessment system was developed and implemented in seven months. The scores of the generic rubrics and the adapted assessment tool completed by the supervising doctors showed good correlations in some items as opposed to others, assessed mainly with other tools. Participant interviews revealed that local adaptation decreased their cognitive load leading to consistent ratings, increased writing of comments, and promoting reflection on instruction. CONCLUSIONS: This adaptation process is a feasible way to begin the implementation of OBE. Local adaptation has advantages over direct use of generic rubrics.

Perceived positive social interdependence in online versus face-to-face team-based learning styles of collaborative learning: a randomized, controlled, mixed-methods study.

BACKGROUND: Collaborative learning is a group learning approach in which positive social interdependence within a group is key to better learning performance and future attitudes toward team practice. Recent attempts to replace a face-to-face environment with an online one have been developed using information communication technology. However, this raises the concern that online collaborative learning (OCL) may reduce positive social interdependence. Therefore, this study aimed to compare the degree of social interdependence in OCL with face-to-face environments and clarify aspects that affect social interdependence in OCL. METHODS: We conducted a crossover study comparing online and face-to-face collaborative learning environments in a clinical reasoning class using team-based learning for medical students (n = 124) in 2021. The participants were randomly assigned to two cohorts: Cohort A began in an online environment, while Cohort B began in a face-to-face environment. At the study's midpoint, the two cohorts exchanged the environments as a washout. The participants completed surveys using the social interdependence in collaborative learning scale (SOCS) to measure their perceived positive social interdependence before and after the class. Changes in the mean SOCS scores were compared using paired t-tests. Qualitative data related to the characteristics of the online environment were obtained from the focus groups and coded using thematic analysis. RESULTS: The matched-pair tests of SOCS showed significant progression between pre- and post-program scores in the online and face-to-face groups. There were no significant differences in overall SOCS scores between the two groups. Sub-analysis by subcategory showed significant improvement in boundary (discontinuities among individuals) and means interdependence (resources, roles, and tasks) in both groups, but outcome interdependence (goals and rewards) improved significantly only in the online group. Qualitative analysis revealed four major themes affecting social interdependence in OCL: communication, task-sharing process, perception of other groups, and working facilities. CONCLUSIONS: There is a difference in the communication styles of students in face-to-face and online environments, and these various influences equalize the social interdependence in a face-to-face and online environment.

Shaping the right conditions in programmatic assessment: how quality of narrative information affects the quality of high-stakes decision-making.

BACKGROUND: Programmatic assessment is increasingly being implemented within competency-based health professions education. In this approach a multitude of low-stakes assessment activities are aggregated into a holistic high-stakes decision on the student's performance. High-stakes decisions need to be of high quality. Part of this quality is whether an examiner perceives saturation of information when making a holistic decision. The purpose of this study was to explore the influence of narrative information in perceiving saturation of information during the interpretative process of high-stakes decision-making. METHODS: In this mixed-method intervention study the quality of the recorded narrative information was manipulated within multiple portfolios (i.e., feedback and reflection) to investigate its influence on 1) the perception of saturation of information and 2) the examiner's interpretative approach in making a high-stakes decision. Data were collected through surveys, screen recordings of the portfolio assessments, and semi-structured interviews. Descriptive statistics and template analysis were applied to analyze the data. RESULTS: The examiners perceived less frequently saturation of information in the portfolios with low quality of narrative feedback. Additionally, they mentioned consistency of information as a factor that influenced their perception of saturation of information. Even though in general they had their idiosyncratic approach to assessing a portfolio, variations were present caused by certain triggers, such as noticeable deviations in the student's performance and quality of narrative feedback. CONCLUSION: The perception of saturation of information seemed to be influenced by the quality of the narrative feedback and, to a lesser extent, by the quality of reflection. These results emphasize the importance of high-quality narrative feedback in making robust decisions within portfolios that are expected to be more difficult to assess. Furthermore, within these "difficult" portfolios, examiners adapted their interpretative process reacting on the intervention and other triggers by means of an iterative and responsive approach.

Supervisory dyads' communication and alignment regarding the use of workplace-based observations: a qualitative study in general practice residency.

BACKGROUND: In medical residency, performance observations are considered an important strategy to monitor competence development, provide feedback and warrant patient safety. The aim of this study was to gain insight into whether and how supervisor-resident dyads build a working repertoire regarding the use of observations, and how they discuss and align goals and approaches to observation in particular. METHODS: We used a qualitative, social constructivist approach to explore if and how supervisory dyads work towards alignment of goals and preferred approaches to performance observations. We conducted semi-structured interviews with supervisor-resident dyads, performing a template analysis of the data thus obtained. RESULTS: The supervisory dyads did not frequently communicate about the use of observations, except at the start of training and unless they were triggered by internal or external factors. Their working repertoire regarding the use of observations seemed to be primarily driven by patient safety goals and institutional assessment requirements rather than by providing developmental feedback. Although intended as formative, the institutional test was perceived as summative by supervisors and residents, and led to teaching to the test rather than educating for purposes of competence development. CONCLUSIONS: To unlock the full educational potential of performance observations, and to foster the development of an educational alliance, it is essential that supervisory dyads and the training institute communicate clearly about these observations and the role of assessment practices of- and for learning, in order to align their goals and respective approaches.

Modeling the effect of social interdependence in interprofessional collaborative learning.

Interprofessional education (IPE) often uses collaborative learning for better teamwork among multiple professions. Its theoretical background is social interdependence theory (SIT), and positive interdependence is key for successful collaborative learning. As there is little theoretical knowledge on how to optimize the social interdependence in IPE, educators often struggle to develop an effective program. Therefore, a more specific explanation of the relationship between social interdependence and interprofessional readiness might make IPE more effective. We aimed to elucidate how students' social interdependence in collaborative learning relates to interprofessional readiness. Health profession students (n = 259) undertaking a collaborative IPE program were asked to complete two validated questionnaires: Readiness for Interprofessional Learning Scale and Social Interdependence Scale in Collaborative Learning. A structural equation analysis provided a good fit for the data. Awareness of objectives positively predicted interprofessional collaboration and identity, whereas specific roles as health professionals negatively predicted interprofessional roles and identity. As the current healthcare system is highly specialized, there is a dilemma that the more specific a problem is, the clearer it is which profession is responsible for solving it. Hence, strengthening awareness collaboratively may overcome the dilemma. Also, it is necessary to reconstruct a curriculum based on the premise of interprofessional activities.

Ottawa 2020 consensus statement for programmatic assessment - 1. Agreement on the principles.

INTRODUCTION: In the Ottawa 2018 Consensus framework for good assessment, a set of criteria was presented for systems of assessment. Currently, programmatic assessment is being established in an increasing number of programmes. In this Ottawa 2020 consensus statement for programmatic assessment insights from practice and research are used to define the principles of programmatic assessment. METHODS: For fifteen programmes in health professions education affiliated with members of an expert group (n = 20), an inventory was completed for the perceived components, rationale, and importance of a programmatic assessment design. Input from attendees of a programmatic assessment workshop and symposium at the 2020 Ottawa conference was included. The outcome is discussed in concurrence with current theory and research. RESULTS AND DISCUSSION: Twelve principles are presented that are considered as important and recognisable facets of programmatic assessment. Overall these principles were used in the curriculum and assessment design, albeit with a range of approaches and rigor, suggesting that programmatic assessment is an achievable education and assessment model, embedded both in practice and research. Knowledge on and sharing how programmatic assessment is being operationalized may help support educators charting their own implementation journey of programmatic assessment in their respective programmes.

Ottawa 2020 consensus statements for programmatic assessment - 2. Implementation and practice.

INTRODUCTION: Programmatic assessment is a longitudinal, developmental approach that fosters and harnesses the learning function of assessment. Yet the implementation, a critical step to translate theory into practice, can be challenging. As part of the Ottawa 2020 consensus statement on programmatic assessment, we sought to provide descriptions of the implementation of the 12 principles of programmatic assessment and to gain insight into enablers and barriers across different institutions and contexts. METHODS: After the 2020 Ottawa conference, we surveyed 15 Health Profession Education programmes from six different countries about the implementation of the 12 principles of programmatic assessment. Survey responses were analysed using a deductive thematic analysis. RESULTS AND DISCUSSION: A wide range of implementations were reported although the principles remained, for the most part, faithful to the original enunciation and rationale. Enablers included strong leadership support, ongoing faculty development, providing students with clear expectations about assessment, simultaneous curriculum renewal and organisational commitment to change. Most barriers were related to the need for a paradigm shift in the culture of assessment. Descriptions of implementations in relation to the theoretical principles, across multiple educational contexts, coupled with explanations of enablers and barriers, provided new insights and a clearer understanding of the strategic and operational considerations in the implementation of programmatic assessment. Future research is needed to further explore how contextual and cultural factors affect implementation.

Contextual attributes to promote positive social interdependence in problem-based learning: a focus group study.

BACKGROUND: Problem-based learning (PBL) is classified as a collaborative learning approach, wherein students learn while contributing meaning to experiences and interactions with others. An important theoretical fundament of PBL is social interdependence theory (SIT) because positive social interdependence within a group has been found to be key to better learning performance and future attitudes towards team practice. However, most previous studies in health professions education focused on cognitive outcomes, and few studies have focused on collaborative behaviors in PBL groups. The lack of this empirical insight makes implementation of PBL difficult, especially in contexts where there is limited experience with collaborative learning. Therefore, the aim of this study was to elucidate what promotes or hinders positive social interdependence and how the attributes work during PBL. METHODS: We conducted four focus groups among clinical year medical students (n = 26) who participated in PBL tutorials in the formal curriculum. We asked semi-structured questions that corresponded with the overall concept of SIT. We analyzed the transcript using constructivist grounded theory and developed a model to explain contextual attributes that promote or hinder positive social interdependence in PBL. RESULTS: Two contextual attributes of "academic inquisition" and "desire for efficiency" affect social interdependence among a student group in PBL. Academic inquisition is students' desire to engage in their academic learning, and desire for efficiency is students' attitude toward learning as an imposed duty and desire to complete it as quickly as possible. These attributes are initially mutually conflicting and constructing social interdependence through multiple steps including inquisition from a case, seeking efficient work, sharing interest in problem solving, expecting mutual contributions, and complementing learning objectives. CONCLUSION: These findings will contribute to understanding collaborative learning environments in PBL and may help explain contexts where PBL is less successful. The model can also be used as a tool to support innovation of PBL as collaborative learning.

Extending the use of the conceptions of learning and teaching (COLT) instrument to the postgraduate setting.

BACKGROUND: Several studies have shown that conceptions of teachers on teaching and learning can influence the teaching practices and behavior in higher education. This association is also found in undergraduate medical education but not yet established in postgraduate medical setting. An instrument, Conceptions of Learning and Teaching (COLT) was developed to measure conception of teachers in undergraduate medical education. COLT is a 3-factor 18-item questionnaire. The objective of this study is to evaluate if COLT is valid for postgraduate medical education. METHODS: We invited postgraduate clinical faculty from 3 hospitals in the Netherlands to fill out the COLT. Confirmatory and exploratory factor analysis were performed to evaluate the fit of the postgraduate clinical faculty data to the COLT. Analysis of variance was done to evaluate if there was difference among the 3 hospitals in terms of the response by the clinical faculty. RESULTS: Confirmatory factor analysis showed that the postgraduate faculty data had a 2 factor structure after removal of five items. These factors were Teacher Centeredness (TC) and combined Appreciation of Active Learning and Orientation to Professional Practice (A-P) and were considered as comparable to the factors in the original COLT, expressing the post-graduate learning and teaching setting. As several items were removed, the fit was suboptimal, yet did suggest validity for use of the COLT for postgraduate medical education. CONCLUSION: The modified COLT can be used to measure conceptions of teaching and learning in postgraduate medical education. We recommend further study to improve the factor structure of the modified COLT.

Limited effects from professional identity formation-oriented intervention on self-regulated learning in a preclinical setting: a randomized-controlled study in Japan.

BACKGROUND: Developing self-regulated learning in preclinical settings is important for future lifelong learning. Previous studies indicate professional identity formation, i.e., formation of self-identity with internalized values and norms of professionalism, might promote self-regulated learning. We designed a professional identity formation-oriented reflection and learning plan format, then tested effectiveness on raising self-regulated learning in a preclinical year curriculum. METHODS: A randomized controlled crossover trial was conducted using 112 students at Jichi Medical University. In six one-day problem-based learning sessions in a 7-month pre-clinical year curriculum, Groups A (n = 56, female 18, mean age 21.5y ± 0.7) and B (n = 56, female 11, mean age 21.7y ± 1.0) experienced professional identity formation-oriented format: Group A had three sessions with the intervention format in the first half, B in the second half. Between-group identity stages and self-regulated learning levels were compared using professional identity essays and the Motivated Strategies for Learning Questionnaire. RESULTS: Two-level regression analyses showed no improvement in questionnaire categories but moderate improvement of professional identity stages over time (R2 = 0.069), regardless of timing of intervention. CONCLUSIONS: Professional identity moderately forms during the pre-clinical year curriculum. However, neither identity nor self-regulated learning is raised significantly by limited intervention.

Between trust and control: Teachers' assessment conceptualisations within programmatic assessment.

OBJECTIVES: Programmatic assessment attempts to facilitate learning through individual assessments designed to be of low-stakes and used only for high-stake decisions when aggregated. In practice, low-stake assessments have yet to reach their potential as catalysts for learning. We explored how teachers conceptualise assessments within programmatic assessment and how they engage with learners in assessment relationships. METHODS: We used a constructivist grounded theory approach to explore teachers' assessment conceptualisations and assessment relationships in the context of programmatic assessment. We conducted 23 semi-structured interviews at two different graduate-entry medical training programmes following a purposeful sampling approach. Data collection and analysis were conducted iteratively until we reached theoretical sufficiency. We identified themes using a process of constant comparison. RESULTS: Results showed that teachers conceptualise low-stake assessments in three different ways: to stimulate and facilitate learning; to prepare learners for the next step, and to use as feedback to gauge the teacher's own effectiveness. Teachers intended to engage in and preserve safe, yet professional and productive working relationships with learners to enable assessment for learning when securing high-quality performance and achievement of standards. When teachers' assessment conceptualisations were more focused on accounting conceptions, this risked creating tension in the teacher-learner assessment relationship. Teachers struggled between taking control and allowing learners' independence. CONCLUSIONS: Teachers believe programmatic assessment can have a positive impact on both teaching and student learning. However, teachers' conceptualisations of low-stake assessments are not focused solely on learning and also involve stakes for teachers. Sampling across different assessments and the introduction of progress committees were identified as important design features to support teachers and preserve the benefits of prolonged engagement in assessment relationships. These insights contribute to the design of effective implementations of programmatic assessment within the medical education context.