Managing assessment during curriculum change: Ottawa Consensus Statement.

Curriculum change is relatively frequent in health professional education. Formal, planned curriculum review must be conducted periodically to incorporate new knowledge and skills, changing teaching and learning methods or changing roles and expectations of graduates. Unplanned curriculum evolution arguably happens continually, usually taking the form of "minor" changes that in combination over time may produce a substantially different programme. However, reviewing assessment practices is less likely to be a major consideration during curriculum change, overlooking the potential for unintended consequences for learning. This includes potentially undermining or negating the impact of even well-designed and important curriculum changes. Changes to any component of the curriculum "ecosystem "- graduate outcomes, content, delivery or assessment of learning - should trigger an automatic review of the whole ecosystem to maintain constructive alignment. Consideration of potential impact on assessment is essential to support curriculum change. Powerful contextual drivers of a curriculum include national examinations and programme accreditation, so each assessment programme sits within its own external context. Internal drivers are also important, such as adoption of new learning technologies and learning preferences of students and faculty. Achieving optimal and sustainable outcomes from a curriculum review requires strong governance and support, stakeholder engagement, curriculum and assessment expertise and internal quality assurance processes. This consensus paper provides guidance on managing assessment during curriculum change, building on evidence and the contributions of previous consensus papers.

Twelve tips for improving the quality of assessor judgements in senior medical student clinical assessments.

Assessment of senior medical students is usually calibrated at the level of achieving expected learning outcomes for graduation. Recent research reveals that clinical assessors often balance two slightly different perspectives on this benchmark. The first is the formal learning outcomes at graduation, ideally as part of a systematic, program-wide assessment approach that measures learning achievement, while the second is consideration of the candidate's contribution to safe care and readiness for practice as a junior doctor. The second is more intuitive to the workplace, based on experience working with junior doctors. This perspective may enhance authenticity in assessment decisions made in OSCEs and work-based assessments to better align judgements and feedback with professional expectations that will guide senior medical students and junior doctors' future career development. Modern assessment practices should include consideration of qualitative as well as quantitative information, overtly including perspectives of patients, employers, and regulators. This article presents 12 tips for how medical education faculty might support clinical assessors by capturing workplace expectations of first year medical graduates and develop graduate assessments based on a shared heuristic of 'work-readiness'. Peer-to-peer assessor interaction should be facilitated to achieve correct calibration that 'merges' the differing perspectives to produce a shared construct of an acceptable candidate.

Understanding medical students' transformative experiences of early preclinical international rural placement over a 20-year period.

INTRODUCTION: Rural placements are an important component of rural medical education programs seeking to develop rural practice pathways for medical students. These placements are usually domestic, but James Cook University in Australia developed an international rural placement program in the first half of the medical course that was funded through bursaries. This study explores how the international rural placement helped to shape the lives (personal development and learning) of the participants, using Transformational Learning Theory as a framework for identifying and describing the transformational elements, process and impact of the program. METHODS: Sixty-five students received a bursary for an international rural placement between 2001-2019. All were contacted by email and invited to participate in a short survey and a follow-up interview. Fifteen participants agreed and twelve were able to participate in individual semi-structured interviews which were recorded, transcribed and analysed using inductive thematic analysis. RESULTS: Participants reported that the bursary provided a "once in a lifetime opportunity" to "experience eye-opening and culturally rich difference". Nonetheless, some elements of the placement experience presented disorientating dilemmas that triggered deep reflections and shifts in perceptions. The bursary recipients realised that "being open-minded" allowed them "enjoy good company". They were also able to assume "outsider view which allowed reassessment of their own country" and the "isolation experiences gingered desire to right health wrongs". The triggers and mental shifts had significant impact on the bursary recipients and fostered the development of "inspirational new horizons" based on an appreciation of the "value of rural practice" and "role-modelling for life-long learning." These findings are consistent with Transformational Learning Theory. CONCLUSION: Participants in this study reported meaningful and strongly positive impacts from the experiences gained during an international rural clinical placement early in their course. They described transformative experiences which appear to contribute strongly to personal development. This finding supports maintaining opportunities for international experiences during rurally-oriented medical programs as these may impact longer term career choice.

Examiners' decision-making processes in observation-based clinical examinations.

BACKGROUND: Objective structured clinical examinations (OSCEs) are commonly used to assess the clinical skills of health professional students. Examiner judgement is one acknowledged source of variation in candidate marks. This paper reports an exploration of examiner decision making to better characterise the cognitive processes and workload associated with making judgements of clinical performance in exit-level OSCEs. METHODS: Fifty-five examiners for exit-level OSCEs at five Australian medical schools completed a NASA Task Load Index (TLX) measure of cognitive load and participated in focus group interviews immediately after the OSCE session. Discussions focused on how decisions were made for borderline and clear pass candidates. Interviews were transcribed, coded and thematically analysed. NASA TLX results were quantitatively analysed. RESULTS: Examiners self-reported higher cognitive workload levels when assessing a borderline candidate in comparison with a clear pass candidate. Further analysis revealed five major themes considered by examiners when marking candidate performance in an OSCE: (a) use of marking criteria as a source of reassurance; (b) difficulty adhering to the marking sheet under certain conditions; (c) demeanour of candidates; (d) patient safety, and (e) calibration using a mental construct of the 'mythical [prototypical] intern'. Examiners demonstrated particularly higher mental demand when assessing borderline compared to clear pass candidates. CONCLUSIONS: Examiners demonstrate that judging candidate performance is a complex, cognitively difficult task, particularly when performance is of borderline or lower standard. At programme exit level, examiners intuitively want to rate candidates against a construct of a prototypical graduate when marking criteria appear not to describe both what and how a passing candidate should demonstrate when completing clinical tasks. This construct should be shared, agreed upon and aligned with marking criteria to best guide examiner training and calibration. Achieving this integration may improve the accuracy and consistency of examiner judgements and reduce cognitive workload.

Twelve tips for expanding undergraduate clinical teaching capacity.

Undergraduate medical education has expanded substantially in recent years, through both establishing new programs and increasing student numbers in existing programs. This expansion has placed pressure on the capacity for training students in clinical placements, raising concerns about the risk of dilution of experience, and reducing work readiness. The concerns have been greatest in more traditional environments, where clinical placements in large academic medical centers are often the "gold standard". However, there are ways of exposing medical students to patient interactions and clinical supervisors in many other contexts. In this paper, we share our experiences and observations of expanding clinical placements for both existing and new medical programs in several international locations. While this is not necessarily an easy task, a wide range of opportunities can be accessed by asking the right questions of the right people, often with only relatively modest changes in resource allocation.

Teaching undergraduate students in rural general practice: an evaluation of a new rural campus in England.

INTRODUCTION: One approach to facilitating student interactions with patient pathways at Keele University School of Medicine, England, is the placement of medical students for 25% of their clinical placement time in general practices. The largest component is a 15-week 'student attachment' in primary care during the final year, which required the development of a new network of teaching practices in a rural district of England about 90 km (60 mi) from the main campus in North Staffordshire. The new accommodation and education hub was established in 2011-2012 to enable students to become immersed in those communities and learn about medical practice within a rural and remote context. Objectives were to evaluate the rural teaching from the perspectives of four groups: patients, general practice tutors, community hospital staff and students. Learning outcomes (as measured by objective structured clinical examinations) of students learning in rural practices in the final year were compared with those in other practices. METHODS: Data were gathered from a variety of sources. Students' scores in cohort-wide clinical assessment were compared with those in other locations. Semi-structured interviews were conducted with general practice tutors and community hospital staff. Serial focus groups explored the perceptions of the students, and questionnaires were used to gather the views of patients. RESULTS: Patients reported positive experiences of students in their consultations, with 97% expressing willingness to see students. The majority of patients considered that teaching in general practice was a good thing. They also expressed altruistic ideas about facilitating learning. The tutors were enthusiastic and perceived that teaching had positive impacts on their practices despite negative effects on their workload. The community hospital staff welcomed students and expressed altruistic ideas about helping them learn. There was no significant difference between the rurally placed students' objective structured clinical examination performance and that of their peers in other locations. Some students had difficulty with the isolation from peers and academic activities, and travel was a problem despite their accommodation close to the practices. CONCLUSIONS: Students valued the learning opportunities offered by the rural practice placements. The general practice tutors, patients and community hospital staff found teaching to be a positive experience overall and perceived a value to the health system and broader community in students learning locally for substantial periods of time. The evaluation has identified some student concerns about transport times and costs, social isolation, and access to resources and administrative tasks, and these are being addressed.

Full medical program fees and medical student career intention.

OBJECTIVE: To explore the future career preferences of Commonwealth-supported place (CSP) and full-fee paying (FFP) medical students in Australia. DESIGN, SETTING AND PARTICIPANTS: Data from the Medical Schools Outcomes Database and Longitudinal Tracking (MSOD) Project exit questionnaire for CSP and FFP students who graduated between 2008 and 2011 were analysed using logistic regression. The influence of age, sex, marital status, rural background and fee-paying status on future career preference were explored. MAIN OUTCOME MEASURE: Future career preference (location and specialty) at graduation. RESULTS: Compared with CSP students, domestic FFP students were more likely to nominate as their first preference both urban locations (odds ratio [OR], 5.58; 95% CI, 2.04-15.26; P < 0.001) and higher-income specialties (OR, 1.37; 95% CI, 1.07-1.75; P < 0.05), and less likely to nominate as their first preference in-need specialties (OR, 0.72; 95% CI, 0.52-1.00; P < 0.05), specifically general practice (OR, 0.71; 95% CI, 0.52-0.99; P < 0.05). There was a significant domestic FFP student by marital status interaction effect, such that domestic FFP students who were married or partnered on exit from medical school were more likely to prefer a rural location (OR, 0.64; 95% CI, 0.44-0.95; P < 0.05). Also, students who were married or partnered were less likely to select a one of the higher-income specialties as their first preference (OR, 0.77; 95% CI, 0.64-0.92; P < 0.01). A rural background increased preferences for rural location (OR, 0.18; 95% CI, 0.15-0.22; P < 0.001) and in-need specialties (OR, 1.28; 95% CI, 1.04-1.57; P < 0.05), and being older on entry to medical school also increased preferences for rural location (OR, 0.96; 95% CI, 0.95-0.98; P < 0.001) and in-need specialties (OR, 1.03; 95% CI, 1.01-1.04; P < 0.01). International FFP students were more likely to prefer urban practice (OR, 1.79; 95% CI, 1.19-2.72; P < 0.01). CONCLUSION: Domestic FFP graduates are less likely to prefer careers in rural locations and in lower-paid and in-need specialties, particularly general practice. Current workforce implications might be minor, but if fees for CSP students increase or more FFP places become available, potential impacts on workforce distribution will need to be considered.

Twelve tips for increasing the defensibility of assessment decisions.

In an era of increasing scrutiny of the performance of graduates in the workplace, there can be frustrations when decisions about learners with borderline or poor performance in formal assessments are over-turned in appeal processes. This article addresses the approach to reducing the uncertainty about assessment decisions and surviving appeals.

Utilisation of the healthcare system for authentic early experience placements.

INTRODUCTION: Authentic early experience in clinical contexts adds interest and relevance to basic medical education, and is regarded positively by both learners and teachers. However, with the recent expansion of medical education, the healthcare system appears close to reaching its capacity for student supervision. This study explores the utilisation of the healthcare system for early clinical placements. METHODS: A secondary analysis was conducted of data from the Medical Schools Outcomes Database, collected from the 2010 annual questionnaire, focusing on the timing, duration and location of clinical placements during 2009 within the first half of basic medical education programs in Australia. RESULTS: Data was received for 67% of Australian medical students, reporting a total of 16 812 early clinical placements that occupied 97 319 days of supervised time in a wide variety of hospital, general practice and Indigenous health contexts, both urban and rural, across the Australian healthcare system. These early placements occupied about 16% of total clinical placement time for all students in all training years during 2009. The majority of these placements were for only a few hours or days; exceptions were longitudinal placements in regional and rural communities at a minority of schools. CONCLUSION: Early clinical placements may pose significant resource costs for placement providers, particularly supervision time and expertise. As medical education expands and the teaching capacity of the Australian healthcare system appears to reach its limits, it may be necessary to allocate placements according to their specific learning outcomes, prioritise more acute settings for more senior students, and increase capacity in less acute health and social care settings.

"Could You Work in My Team?": Exploring How Professional Clinical Role Expectations Influence Decision-Making of Assessors During Exit-Level Medical School OSCEs.

Decision-making in clinical assessment, such as exit-level medical school Objective Structured Clinical Examinations (OSCEs), is complex. This study utilized an empirical phenomenological qualitative approach with thematic analysis to explore OSCE assessors' perceptions of the concept of a "prototypical intern" expressed during focus group discussions. Topics discussed included the concept of a prototypical intern, qualities to be assessed, and approaches to clinical assessment decision-making. The thematic analysis was then applied to a theoretical framework (Cultural Historical Activity Theory-CHAT) that explored the complexity of making assessment decisions amidst potentially contradicting pressures from academic and clinical perspectives. Ten Australasian medical schools were involved with 15 experienced and five less experienced assessors participating. Thematic analysis of the data revealed four major themes in relation to how the prototypical intern concept influences clinical assessors' judgements: (a) Suitability of marking rubric based on assessor characteristics and expectations; (b) Competence as final year student vs. performance as a prototypical intern; (c) Safety, trustworthiness and reliability as constructs requiring assessment and (d) Contradictions in decision making process due to assessor differences. These themes mapped well within the interaction between two proposed activity systems in the CHAT model: academic and clinical. More clinically engaged and more experienced assessors tend to fall back on a heuristic, mental construct of a "prototypical intern," to calibrate judgements, particularly, in difficult situations. Further research is needed to explore whether consensus on desirable intern qualities and their inclusion into OSCE marksheets decreases the cognitive load and increases the validity of assessor decision making.

Australian and overseas models of general practice training.

General practice training in Australia continues to evolve. It is now the responsibility of an independent organisation, is delivered by regional training providers, and comprises a structured training program. Overseas, general practice varies in its importance to health care systems, and training models differ considerably. In some cases training is mandatory, in others voluntary, but the aim is always similar--to improve the quality of care delivered to the large majority of populations that access health care through primary care. We review the current status of vocational general practice training in Australia, compare it with selected training programs in international contexts, and describe how the local model is well placed to address future challenges. Challenges include changes in population demographics, increasing comorbidity, increasing costs of technology-based health care, increasing globalisation of health, and workforce shortages. Although general practice training in Australia is strong, it can improve further by learning from other training programs to meet these challengers.

Problems presented by medical students seeking support: a possible intervention framework.

BACKGROUND: Academically bright and ambitious medical students must cope with a combination of curriculum, assessment, career choice, personal, family and social pressures. Many seek support, and some present complex problems that consume substantial resources. A few continue unresolved into postgraduate life and may affect future professional performance. Early intervention may be helpful in minimising the impact on both individual students and school resources. AIM: To develop a broad classification framework for personal problems presented by medical students to student support services. METHODS: We report a study of more complex student problems presenting to student support services. A three stage iterative process, involving student support faculty from several international medical schools, discussed case scenarios based on pooled experience and developed a framework for categorising and considering management. RESULTS: Seven profiles of student problems were developed, based on composites of unidentified cases drawn from participants' experiences, reflecting: immaturity; learning skills; organisational skills; health or personal issues and poor insight. Broad characteristics and brief prognostic comments are provided for each profile. DISCUSSION: While profiles may overlap and each student may have a unique set of issues, there appears to be similarities that allow some categorisation for management strategies and prognostic features in both undergraduate and postgraduate medical education. Further work is needed to develop 'diagnostic' assessments that might guide intervention. CONCLUSION: This classification may assist early intervention into personal problems presented by medical students to student support services.

Perceptions of UK medical students on rural clinical placements.

INTRODUCTION: Rural clinical placements are now commonly used to both promote awareness of rural health careers and expand clinical placement networks in Australia, North America and elsewhere. However in the United Kingdom (UK) there is no clear workforce or health education policy that encourages rural background student recruitment, rural clinical placements, rural oriented curricula or rural health education infrastructure, because deprivation and poor health status are regarded as urban phenomena. The question for one new medical school in regional/rural UK is: can increasing the utilisation of rural primary care practices both resolve teaching capacity constraints and offer students valuable learning opportunities about rural health? This article reports an exploration of students' views on the value of rural clinical placements in a new curriculum designed to address regional deprivation and workforce needs. METHODS: Medical students in Year 1 and Year 3 of a regional medical school were invited to attend focus group discussions that explored their understanding of rural health and life and the attractions of and barriers to expanding rural clinical placements. The Year 1 students were in the new curriculum and therefore more likely to be allocated a rural clinical placement in their more senior years. The discussions were audio-recorded, transcribed and thematically analysed. RESULTS: Themes that emerged were: what is rural? how different is rural from urban?; differences in rural and urban learning, logistic issues, and choosing a rural placement. Student perceptions in both groups were rather negative about rural placements. Rural practices were thought to provide a narrow range of patient contact and learning opportunities, and rural life was thought to be unattractive, especially out of formal placement hours. Even relatively small distances from friends and social outlets were regarded as barriers. Year 1 students were more positive about the possibility of a rural placement, although they knew at admission that the school would be expanding into rural communities and may therefore have been more open to the concept. CONCLUSION: Ruralization of health professional education in the UK faces substantial challenges. More may need to be done to increase recruitment of rural-interested students, foster rural career interest during courses, recruit rural role models, develop rural curricula and establish the infrastructure to support rural clinical placements. These initiatives will require greater political will and some investment by education, heath and community development agencies.

Transition from an MBBS to an MD program - Using innovation and thinking outside the square.

This article was migrated. The article was marked as recommended. Background There has been a trend globally to move from a Bachelor of Medicine, Bachelor of Surgery (MBBS) to a Doctor of Medicine (MD) for primary medical education. This shift has seen many Australian universities change to an MD, mostly from graduate entry programs. This paper describes the novel and unique 3+2 model from one Australian university, that enabled undergraduate entry, student flexibility, and a master's exit qualification without increasing time. Methods The method included a curriculum review in 2013 where its problem-based learning curriculum shifted from a seven to a five-semester program; changing the third year to a virtual hospital clinical year using simulation, and introducing in 2016 a new 3+2 curriculum model in the final two years using a 100 point system as a masters level program. Results The MD model was described in the external evaluation as 'novel and innovative', where students can choose from three project options - a research project, or a professional project or an international capstone experience as well as a number of scholarly tasks. The structure is fully integrated with the existing curriculum and assessment process, supported by an innovative technology platform. Conclusion Now in its third year of implementation this innovative model is breaking new ground in the way in which a masters level MD program could be developed, whilst maintaining undergraduate entry.

Lessons Learnt from the Development and Implementation of an Online Assessment System for Medical Science Programmes.

Challenges in assessment implementation include assessment blueprinting, accuracy and timeliness of result processing, item analysis and personalised feedback. These challenges were addressed by an online assessment system (OAS) that translates the educational framework of outcome-based education (OBE) into an integrated platform applicable across academic programmes. OBE principles of transparency, measurability, relevance, and individualisation was feasibly addressed using technology. The system development required continuous stakeholder engagement and feedback. Its implementation involved change management at institutional level with shifts in procedures, responsibilities, staff competencies and resource allocation. This article describes approaches taken and the lessons learned in developing and implementing OAS to enhance assessment practice for medical science programmes.

Interprofessional education in rural practice: how, when and where?

Interprofessional education (IPE) has been suggested as an answer to improving the effectiveness of health professional teamwork, which in turn is regarded as a key strategy for improving the delivery and outcomes of increasingly complex healthcare approaches. There is a strong theoretical base to support the implementation of IPE for all health professionals, and in response many training programs now do this, although in a wide variety of ways. There is, however, little evidence so far that IPE has the desired effect, and one reason for this may be the design of the IPE learning activities. This article presents some theory-based but practical advice for how to develop effective IPE activities. The focus is on rural practice, which is an ideal location for IPE because small teams must work together in small communities to provide optimal health care.

Rural medical education in Europe: the relevance of the Australian experience.

The drive to increase recognition of the different health care needs of rural communities has been more successful in North America and Australia than in Europe. This success has translated into political support for a range of education, workforce and service model initiatives that appear to be effective in providing a better prepared and supported healthcare workforce in rural communities, providing services through specifically developed delivery models, all with the aim of improving the quality of health care for rural people. The reasons for the differences between Europe and nations with greater success appear to relate to: the absence of a clear, shared definition of rurality across a very diverse group of nations within Europe, and a weaker coalition of interests advocating rural health issues. As a result, although very similar rural health issues are present in Europe, governments are not particularly supportive of initiatives aimed specifically at rural health and so rural medical education is much less well developed. This article explores how the rural health movement in Europe might build on success elsewhere to develop and successfully promote locally relevant models of rural medical education.