Managing assessment during curriculum change: Ottawa Consensus Statement.

Curriculum change is relatively frequent in health professional education. Formal, planned curriculum review must be conducted periodically to incorporate new knowledge and skills, changing teaching and learning methods or changing roles and expectations of graduates. Unplanned curriculum evolution arguably happens continually, usually taking the form of "minor" changes that in combination over time may produce a substantially different programme. However, reviewing assessment practices is less likely to be a major consideration during curriculum change, overlooking the potential for unintended consequences for learning. This includes potentially undermining or negating the impact of even well-designed and important curriculum changes. Changes to any component of the curriculum "ecosystem "- graduate outcomes, content, delivery or assessment of learning - should trigger an automatic review of the whole ecosystem to maintain constructive alignment. Consideration of potential impact on assessment is essential to support curriculum change. Powerful contextual drivers of a curriculum include national examinations and programme accreditation, so each assessment programme sits within its own external context. Internal drivers are also important, such as adoption of new learning technologies and learning preferences of students and faculty. Achieving optimal and sustainable outcomes from a curriculum review requires strong governance and support, stakeholder engagement, curriculum and assessment expertise and internal quality assurance processes. This consensus paper provides guidance on managing assessment during curriculum change, building on evidence and the contributions of previous consensus papers.

2018 Consensus framework for good assessment.

Introduction: In 2010, the Ottawa Conference produced a set of consensus criteria for good assessment. These were well received and since then the working group monitored their use. As part of the 2010 report, it was recommended that consideration be given in the future to preparing similar criteria for systems of assessment. Recent developments in the field suggest that it would be timely to undertake that task and so the working group was reconvened, with changes in membership to reflect broad global representation.Methods: Consideration was given to whether the initially proposed criteria continued to be appropriate for single assessments and the group believed that they were. Consequently, we reiterate the criteria that apply to individual assessments and duplicate relevant portions of the 2010 report.Results and discussion: This paper also presents a new set of criteria that apply to systems of assessment and, recognizing the challenges of implementation, offers several issues for further consideration. Among these issues are the increasing diversity of candidates and programs, the importance of legal defensibility in high stakes assessments, globalization and the interest in portable recognition of medical training, and the interest among employers and patients in how medical education is delivered and how progression decisions are made.

The Associations Between United States Medical Licensing Examination Performance and Outcomes of Patient Care.

PURPOSE: The United States Medical Licensing Examination (USMLE) comprises a series of assessments required for the licensure of U.S. MD-trained graduates as well as those who are trained internationally. Demonstration of a relationship between these examinations and outcomes of care is desirable for a process seeking to provide patients with safe and effective health care. METHOD: This was a retrospective cohort study of 196,881 hospitalizations in Pennsylvania over a 3-year period (January 1, 2017 to December 31, 2019) for 5 primary diagnoses: heart failure, acute myocardial infarction, stroke, pneumonia, or chronic obstructive pulmonary disease. The 1,765 attending physicians for these hospitalizations self-identified as family physicians or general internists. A converted score based on USMLE Step 1, Step 2 Clinical Knowledge, and Step 3 scores was available, and the outcome measures were in-hospital mortality and log length of stay (LOS). The research team controlled for characteristics of patients, hospitals, and physicians. RESULTS: For in-hospital mortality, the adjusted odds ratio was 0.94 (95% confidence interval [CI] = 0.90, 0.99; P < .02). Each standard deviation increase in the converted score was associated with a 5.51% reduction in the odds of in-hospital mortality. For log LOS, the adjusted estimate was 0.99 (95% CI = 0.98, 0.99; P < .001). Each standard deviation increase in the converted score was associated with a 1.34% reduction in log LOS. CONCLUSIONS: Better provider USMLE performance was associated with lower in-hospital mortality and shorter log LOS for patients, although the magnitude of the latter is unlikely to be of practical significance. These findings add to the body of evidence that examines the validity of the USMLE licensure program.

Conference proceedings and consensus statements of the millennium conference 2007: a collaborative approach to educational research.

BACKGROUND: Research in medical education is stymied by a variety of methodological, logistical, and institutional challenges. Multicenter collaboration may advance the current state of medical education research by increasing subject sample size, harnessing the power of collective expertise, and garnering visibility for issues of national importance. SUMMARY: We convened the "Millennium Conference 2007: A Collaborative Approach to Educational Research" in May 2007 to consider the role of national initiatives in addressing challenges in educational research. We selected 9 medical schools through a competitive application process to participate as school teams. We led participants through structured discussions about the challenges of educational research and the opportunities offered through national collaboratives. School teams also met to refine their current local educational research initiatives. CONCLUSIONS: A group of committed stakeholders met to consider a national educational research agenda. The recommendations from this conference proceed from consensus reached by the participants.

Caring attitudes in medical education: perceptions of deans and curriculum leaders.

BACKGROUND: Systems of undergraduate medical education and patient care can create barriers to fostering caring attitudes. OBJECTIVE: The aim of this study is to survey associate deans and curriculum leaders about teaching and assessment of caring attitudes in their medical schools. PARTICIPANTS: The participants of this study include 134 leaders of medical education in the USA and Canada. METHODS: We developed a survey with 26 quantitative questions and 1 open-ended question. In September to October 2005, the Association of American Medical Colleges distributed it electronically to curricular leaders. We used descriptive statistics to analyze quantitative data, and the constant comparison technique for qualitative analysis. RESULTS: We received 73 responses from 134 medical schools. Most respondents believed that their schools strongly emphasized caring attitudes. At the same time, 35% thought caring attitudes were emphasized less than scientific knowledge. Frequently used methods to teach caring attitudes included small-group discussion and didactics in the preclinical years, role modeling and mentoring in the clinical years, and skills training with feedback throughout all years. Barriers to fostering caring attitudes included time and productivity pressures and lack of faculty development. Respondents with supportive learning environments were more likely to screen applicants' caring attitudes, encourage collaborative learning, give humanism awards to faculty, and provide faculty development that emphasized teaching of caring attitudes. CONCLUSIONS: The majority of educational leaders value caring attitudes, but overall, educational systems inconsistently foster them. Schools may facilitate caring learning environments by providing faculty development and support, by assessing students and applicants for caring attitudes, and by encouraging collaboration.