Developing Comprehensive Strategies to Evaluate Medical School Curricula.

Evaluation of medical school curriculum is important to document outcomes, effectiveness of learning, engagement in quality improvement, and to meet accreditation compliance. This monograph provides a roadmap and resource for medical schools to meaningfully evaluate their curriculum based on specific metrics. The method of evaluation includes an examination of Kirkpatrick's levels of outcomes including reactions, learning, behavior, and impact. It is important that student outcomes are mapped in relation to curricular objectives. There are specific outcomes that may be utilized to determine if the curriculum has met the institution's goals. The first is comparison to national metrics (United States Medical Licensing Examinations and American Association of Medical Colleges Graduation Questionnaire). Second, medical schools collect internal program metrics, which include specific student performance metrics, such as number of students graduating, attrition, and matching to specialty. Further, schools may examine student performance and surveys in the preclerkship and clinical phases (e.g., grades, failing courses, survey responses about the curriculum), including qualitative responses on surveys or focus groups. As the learning environment is critical to learning, a deep dive to understand the environment and mistreatment may be important for program evaluation. This may be performed by specifically examining the Graduation Questionnaire, internal surveys, and mistreatment reporting. Finally, there are numerous attitudinal instruments that may help medical schools understand their students' development at one point or over time. These include measurements of stress, wellness, burnout, lifelong learning, and attitudes toward patient safety. Together, examining the composite of outcomes helps to understand and improve the medical school curriculum.

Training wheels needed: Lessons in professionalism from a liberal deferral policy.

Learning to self-regulate is an important aspect of professionalism. Thus, in 2015-16, the University of Michigan implemented a learner-centred 'deferral' policy called 'trust and track' in the preclinical phase. This gave students the autonomy to decide whether to attend required experiences, take quizzes and exams on schedule, or submit assignments on time. Surprisingly, quiz and exam deferrals remained relatively stable, but required experience deferrals more than doubled. While late assignments were not specifically tracked, there were multiple reports of assignments being months overdue. Some reasons for deferrals exhibited questionable judgement. Behavioural patterns carried forward, with an unusual spike in deferrals of licensure exams and requests for time off in the clinical phase. Wellness indices did not improve, despite learners having more autonomy and flexibility. It became clear to us that novice learners need clear professional expectations with limits to assist in developing professional behaviours. In 2016-17, we implemented a stricter policy that set clear expectations, established limits, and provided guidance on acceptable reasons to defer. We simultaneously implemented other measures to promote wellness. The moral of the story is that 'training wheels' are needed to help early learners develop the professional behaviours expected of practising physicians.

Evaluation of Feedback Systems for the Third-Year Surgical Clerkship.

OBJECTIVE: To compare faculty-to-student feedback rates from 2 different data sets: direct observation cards (direct evidence) and end-of-clerkship questionnaires (ECQs, secondary student reporting). We as authors hypothesized that direct evidence from observation cards would yield higher rates of feedback, compared with student-reported ECQs. DESIGN: Data were gathered from 2 consecutive medical classes of the third-year surgical clerkship. Data were analyzed retrospectively. Subjects and authors were blinded during data collection and analysis. SETTING: University of Michigan Medical School (UMMS) and University of Michigan Health System (UMHS). UMHS is an academic tertiary care center, located in Ann Arbor, Michigan. PARTICIPANTS: Data were gathered from all third-year medical students completing the surgical clerkship from 2012 to 2014. Data were available for 309 students, compared to 313 students who graduated from UMMS during that time. RESULTS: Directly measured rates of feedback showed higher rates of midclerkship feedback (95.43% vs 83.57%, p < 0.05), feedback on an observed history-taking (97.62% vs 84.87%, p < 0.05), and feedback on an observed physical examination (99.67% vs 79.71%, p < 0.05), when compared with the ECQ. CONCLUSION: These data suggest that solely using ECQs to assess feedback rates may be inadequate. The use of multiple methods to assess feedback may therefore be a prudent choice for surgical clerkships.