Criterion-Based Assessment in a Norm-Based World: How Can We Move Past Grades?

In the United States, the medical education community has begun a shift from the Flexnerian time-based model to a competency-based medical education model. The graduate medical education (GME) community is substantially farther along in this transition than is the undergraduate medical education (UME) community.GME has largely adopted the use of competencies and their attendant milestones and increasingly is employing the framework of entrustable professional activities (EPAs) to assess trainee competence. The UME community faces several challenges to successfully navigating a similar transition. First is the reliance on norm-based reference standards in the UME-GME transition, comparing students' performance versus their peers' with grades, United States Medical Licensing Examination Step 1 and Step 2 score interpretation, and the structured Medical School Performance Evaluation, or dean's letter. Second is the reliance on proxy assessments rather than direct observation of learners. Third is the emphasis on summative rather than formative assessments.Educators have overcome a major barrier to change by establishing UME outcomes assessment criteria with the advent and general acceptance of the physician competency reference set and the Core EPAs for Entering Residency in UME. Now is the time for the hard work of developing assessments steeped in direct observation that can be accepted by learners and faculty across the educational continuum and can be shown to predict clinical performance in a much more meaningful way than the current measures of grades and examinations. The acceptance of such assessments will facilitate the UME transition toward competency-based medical education.

Describing Failure in a Clinical Clerkship: Implications for Identification, Assessment and Remediation for Struggling Learners.

BACKGROUND: In addition to training future members of the profession, medical schools perform the critical role of identifying students who are failing to meet minimum standards in core competencies. OBJECTIVE: To better understand reasons for failure in an internal medicine clerkship. DESIGN: A qualitative content analysis of letters describing reasons for students' failure. PARTICIPANTS: Forty-three students (31 men) who failed the internal medicine clerkship at the University of Minnesota Medical School, 2002-2013. APPROACH: We conducted a qualitative content analysis of the 43 letters describing reasons for students' failure. We coded critical deficiencies and mapped them to the Physician Competency Reference Set (PCRS) competency domains and classified them into two categories: conduct (unprofessional behaviors) and knowledge and skills specific to the practice of medicine. We then calculated the frequency of each critical deficiency. We statistically tested for relationships between gender and critical deficiencies in each of the competency domains. KEY RESULTS: We coded 50 critical deficiencies with all codes mapping to a PCRS competency domain. The most frequently cited deficiencies were "insufficient knowledge" (79 % of students) and "inadequate patient presentation skills" (74 %). Students exhibited critical deficiencies in all eight competency domains, with the highest concentrations in Knowledge for Practice (98 %) and Interpersonal and Communication Skills (91 %). All students demonstrated deficiencies in multiple competencies, with 98 % having deficiencies in three or more. All 43 students demonstrated deficits in the knowledge and skills category, and 81 % had concurrent conduct issues. There were no statistically significant relationships between gender and critical deficiencies in any competency domain. CONCLUSIONS: This study highlights both the diversity and commonality of reasons that students fail a clinical clerkship. Knowing the range of areas where students struggle, as well as the most likely areas of difficulty, may aid faculty in identifying students who are failing and in developing remediation strategies.

Content and timing of feedback and reflection: a multi-center qualitative study of experienced bedside teachers.

BACKGROUND: Competency-based medical education increasingly recognizes the importance of observation, feedback, and reflection for trainee development. Although bedside rounds provide opportunities for authentic workplace-based implementation of feedback and team-based reflection strategies, this relationship has not been well described. The authors sought to understand the content and timing of feedback and team-based reflection provided by bedside teachers in the context of patient-centered bedside rounds. METHODS: The authors conducted a thematic analysis qualitative study using transcripts from audio-recorded, semi-structured telephone interviews with internal medicine attending physicians (n= 34) identified as respected bedside teachers from 10 academic US institutions (2010-2011). RESULTS: Half of the respondents (50%) were associate/full professors, with an average of 14 years of academic experience. In the context of bedside encounters, bedside teachers reported providing feedback on history-taking, physical-examination, and case-presentation skills, patient-centered communication, clinical decision-making, leadership, teaching skills, and professionalism. Positive feedback about physical-exam skills or clinical decision-making occurred during encounters, positive or constructive team-based feedback occurred immediately following encounters, and individualized constructive feedback occurred in one-on-one settings following rounding sessions. Compared to less frequent, emotionally-charged events, bedside teachers initiated team-based reflection on commonplace "teachable moments" related to patient characteristics or emotions, trainee actions and emotions, and attending physician role modeling. CONCLUSIONS: Bedside teachers use bedside rounds as a workplace-based method to provide assessment, feedback, and reflection, which are aligned with the goals of competency-based medical education. Embedded in patient-centered activities, clinical teachers should be encouraged to incorporate these content- and timing-related feedback and reflection strategies into their bedside teaching.

Identifying and overcoming the barriers to bedside rounds: a multicenter qualitative study.

PURPOSE: The use of bedside rounds in teaching hospitals has declined, despite recommendations from educational leaders to promote this effective teaching strategy. The authors sought to identify reasons for the decrease in bedside rounds, actual barriers to bedside rounds, methods to overcome trainee apprehensions, and proposed strategies to educate faculty. METHOD: A qualitative inductive thematic analysis using transcripts from audio-recorded, semistructured telephone interviews with a purposive sampling of 34 inpatient attending physicians from 10 academic U.S. institutions who met specific inclusion criteria for "bedside rounds" was performed in 2010. Main outcomes were themes pertaining to barriers, methods to overcome trainee apprehensions, and strategies to educate faculty. Quotations highlighting themes are reported. RESULTS: Half of respondents (50%) were associate or full professors, averaging 14 years in academic medicine. Primary reasons for the perceived decline in bedside rounds were physician- and systems related, although actual barriers encountered related to systems, time, and physician-specific issues. To address resident apprehensions, six themes were identified: build partnerships, create safe learning environments, overcome with experience, make bedside rounds educationally worthwhile, respect trainee time, and highlight positive impact on patient care. Potential strategies for educating faculty were identified, most commonly faculty development initiatives, divisional/departmental culture change, and one-on-one shadowing opportunities. CONCLUSIONS: Bedside teachers encountered primarily systems- and time-related barriers and overcame resident apprehensions by creating a learner-oriented environment. Strategies used by experienced bedside teachers can be used for faculty development aimed at promoting bedside rounds.

The value of bedside rounds: a multicenter qualitative study.

BACKGROUND: Bedside rounds have decreased on teaching services, raising concern about trainees' clinical skills and patient-physician relationships. PURPOSE: We sought to identify recognized bedside teachers' perceived value of bedside rounds to assist in the promotion of bedside rounds on teaching services. METHODS: Authors used a grounded theory, qualitative study design of telephone semistructured interviews with bedside teachers (n = 34) from 10 U.S. institutions (2010-2011). Main outcomes were characteristics of participants, themes pertaining to the perceived value of bedside rounds, and quotations highlighting each respective theme. RESULTS: The mean years in academic medicine was 13.7, and 51% were associate or full professors. Six main themes emerged: (a) skill development for learners (e.g., physical examination, communication, and clinical decision-making skills); (b) observation and feedback; (c) role-modeling; (d) team building among trainees, attending, and patient; (e) improved patient care delivery through combined clinical decision-making and team consensus; and (f) the culture of medicine as patient-centered care, which was embodied in all themes. CONCLUSIONS: Bedside teachers identify potential benefits of bedside rounds, many of which align with national calls to change our approach to medical education. The practice of bedside rounds enables activities essential to high-quality patient care and education.

The art of bedside rounds: a multi-center qualitative study of strategies used by experienced bedside teachers.

BACKGROUND: Medical educators believe bedside rounds are effective for the delivery of patient-centered care, and are necessary in helping trainees acquire competence in clinical care. Although recommendations for bedside rounds have been reported, a recent, systematic assessment of strategies used by current-day bedside teachers was needed to advance knowledge of this teaching method. OBJECTIVE: To identify and understand bedside teachers' 1) preparatory steps, 2) patient selection, and 3) role allocation during the process of bedside rounds. DESIGN: A qualitative inductive thematic analysis using transcripts from audio-recorded, semi-structured telephone interviews. PARTICIPANTS: Internal medicine physicians (n = 34) who perform bedside rounds from ten academic US institutions (2010-2011). APPROACH: A purposive sampling strategy was utilized to identify physicians who were active inpatient attending physicians and met specific inclusion criteria for "bedside rounds." A total of 34 interviews were completed, and each was recorded and transcribed verbatim. A thematic analysis extracted key themes pertaining to the three objectives. KEY RESULTS: Most respondents (51 %) were associate or full professors, with an average of 14 years of academic experience. Attending physicians prepared using trainee-specific, patient-specific and disease-specific information, while also mentally preparing for bedside rounds. They sought trainee buy-in and learning objectives, reviewed expectations and methods to ensure patient comfort, and provided early guidance with bedside encounters. Patients were selected if they required immediate care, were new to the service, or had a high educational value, while patients were deferred if unavailable, unwilling, or unable to communicate. The team members' roles during bedside rounds varied, with trainees being given graduated autonomy with increased experience. CONCLUSIONS: Bedside teachers' methods for preparation, patient selection, and role allocation during bedside rounds enhance trainees' education within the workplace. Strategies used by experienced bedside teachers can be used for faculty development efforts aimed at promoting this activity.