Examining the Educational Value of Student-Run Clinics for Health Care Students.

PURPOSE: Student-run clinics (SRCs) are increasingly recognized as an educational experience in many health professions' curricula. Several benefits have been documented, including students with SRC experience using patient-centered approaches to care, showing interest in working with marginalized populations, and more fully appreciating the care provided by interprofessional teams. Yet, few studies have explored student experiences within SRCs or examined how these experiences affect and shape these documented attitudes. This study explored the experiences of students at an SRC and the effect of these experiences on their learnings. METHOD: From November 2016 to July 2017, 23 students in the Community Health Initiative by University Students SRC at the University of British Columbia participated in 2 focus group interviews: the first after their first clinic day and the second on their final clinic day. Open- and closed-ended questions were used to explore participants' learnings from the SRC. Using a grounded theory approach, the authors iteratively analyzed the transcribed interviews, adjusting questions for subsequent focus groups as new themes evolved. Three investigators each separately coded the data; the full team then collectively consolidated the themes and developed explanatory models for each theme. RESULTS: Two themes were identified from the focus group input: (1) through managing real, complex patients-in situations unlike those offered in classroom and case-based learning environments-students gained insights into the intricacies of incorporating the patient's perspective into their definition and management of the patient's problem, and (2) by working as a team instead of focusing on delineating scopes of practice, students gained a meaningful understanding of the roles of practitioners from other health professions. CONCLUSIONS: This study provides insights into the unique opportunities SRCs offer health care students early in their training, enabling them to develop a richer understanding and appreciation of holistic and interprofessional approaches to patient care.

Do in-training evaluation reports deserve their bad reputations? A study of the reliability and predictive ability of ITER scores and narrative comments.

PURPOSE: Although scores on in-training evaluation reports (ITERs) are often criticized for poor reliability and validity, ITER comments may yield valuable information. The authors assessed across-rotation reliability of ITER scores in one internal medicine program, ability of ITER scores and comments to predict postgraduate year three (PGY3) performance, and reliability and incremental predictive validity of attendings' analysis of written comments. METHOD: Numeric and narrative data from the first two years of ITERs for one cohort of residents at the University of Toronto Faculty of Medicine (2009-2011) were assessed for reliability and predictive validity of third-year performance. Twenty-four faculty attendings rank-ordered comments (without scores) such that each resident was ranked by three faculty. Mean ITER scores and comment rankings were submitted to regression analyses; dependent variables were PGY3 ITER scores and program directors' rankings. RESULTS: Reliabilities of ITER scores across nine rotations for 63 residents were 0.53 for both postgraduate year one (PGY1) and postgraduate year two (PGY2). Interrater reliabilities across three attendings' rankings were 0.83 for PGY1 and 0.79 for PGY2. There were strong correlations between ITER scores and comments within each year (0.72 and 0.70). Regressions revealed that PGY1 and PGY2 ITER scores collectively explained 25% of variance in PGY3 scores and 46% of variance in PGY3 rankings. Comment rankings did not improve predictions. CONCLUSIONS: ITER scores across multiple rotations showed decent reliability and predictive validity. Comment ranks did not add to the predictive ability, but correlation analyses suggest that trainee performance can be measured through these comments.

Toward authentic clinical evaluation: pitfalls in the pursuit of competency.

PURPOSE: The drive toward competency-based education frameworks has created a tension between competing desires-for quantified, standardized measures on one hand, and for an authentic representation of what it means to be a good doctor on the other. The purpose of this study was to better understand the tensions that exist between competency frameworks and faculty's real-life experiences in evaluating residents. METHOD: Interviews were conducted with 19 experienced internal medicine attendings at two Canadian universities in 2007. Attendings each discussed a specific outstanding, average, and problematic resident they had supervised. Interviews were analyzed using grounded theory. RESULTS: Eight major themes emerged reflecting how faculty conceptualize residents' performance: knowledge, professionalism, patient interactions, team interactions, systems, disposition, trust, and impact on staff. Attendings' impressions of residents did not seem to result from a linear sum of dimensions; rather, domains idiosyncratically took on variable degrees of importance depending on the resident. Relative deficiencies in outstanding residents could be overlooked, whereas strengths in problematic residents could be discounted. Some constructs (e.g., impact on staff) were not competencies at all; rather, they seem to act as explanations or evidence of attendings' opinions. Standardized evaluation forms might constrain authentic depictions of residents' performance. CONCLUSIONS: Despite concerted efforts to create standardized, objective, competency-based evaluations, the assessment of residents' clinical performance still has a strong subjective influence. Attendings' holistic impressions should not be considered invalid simply because they are subjective. Instead, assessment methods should consider novel ways of accommodating these impressions to improve evaluation.