Developing Evidence for Equitable Assessment Characteristics Based on Clinical Learner Preferences Using Discrete Choice Experiments.

PURPOSE: Medical education is only beginning to explore the factors that contribute to equitable assessment in clinical settings. Increasing knowledge about equitable assessment ensures a quality medical education experience that produces an excellent, diverse physician workforce equipped to address the health care disparities facing patients and communities. Through the lens of the Anti-Deficit Achievement framework, the authors aimed to obtain evidence for a model for equitable assessment in clinical training. METHOD: A discrete choice experiment approach was used which included an instrument with 6 attributes each at 2 levels to reveal learner preferences for the inclusion of each attribute in equitable assessment. Self-identified underrepresented in medicine (UIM) and not underrepresented in medicine (non-UIM) (N = 306) fourth-year medical students and senior residents in medicine, pediatrics, and surgery at 9 institutions across the United States completed the instrument. A mixed-effects logit model was used to determine attributes learners valued most. RESULTS: Participants valued the inclusion of all assessment attributes provided except for peer comparison. The most valued attribute of an equitable assessment was how learner identity, background, and trajectory were appreciated by clinical supervisors. The next most valued attributes were assessment of growth, supervisor bias training, narrative assessments, and assessment of learner's patient care, with participants willing to trade off any of the attributes to get several others. There were no significant differences in value placed on assessment attributes between UIM and non-UIM learners. Residents valued clinical supervisors valuing learner identity, background, and trajectory and clinical supervisor bias training more so than medical students. CONCLUSIONS: This study offers support for the components of an antideficit-focused model for equity in assessment and informs efforts to promote UIM learner success and guide equity, diversity, and inclusion initiatives in medical education.

Criterion-Based Assessment in a Norm-Based World: How Can We Move Past Grades?

In the United States, the medical education community has begun a shift from the Flexnerian time-based model to a competency-based medical education model. The graduate medical education (GME) community is substantially farther along in this transition than is the undergraduate medical education (UME) community.GME has largely adopted the use of competencies and their attendant milestones and increasingly is employing the framework of entrustable professional activities (EPAs) to assess trainee competence. The UME community faces several challenges to successfully navigating a similar transition. First is the reliance on norm-based reference standards in the UME-GME transition, comparing students' performance versus their peers' with grades, United States Medical Licensing Examination Step 1 and Step 2 score interpretation, and the structured Medical School Performance Evaluation, or dean's letter. Second is the reliance on proxy assessments rather than direct observation of learners. Third is the emphasis on summative rather than formative assessments.Educators have overcome a major barrier to change by establishing UME outcomes assessment criteria with the advent and general acceptance of the physician competency reference set and the Core EPAs for Entering Residency in UME. Now is the time for the hard work of developing assessments steeped in direct observation that can be accepted by learners and faculty across the educational continuum and can be shown to predict clinical performance in a much more meaningful way than the current measures of grades and examinations. The acceptance of such assessments will facilitate the UME transition toward competency-based medical education.

It depends on your perspective: Resident satisfaction with operative experience.

INTRODUCTION: Resident satisfaction is a key performance metric for surgery programs; we studied factors influencing resident satisfaction in operative cases, and the concordance of faculty and resident perceptions on these factors. METHODS: Resident and faculty were separately queried on satisfaction immediately following operative cases. Statistical significance of the associations between resident and faculty satisfaction and case-related factors were tested by Chi-square or Fisher's exact test. RESULTS: Residents and faculty were very satisfied in 56/87 (64%) and 36/87 (41%) of cases respectively. Resident satisfaction was associated with their perceived role as surgeon (p < 0.04), performing >50% of the case (p < 0.01), autonomy (p < 0.03), and PGY year 4-5(p < 0.02). Faculty taking over the case was associated with both resident and faculty dissatisfaction. Faculty satisfaction was associated with resident preparation (p < 0.01), faculty perception of resident autonomy (p < 0.01), and faculty familiarity with resident's skills (p < 0.01). CONCLUSIONS: Resident and faculty satisfaction are associated with the resident's competent performance of the case, suggesting interventions to optimize resident preparation for a case or faculty's ability to facilitate resident autonomy will improve satisfaction with OR experience.

Who did the case? Perceptions on resident operative participation.

BACKGROUND: The ACGME case log is one of the primary metrics used to determine resident competency; it is unclear if this is an accurate reflection of the residents' role and participation. METHODS: Residents and faculty were independently administered 16-question surveys following each case over a three-week period. The main outcome was agreement between resident and faculty on resident role and percent of the case performed by the resident. RESULTS: Matched responses were collected for 87 cases. Agreement on percent performed occurred in 61% of cases, on role in 63%, and on both in 47%. Disagreement was more often due to resident perception they performed more of the case. Faculty with <10 years experience were more likely to have disagreement compared to faculty with ≥10 years (p = 0.009). CONCLUSIONS: There was a high degree of disagreement between faculty and residents regarding percent of the case performed and role. Accurate understanding of participation and competency is vital for accrediting institutions and for resident self-assessment meriting further study of the causes for this disagreement to improve training and evaluation.

Educational Tools: Thinking Outside the Box.

The understanding, study, and use of educational tools and their application to the education of adults in professional fields are increasingly important. In this review, we have compiled a description of educational tools on the basis of the teaching and learning setting: the classroom, simulation center, hospital or clinic, and independent learning space. When available, examples of tools used in nephrology are provided. We emphasize that time should be taken to consider the goals of the educational activity and the type of learners and use the most appropriate tools needed to meet the goals. Constant reassessment of tools is important to discover innovation and reforms that improve teaching and learning.

Inspiring innovation in medical education.

Traditionally, changes to medical education come from the top down, an approach that potentially misses important contributions from medical students, residents, faculty and staff. In order to provide an avenue for them to bring forward their ideas for educational improvements, the University of Minnesota Medical School sponsored the "What's the Bright Idea?" contest. Through the contest, we sought to foster a culture of innovation and collaboration among faculty, staff and students. The contest included five phases: launch, idea submission, online voting, follow-up and implementation. Seventy-six ideas were submitted, and 902 people participated in the online voting. When asked in a follow-up survey whether the submitter would have developed their idea without the contest, 27% of respondents answered "no" and 18% answered "maybe." Three-fourths stated the contest stimulated networking and collaboration. Four of the recommendations are now being implemented.

A randomized controlled trial of behavior change counseling education for medical students.

BACKGROUND: Educating medical students about how to effectively counsel patients with negative health behaviors (i.e., lack of exercise, smoking) is vitally important. Behavior change counseling is a promising method that can be used by physicians to encourage positive changes in health behaviors. PURPOSE: To examine the effectiveness of a 2 h workshop in behavior change counseling for medical students. METHODS: This study used a pre-post control group design with 35 second-year medical students who were randomly assigned to participate in a behavior change counseling intervention or wait-list control group. Student knowledge and attitudes were assessed using multiple choice items and open-ended question prompts. Student skills were assessed via performance in a standardized patient encounter rated using the Behavior Change Counseling Index (BECCI). RESULTS: Student attitudes toward behavior change counseling were positive at both pre- and post-test assessment in both groups. Knowledge scores and BECCI total scores showed significantly greater improvement in the intervention group compared to the wait-list control group. CONCLUSIONS: This study found that a brief educational intervention had a positive impact on medical students' knowledge and skills in behavior change counseling, and that student attitudes about the counseling method were very positive.

Defining family-centered rounds.

BACKGROUND: Physicians are required to provide safe, effective, and high-quality care that is patient-centered. Continuing to meet the educational needs of residents and medical students in the setting of patient-centered care will require developing new models for hospital "work rounds." Family-centered rounds is a model of communicating and learning between the patient, family, medical professionals, and students on an academic, inpatient ward setting. Unfortunately, in the medical literature, there is no consensus on the definition of family-centered rounds. SUMMARY: Despite the increased utilization of hospitalists and the recognition that bedside teaching has many benefits, bedside rounds are underutilized. In this article, we present a description of family-centered rounds that is supported by a review of the literature on bedside teaching, family-centered care, and interdisciplinary care. The key difference between family-centered rounds and traditional bedside teaching is the active participation of the patient and family in the discussion. Interdisciplinary care implies that professionals from a variety of disciplines work collaboratively to develop a unified care plan. Family-centered rounding provides an interface between families and medical professionals that allows education of medical students and residents as well as the development of a unified care plan. CONCLUSIONS: Family-centered rounds hold potential to create a patient-centered environment, enhance medical education, and improve patient outcomes. The model is a planned, purposeful interaction that requires the permission of patients and families as well as the cooperation of physicians, nurses, and ancillary staff.

A Graduate Medical Education Initiative to Promote Professional Excellence Among Residency Program Coordinators.

BACKGROUND: The authors describe the scope and impact of a professional development program for residency and fellowship program coordinators (PCs) at the University of Arkansas for Medical Sciences (UAMS) College of Medicine. PCs are vital in the success of their residency programs, yet few articles to date have addressed their increasingly complex roles. PURPOSE: This exploratory study examines PCs' professional characteristics, perceptions that influence professional development meeting attendance, and the impact of the Program Coordinators' Organization (PCO). METHODS: All 44 PCs serving 53 residency and fellowship programs at UAMS were surveyed about their perceptions of the PCO in January 2006. RESULTS: The majority of respondents agreed that the PCO has improved their abilities and interactions with their supervisors, colleagues, and residents and that the PCO has made an institution-wide impact on residency education. CONCLUSIONS: Sponsoring a PCO may be an effective tool for organizations to enhance the role of PCs and their graduate medical education programs.