Automatic scoring of medical students' clinical notes to monitor learning in the workplace.

BACKGROUND: Educators need efficient and effective means to track students' clinical experiences to monitor their progress toward competency goals. AIM: To validate an electronic scoring system that rates medical students' clinical notes for relevance to priority topics of the medical school curriculum. METHOD: The Vanderbilt School of Medicine Core Clinical Curriculum enumerates 25 core clinical problems (CCP) that graduating medical students must understand. Medical students upload clinical notes pertinent to each CCP to a web-based dashboard, but criteria for determining relevance of a note and consistent uploading practices by students are lacking. The Vanderbilt Learning Portfolio (VLP) system automates both tasks by rating relevance for each CCP and uploading the note to the student's electronic dashboard. We validated this electronic scoring system by comparing the relevance of 265 clinical notes written by third year medical students to each of the 25 core patient problems as scored by VLP verses an expert panel of raters. RESULTS: We established the threshold score which yielded 75% positive prediction of relevance for 16 of the 25 clinical problems to expert opinion. DISCUSSION: Automated scoring of student's clinical notes provides a novel, efficient and standardized means of tracking student's progress toward institutional competency goals.

"Finding My Piece in That Puzzle": A Qualitative Study Exploring How Medical Students at Four U.S. Schools Envision Their Future Professional Identity in Relation to Health Systems.

PURPOSE: Health systems science (HSS) curricula equip future physicians to improve patient, population, and health systems outcomes (i.e., to become "systems citizens"), but the degree to which medical students internalize this conception of the physician role remains unclear. This study aimed to explore how students envision their future professional identity in relation to the system and identify experiences relevant to this aspect of identity formation. METHOD: Between December 2018 and September 2019, authors interviewed 48 students at 4 U.S. medical schools with HSS curricula. Semistructured interviews were audiorecorded, transcribed, and analyzed iteratively using inductive thematic analysis. Interview questions explored how students understood the health system, systems-related activities they envisioned as future physicians, and experiences and considerations shaping their perspectives. RESULTS: Most students anticipated enacting one or more systems-related roles as a future physician, categorized as "bottom-up" efforts enacted at a patient or community level (humanist, connector, steward) or "top-down" efforts enacted at a system or policy level (system improver, system scholar, policy advocate). Corresponding activities included attending to social determinants of health or serving medically underserved populations, connecting patients with team members to address systems-related barriers, stewarding health care resources, conducting quality improvement projects, researching/teaching systems topics, and advocating for policy change. Students attributed systems-related aspirations to experiences beyond HSS curricula (e.g., low-income background; work or volunteer experience; undergraduate studies; exposure to systems challenges affecting patients; supportive classmates, faculty, and institutional culture). Students also described future-oriented considerations promoting or undermining identification with systems-related roles (responsibility, affinity, ability, efficacy, priority, reality, consequences). CONCLUSIONS: This study illuminates systems-related roles medical students at 4 schools with HSS curricula envisioned as part of their future physician identity and highlights past/present experiences and future-oriented considerations shaping identification with such roles. These findings inform practical strategies to support professional identity formation inclusive of systems engagement.

Defining Successful Practice Within Health Systems Science Among Entering Residents: A Single-Institution Qualitative Study of Graduate Medical Education Faculty Observations.

PURPOSE: The American Medical Association's Accelerating Change in Medical Education consortium defined health systems science (HSS) as the study of how health care is delivered, how health care professionals work together to deliver that care, and how the health system can improve patient care and health care delivery. This framework is increasingly being incorporated into medical school curricula. Graduate medical education (GME) had previously elevated systems-based practice (SBP) as a core competency, but expectations are defined by specialty-specific milestones. The lack of a shared competency framework between undergraduate medical education (UME) and GME makes it challenging to ensure that entering residents are prepared to implement HSS/SBP concepts in the workplace. The authors explored GME faculty observations of residents exemplifying successful practice across HSS domains to inform targets for UME training and assessment. METHOD: Authors performed a single-institution qualitative study using transcribed phone interviews with eligible Vanderbilt residency program directors, associate program directors, and core faculty. Participants described observations of successful residents within each HSS domain. Two researchers independently coded, discussed, and reconciled deidentified transcripts using inductive-deductive approaches to identify themes. RESULTS: Seventeen faculty participated across specialties (17/39, 45%). Faculty responses emphasize precurricular experiences including professional degrees, work experience, extracurriculars, and medical school exposure. Importantly, successful residents exhibit foundational core workforce characteristics including growth mindset, curiosity, and a desire to learn about systems. GME faculty identified HSS domain-specific skills, noting distinctions among learning environments. Outcomes resulting from residents' application of HSS concepts include delivering high-quality, person-centered care and systems improvements. CONCLUSIONS: Descriptions of successful practice within HSS domains highlight preparatory experiences and core workforce characteristics and outline entry-level HSS behaviors. Conceptualized in a logic model framework, these findings describe key inputs, learning activities, outputs, and outcomes for systems-prepared entering residents bridging the UME-GME transition.

Exploring the thesis experience of Master of Health professions education graduates: a qualitative study.

OBJECTIVES: To explore the thesis experience of recent Master of Health Professions Education (MHPE) graduates in the University of Illinois at Chicago (UIC) program. METHODS: This is a qualitative case study exploring the experience of MHPE graduates between 2014 and 2016 (n=31). Using convenience sampling, all graduates with an email address (n=30) were invited to participate in an online survey and semi-structured interviews. Interviews were completed in-person or via telephone or video conference; interviewers collected detailed notes and audio recordings.  Two authors independently analyzed the data iteratively using thematic analysis and discrepancies were discussed and resolved. RESULTS: Survey results (n=20, 67%) revealed an average graduation of 5.1 years; 10 graduates (33%) were interviewed. Three themes related to the thesis experience were identified: success factors, challenges, and outcomes. Success factors, when present, promoted completion of a thesis; these included: a supportive program environment, time management, available resources, MHPE foundational coursework, aligning theses with career goals, and identifying a project with limited scope. Challenges made thesis completion more difficult for graduates; these included: institutional factors, personal or professional responsibilities, burnout, externally-imposed deadlines, and barriers in the research process. Despite these challenges, completing the thesis resulted in many professional or personal benefits (outcomes). CONCLUSIONS: Multiple success factors and challenges were identified in the master's thesis process among MHPE graduates at UIC. These findings can help students conducting education-based scholarship through the master's thesis process. This study also informs program evaluation and improvements and outlines personal and professional outcomes of completing a master's thesis.

Struggling Medical Learners: A Competency-Based Approach to Improving Performance.

Introduction: Faculty must be trained to recognize, analyze, and provide feedback and resources to struggling medical learners. Training programs must be equipped to intervene when necessary with individualized remediation efforts to ensure learner success. Methods: This 90-minute interactive faculty development workshop provides a foundational competency-based framework for identifying and assisting the struggling medical learner. The workshop uses a mock academic promotions committee meeting addressing the case of a struggling undergraduate learner. The workshop was presented at two regional conferences, and participants completed an anonymous evaluation form containing 10 items on a 5-point Likert scale and two open-ended questions. Data were analyzed and a subgroup analysis performed using an independent t test and correlation. Qualitative data were read and coded for representative themes by two authors. Results: Fifty-five participants completed an evaluation form. The quality of the workshop was high (M = 4.5, SD = 0.6); participants agreed that the learning objectives were achieved and relevant to their educational needs (M = 4.4, SD = 0.7). A significant positive correlation existed between perceived quality and the interactive elements (.70, p < .05) as well as the intention to apply learning (.60, p < .05). Written comments revealed six themes: role-play, resources, interaction with colleagues, modeling, relevant content, and the process of learning. Discussion: The workshop's quality, relevance, and applicability were rated excellent among medical educators. Participants felt the interactive nature of the workshop was its most useful aspect, and a majority intended to apply the learning to their practice.

Discharging Mrs. Fox: A Team-Based Interprofessional Collaborative Standardized Patient Encounter.

INTRODUCTION: In 2003, the Institute of Medicine recommended that interprofessional education be incorporated into the training programs of health care professionals. However, many logistical challenges hinder formal interprofessional learning in health care profession programs. METHODS: This resource is a 3-hour interprofessional small-group session designed for health professions student teams to engage in a standardized patient encounter, each team member contributing a profession-specific perspective to create a collaborative care plan across five discharge decisions. The activity includes a simulated standardized patient encounter and debrief session wherein students discuss the role of bias and communication and create a collaborative care plan. RESULTS: Following the activity, participants were surveyed about the value of the educational experience. Over 12 months, 106 students (81 medicine, nine nursing, 16 pharmacy) participated in the interprofessional activity. Eighty-four students responded to the postevent survey (79% response rate). Students were confident that the experience helped them integrate profession-specific knowledge, create a shared care plan, and understand how interprofessional collaboration contributes to quality care. The debriefing session and interprofessional interaction were an integral component of the experience. DISCUSSION: This resource is a feasible interprofessional small-group activity that has been implemented without excessive faculty time or institutional resources. It is adaptable to institutional needs, local resources, level of trainee, and professions. The session provides interprofessional students the opportunity to engage with one another and with the patient in a collaborative decision-making activity focused around a critical transition of care.