Applying the Lean Startup Method to Structure Project-Based, Student-Driven Curricular Enhancements.

PROBLEM: Formal medical student engagement in curricular evaluation provides significant value through identification of opportunities for curricular change. Students provide diverse perspectives and have a unique vantage point, which allows them to see aspects of the curriculum that educators and administrators might not recognize. Current descriptions of student engagement are focused largely on collection, analysis, and presentation of summative feedback in the pre-clerkship curriculum. However, medical students could potentially contribute to curricular improvement in ways extending beyond post hoc curricular evaluation. Student teams focused on identification of specific needs and project-based implementation of solutions represent one means of doing so but require a structured, organizing method in order to succeed. INTERVENTION: We describe a novel, project-based, student-driven medical education initiative, the Special Projects Team, which is focused on identifying opportunities for forward-looking curricular enhancements beyond single courses or rotations. We adapted and implemented the lean startup method, a model for project management, in order to address the need for organization and accountability in the Special Projects Team. Members of the Special Projects Team were recruited from the first- and second-year medical school classes in the 2018-2020 academic years and provided with training on the lean startup method. Team members selected and pursued projects according to the principles of lean startup method, reporting their progress to the chair of the Special Projects Team and other team members at monthly meetings with pre-defined structure. CONTEXT: The Special Projects Team is part of the local Student Curricular Board at the Chicago campus of the University of Illinois College of Medicine. The Student Curricular Board is responsible for conducting curricular evaluation and improvement, operating under the local medical student council with financial support from the Office of Curricular Affairs. Direct supervision of the Special Projects Team is provided by a student chair, the executive board of the Student Curricular Board, and the curricular dean. IMPACT: The projects initiated as part of the Special Projects Team covered a broad range of themes, including curricular evaluation, technology, and student experiences. Lean startup method contributed to sustained project success and frequent reassessment across the two years of our experience, with aggregate project success or continuation rate of 68.4% (13/19 projects). We further demonstrate how lean startup method increased productivity while providing structure and accountability for a student-led medical education team. LESSONS LEARNED: Lean startup method can be used to structure student-driven, project-based curricular enhancements. This approach is broadly applicable to other medical schools with implementation requiring only a motivated student team, faculty advisor, and basic knowledge of the lean startup method.

A mastery learning approach to education about fall risk and gait assessment.

Mastery learning is a form of competency-based education in which learning time varies but outcomes are uniform. Trainees must meet a minimum passing standard (MPS) before completing a mastery learning curriculum. The objective of this study was to establish a curriculum for fall risk and gait assessment for medical students, determine an MPS for a fall risk and gait assessment clinical skills examination (CSE), and apply the MPS to a sample of medical students completing a fall risk and gait assessment CSE. Medical students completed an interactive session about fall risk and gait assessment including the Timed Up and Go (TUG) test and completed deliberate practice with 3 patients. Skills were evaluated using an 18-item skills checklist. A panel of clinical experts set the MPS at 82%. Eighty-seven medical students participated. The average score on the checklist was 14.7 of 18 (81.4%.) Although almost all performed the TUG correctly, only 61% met the MPS for the checklist. Our results suggest that a mastery learning approach may better prepare the 39% of students that did not meet MPS to complete a fall risk and gait assessment.

Promoting Responsible Electronic Documentation: Validity Evidence for a Checklist to Assess Progress Notes in the Electronic Health Record.

Construct: We aimed to develop an instrument to measure the quality of inpatient electronic health record- (EHR-) generated progress notes without requiring raters to review the detailed chart or know the patient. BACKGROUND: Notes written in EHRs have generated criticism for being unnecessarily long and redundant, perpetuating inaccuracy and obscuring providers' clinical reasoning. Available assessment tools either focus on outpatient progress notes or require chart review by raters to develop familiarity with the patient. APPROACH: We used medical literature, local expert review, and attending focus groups to develop and refine an instrument to evaluate inpatient progress notes. We measured interrater reliability and scored the selected-response elements of the checklist for a sample of 100 progress notes written by PGY-1 trainees on the general medicine service. RESULTS: We developed an instrument with 18 selected-response items and four open-ended items to measure the quality of inpatient progress notes written in the EHR. The mean Cohen's kappa coefficient demonstrated good agreement at .67. The mean note score was 66.9% of maximum possible points (SD = 10.6, range = 34.4%-93.3%). CONCLUSIONS: We present validity evidence in the domains of content, internal structure, and response process for a new checklist for rating inpatient progress notes. The scored checklist can be completed in approximately 7 minutes by a rater who is not familiar with the patient and can be done without extensive chart review. We further demonstrate that trainee notes show substantial room for improvement.

Medical students' observations, practices, and attitudes regarding electronic health record documentation.

BACKGROUND: Medical students are increasingly documenting their patient notes in electronic health records (EHRs). Documentation short-cuts, such as copy-paste and templates, have raised concern among clinician-educators because they may perpetuate redundant, inaccurate, or even plagiarized notes. Little is known about medical students' experiences with copy-paste, templates and other "efficiency tools" in EHRs. PURPOSES: We sought to understand medical students' observations, practices, and attitudes regarding electronic documentation efficiency tools. METHODS: We surveyed 3rd-year medical students at one medical school. We asked about efficiency tools including copy-paste, templates, auto-inserted data, and "scribing" (documentation under a supervisor's name). RESULTS: Overall, 123 of 163 students (75%) responded; almost all frequently use an EHR for documentation. Eighty-six percent (102/119) reported at least sometimes observing residents copying data from other providers' notes and 60% (70/116) reported observing attending physicians doing so. Most students (95%, 113/119) reported copying from their own previous notes, and 22% (26/119) reported copying from residents. Only 10% (12/119) indicated that copying from other providers is acceptable, whereas 83% (98/118) believe copying from their own notes is acceptable. Most students use templates and auto-inserted data; 43% (51/120) reported documenting while signed in under an attending's name. Greater use of documentation efficiency tools is associated with plans to enter a procedural specialty and with lack of awareness of the medical school copy-paste policy. CONCLUSIONS: Students frequently use a range of efficiency tools to document in the electronic health record, most commonly copying their own notes. Although the vast majority of students believe it is unacceptable to copy-paste from other providers, most have observed clinical supervisors doing so.

E-learning and deliberate practice for oral case presentation skills: a randomized trial.

BACKGROUND: Oral case presentations are critical for patient care and student assessment. The best method to prepare early medical students for oral presentations is unknown. AIM: We aimed to develop and evaluate a curriculum of on-line learning and deliberate practice to improve pre-clinical students' case presentation skills. METHODS: We developed a web-based, interactive curriculum emphasizing conciseness and clinical reasoning. Using a waitlist control design, we randomly assigned groups of second-year students to receive the curriculum in December 2010 or in April 2011. We evaluated their presentations at three time points. We also examined the performance of an untrained class of students as a historical comparison. RESULTS: We evaluated 132 second-year medical students at three time points. After the curriculum, mean scores of the intervention students improved from 60.2% to 70.1%, while scores of the waitlist control students improved less, from 61.8% to 64.5% (p < 0.01 for between-group difference in improvement). Once all students had received the curriculum, mean scores for the intervention and waitlist control students rose to 77.8% and 78.4%, respectively, compared to 68.1% for the untrained comparison students (p < 0.0001 compared to all curriculum students). CONCLUSION: An on-line curriculum followed by deliberate practice improved students' oral presentation skills.

Description and Early Outcomes of a Comprehensive Curriculum Redesign at the Northwestern University Feinberg School of Medicine.

In 2012, the Northwestern University Feinberg School of Medicine launched a redesigned curriculum addressing the four primary recommendations in the 2010 Carnegie Foundation for the Advancement of Teaching report on reforming medical education. This new curriculum provides a more standardized evaluation of students' competency achievement through a robust portfolio review process coupled with standard evaluations of medical knowledge and clinical skills. It individualizes learning processes through curriculum flexibility, enabling students to take electives earlier and complete clerkships in their preferred order. The new curriculum is integrated both horizontally and vertically, combining disciplines within organ-based modules and deliberately linking elements (science in medicine, clinical medicine, health and society, professional development) and threads (medical decision making, quality and safety, teamwork and leadership, lifestyle medicine, advocacy and equity) across the three phases that replaced the traditional four-year timeline. It encourages students to conduct research in an area of interest and commit to lifelong learning and self-improvement. The curriculum formalizes the process of professional identity formation and requires students to reflect on their experiences with the informal and hidden curricula, which strongly shape their identities.The authors describe the new curriculum structure, explain their approach to each Carnegie report recommendation, describe early outcomes and challenges, and propose areas for further work. Early data from the first cohort to progress through the curriculum show unchanged United States Medical Licensing Examination Step 1 and 2 scores, enhanced student research engagement and career exploration, and improved student confidence in the patient care and professional development domains.

Patient Handoffs: Is Cross Cover or Night Shift Better?

OBJECTIVES: Studies show singular handoffs between health care providers to be risky. Few describe sequential handoffs or compare handoffs from different provider types. We investigated the transfer of information across 2 handoffs using a piloted survey instrument. We compared cross-cover (every fourth night call) with dedicated night-shift residents. METHODS: Surveys assessing provider knowledge of hospitalized patients were administered to pediatric residents. Primary teams were surveyed about their handoff upon completion of daytime coverage of a patient. Night-shift or cross-covering residents were surveyed about their handoff of the same patient upon completion of overnight coverage. Pediatric hospitalists rated the consistency of information between the surveys. Absolute difference was calculated between the 2 providers' rating of a patient's (a) complexity and (b) illness severity. Scores were compared across provider type. RESULTS: Fifty-nine complete handoff pairs were obtained. Fourteen and 45 handoff surveys were completed by a cross-covering and a night-shift provider, respectively. There was no significant difference in information consistency between primary and night-shift (median, 4.0; interquartile range [IQR], 3-4) versus primary and cross-covering providers (median, 4.0; IQR, 3-4). There was no significant difference in median patient complexity ratings (night shift, 3.0; IQR, 1-5, versus cross cover, 3.5; IQR, 1-5) or illness severity ratings (night shift, 2.0; IQR, 1-4, versus cross-cover, 3.0; IQR, 1-6) when comparing provider types giving a handoff. CONCLUSIONS: We did not find a difference in physicians' transfer of information during 2 handoffs among providers taking traditional call or on night shift. Development of tools to measure handoff consistency is needed.

Electronically screening discharge summaries for adverse medical events.

OBJECTIVE: Detecting adverse events is pivotal for measuring and improving medical safety, yet current techniques discourage routine screening. The authors hypothesized that discharge summaries would include information on adverse events, and they developed and evaluated an electronic method for screening medical discharge summaries for adverse events. DESIGN: A cohort study including 424 randomly selected admissions to the medical services of an academic medical center was conducted between January and July 2000. The authors developed a computerized screening tool that searched free-text discharge summaries for trigger words representing possible adverse events. MEASUREMENTS: All discharge summaries with a trigger word present underwent chart review by two independent physician reviewers. The presence of adverse events was assessed using structured implicit judgment. A random sample of discharge summaries without trigger words also was reviewed. RESULTS: Fifty-nine percent (251 of 424) of the discharge summaries contained trigger words. Based on discharge summary review, 44.8% (327 of 730) of the alerted trigger words indicated a possible adverse event. After medical record review, the tool detected 131 adverse events. The sensitivity and specificity of the screening tool were 69% and 48%, respectively. The positive predictive value of the tool was 52%. CONCLUSION: Medical discharge summaries contain information regarding adverse events. Electronic screening of discharge summaries for adverse events using keyword searches is feasible but thus far has poor specificity. Nonetheless, computerized clinical narrative screening methods could potentially offer researchers and quality managers a means to routinely detect adverse events.

Understanding of drug indications by ambulatory care patients.

PURPOSE: Patients' knowledge of the indications of their prescription medications was studied and those medications that were most likely to be taken without patients understanding the correct indication were identified. METHODS: Adult patients who received care at four primary care practices were surveyed. Patients were eligible to participate if they were over 18 years old and had received a prescription from a participating physician at a clinic visit. Patients were telephoned and asked to retrieve the bottles of all medications they were currently taking, identify their medications, and state the reason they took each medicine. The primary outcome was absent or incorrect knowledge of a drug's indication. RESULTS: A total of 2340 prescription medications were used by the 616 patients whose data were analyzed. Eighty-three patients (13.5%) lacked knowledge of the indication for at least one of their prescription medications. They did not know the indication for 148 medications (6.3%). After multivariable adjustment, lack of knowledge was more common for cardiovascular drugs (odds ratio [OR], 1.50; 95% confidence interval [CI], 1.03-2.19) and less common for diabetes medications (OR, 0.37; 95% CI, 0.16-0.84) and analgesics (OR, 0.23; 95% CI, 0.05-1.01) compared with all other medications, and more common if the patient taking these medications was older, black, or had a high school education or less. CONCLUSION: More than 13% of patients in primary care practices did not know the indication of at least one of their prescription medications. Lack of knowledge was most prevalent for cardiovascular medications.

Teaching the physical examination: a longitudinal strategy for tomorrow's physicians.

The physical examination is an essential clinical skill. The traditional approach to teaching the physical exam has involved a comprehensive "head-to-toe" checklist, which is often used to assess students before they begin their clinical clerkships. This method has been criticized for its lack of clinical context and for promoting rote memorization without critical thinking. In response to these concerns, Gowda and colleagues surveyed a national sample of clinical skills educators in order to develop a consensus "core" physical exam, which they report in this issue. The core physical exam is intended to be performed for every patient admitted by students during their medicine clerkships and to be supplemented by symptom-driven "clusters" of additional history and physical exam maneuvers.In this commentary, the authors review the strengths and limitations of this Core + Clusters technique as well as the head-to-toe approach. They propose that the head-to-toe still has a place in medical education, particularly for beginning students with little knowledge of pathophysiology and for patients with vague or multiple symptoms. The authors suggest that the ideal curriculum would include teaching both the head-to-toe and the Core + Clusters exams in sequence. This iterative approach to physical exam teaching would allow a student to assess a patient in a comprehensive manner while incorporating more clinical reasoning as further medical knowledge is acquired.