Development and Validation of a Test for Competence in Evidence-Based Medicine.

BACKGROUND: Medical educators need valid, reliable, and efficient tools to assess evidence-based medicine (EBM) knowledge and skills. Available EBM assessment tools either do not assess skills or are laborious to grade. OBJECTIVE: To validate a multiple-choice-based EBM test-the Resident EBM Skills Evaluation Tool (RESET). DESIGN: Cross-sectional study. PARTICIPANTS: A total of 304 medicine residents from five training programs and 33 EBM experts comprised the validation cohort. MAIN MEASURES: Internal reliability, item difficulty, and item discrimination were assessed. Construct validity was assessed by comparing mean total scores of trainees to experts. Experts were also asked to rate importance of each test item to assess content validity. KEY RESULTS: Experts had higher total scores than trainees (35.6 vs. 29.4, P < 0.001) and also scored significantly higher than residents on 11/18 items. Cronbach's alpha was 0.6 (acceptable), and no items had a low item-total correlation. Item difficulty ranged from 7 to 86%. All items were deemed "important" by > 50% of experts. CONCLUSIONS: The proposed EBM assessment tool is a reliable and valid instrument to assess competence in EBM. It is easy to administer and grade and could be used to guide and assess interventions in EBM education.

The use of podcasts as a tool to teach clinical reasoning: a pseudorandomized and controlled study.

OBJECTIVES: Podcasts have emerged as an efficient method for widespread delivery of educational clinical reasoning (CR) content. However, the impact of such podcasts on CR skills has not been established. We set out to determine whether exposure to expert reasoning in a podcast format leads to enhanced CR skills. METHODS: This is a pseudo-randomized study of third-year medical students (MS3) to either a control group (n=22) of pre-established online CR modules, or intervention group (n=26) with both the online modules and novel CR podcasts. The podcasts were developed from four "clinical unknown" cases presented to expert clinician educators. After completing these assignments in weeks 1-2, weekly history and physical (H&P) notes were collected and graded according to the validated IDEA rubric between weeks 3-7. A longitudinal regression model was used to compare the H&P IDEA scores over time. Usage and perception of the podcasts was also assessed via survey data. RESULTS: Ninety control and 128 intervention H&Ps were scored. There was no statistical difference in the change of average IDEA scores between intervention (0.92, p=0.35) and control groups (-0.33, p=0.83). Intervention participants positively received the podcasts and noted increased discussion of CR principles from both their ward (3.1 vs. 2.4, p=0.08) and teaching (3.2 vs. 2.5, p=0.05) attendings. CONCLUSIONS: This is the first objective, pseudo-randomized assessment of CR podcasts in undergraduate medical education. While we did not demonstrate significant improvement in IDEA scores, our data show that podcasts are a well-received tool that can prime learners to recognize CR principles.

Implementation of a clinical reasoning curriculum for clerkship-level medical students: a pseudo-randomized and controlled study.

Background The National Academies of Sciences report Improving Diagnosis in Healthcare highlighted the need for better training in medical decision-making, but most medical schools lack formal education in clinical reasoning. Methods We conducted a pseudo-randomized and controlled study to evaluate the impact of a clinical reasoning curriculum in an internal medicine clerkship. Students in the intervention group completed six interactive online modules focused on reasoning concepts and a skills-based workshop. We assessed the impact of the curriculum on clinical reasoning knowledge and skills and perception of education by evaluating: (1) performance on a clinical reasoning concept quiz, (2) demonstration of reasoning in hospital admission notes, and (3) awareness of attending physician utilization of clinical reasoning concepts. Results Students in the intervention group demonstrated superior performance on the clinical reasoning knowledge quiz (67% vs. 54%, p < 0.001). Students in the intervention group demonstrated superior written reasoning skills in the data synthesis (2.3 vs. 2.0, p = 0.02) and diagnostic reasoning (2.2 vs. 1.9, p = 0.02) portions of their admission notes, and reported more discussion of clinical reasoning by their attending physicians. Conclusions Exposure to a clinical reasoning curriculum was associated with superior reasoning knowledge and superior written demonstration of clinical reasoning skills by third-year medical students on an internal medicine clerkship.

Video Observation With Guided Reflection: A Method for Continuing Teaching Education.

BACKGROUND: Best practices for faculty development programs include longitudinal, practice-based formats incorporating experiential learning with opportunities for reflection and community building. Peer coaching for faculty development provides personalized, learner-centered, work-based learning. Implementation of traditional 1-on-1 peer coaching programs is challenging due to time, logistics, and methodological barriers. OBJECTIVE: We sought to improve observation and reflection skills and to expand personal teaching practices of clinician educators. METHODS: In 2016, we developed and evaluated an innovative "1-to-many" peer-coaching model utilizing large group review of video-recorded teaching encounters. Forty-three clinician-educator faculty in general internal medicine at the University of Pittsburgh attended at least 1 of 6 sessions between February and August 2016. Sessions were moderated by a master facilitator who guided direct observation of, and reflection on, observed teaching and highlighted efficacious teaching methods. The study evaluated the acceptability and efficacy of this novel faculty development program qualitatively, with semistructured, postcurriculum telephone interviews with 20 participating faculty. RESULTS: All respondents stated that they would continue to attend faculty development sessions and would recommend them to others. The most frequently cited advantages included exposure to new teaching strategies, direct feedback, safe environment, community of practice, and growth mind-set, yet barriers emerged, such as discomfort reviewing video, difficulty giving feedback across hierarchy, and initial skepticism. None described the curriculum as critical or unsafe. Most reported increased self-reflection and adoption of new teaching behaviors. CONCLUSIONS: This peer-coaching, video-based faculty development program was well received, feasible, and effective in changing self-reported teaching attitudes and practices.