The relationship between learner engagement and teaching effectiveness: a novel assessment of student engagement in continuing medical education.

BACKGROUND: Continuing medical education (CME) often uses passive educational models including lectures. However, numerous studies have questioned the effectiveness of these less engaging educational strategies. Studies outside of CME suggest that engaged learning is associated with improved educational outcomes. However, measuring participants' engagement can be challenging. We developed and determined the validity evidence for a novel instrument to assess learner engagement in CME. METHODS: We conducted a cross-sectional validation study at a large, didactic-style CME conference. Content validity evidence was established through review of literature and previously published engagement scales and conceptual frameworks on engagement, along with an iterative process involving experts in the field, to develop an eight-item Learner Engagement Instrument (LEI). Response process validity was established by vetting LEI items on item clarity and perceived meaning prior to implementation, as well as using a well-developed online platform with clear instructions. Internal structure validity evidence was based on factor analysis and calculating internal consistency reliability. Relations to other variables validity evidence was determined by examining associations between LEI and previously validated CME Teaching Effectiveness (CMETE) instrument scores. Following each presentation, all participants were invited to complete the LEI and the CMETE. RESULTS: 51 out of 206 participants completed the LEI and CMETE (response rate 25%) Correlations between the LEI and the CMETE overall scores were strong (r = 0.80). Internal consistency reliability for the LEI was excellent (Cronbach's alpha = 0.96). To support validity to internal structure, a factor analysis was performed and revealed a two dimensional instrument consisting of internal and external engagement domains. The internal consistency reliabilities were 0.96 for the internal engagement domain and 0.95 for the external engagement domain. CONCLUSION: Engagement, as measured by the LEI, is strongly related to teaching effectiveness. The LEI is supported by robust validity evidence including content, response process, internal structure, and relations to other variables. Given the relationship between learner engagement and teaching effectiveness, identifying more engaging and interactive methods for teaching in CME is recommended.

Use of the pause procedure in continuing medical education: A randomized controlled intervention study.

During lectures, a pause procedure (the presenter pauses so students can discuss content) can improve educational outcomes. We aimed to determine whether (1) continuing medical education (CME) presentations with a pause procedure were evaluated more favorably and (2) a pause procedure improved recall. In this randomized controlled intervention study of all participants (N = 214) at the Mayo Clinic Internal Medicine Board Review course, 48 lectures were randomly assigned to an intervention (pause procedure) or control (traditional lecture) group. The pause procedure was a 1-min pause at the middle and end of the presentation. Study outcomes were (1) presentation evaluation instrument scores and (2) number of recalled items per lecture. A total of 214 participants returned 145 surveys (response rate, 68%). Mean presentation evaluation scores were significantly higher for pause procedure than for traditional presentations (70.9% vs 65.8%; 95%CI for the difference, 3.5-6.7; p < .0001). Mean number of rapid recall items was higher for pause procedure presentations (0.68 vs 0.59; 95%CI for the difference, 0.02-0.14; p = .01). In a traditional CME course, presentations with a pause procedure had higher evaluation scores and more content was recalled. The pause procedure could arm CME presenters with an easy technique to improve educational content delivery.