Desirable Difficulty: Theory and application of intentionally challenging learning.

CONTEXT: Health professions trainees must acquire a vast amount of clinical knowledge and skills, and a deliberate instructional design approach is needed to provide trainees with effective learning strategies. One powerful yet counterintuitive strategy that facilitates long-term learning is incorporating intentional difficulties during the learning process. Difficulties that require more effort from learners may impede short-term learning but are ultimately beneficial for long-term learning and are therefore termed Desirable Difficulties. OBJECTIVES: In this cross-cutting edge paper, we describe the Desirable Difficulty effect from three theoretical perspectives originating in different fields, discuss common evidence-based Desirable Difficulty strategies used in Health Professions Education and explore emerging research that could further optimise Desirable Difficulty-enhanced learning for health professions trainees. METHODS: We synthesise theory and research from psychology, cognitive science and Health Professions Education literatures to further the understanding and application of Desirable Difficulties. We introduce three theoretical perspectives that provide a comprehensive overview of the theoretical underpinnings of the Desirable Difficulty effect: the New Theory of Disuse, the Challenge Point Framework and Cognitive Load Theory. We then illustrate how three common Desirable Difficulty strategies in medical education research-retrieval practice, spaced practice and interleaved practice-can be understood through these theoretical lenses. Finally, we provide relevant examples from the literature and explore emerging research in this area. CONCLUSIONS: This paper summarises the theory and empirical research on Desirable Difficulties during the learning process, from explaining what they are and why they may be effective to how they have been applied in different contexts. We argue that providing educators and trainees with a comprehensive theoretical and applied understanding of Desirable Difficulty will promote deliberate instructional design decisions and lead to more effective learning.

Combining desirable difficulty learning strategies to address the amount-to-learn vs. time-to-learn imbalance in residency training.

PURPOSE: Residents have limited time and much to learn. Mounting evidence shows that Desirable Difficulty (DD) learning strategies can ease that imbalance, but few studies have specifically studied combinations of these strategies. METHODS: We tested two different combinations of DD strategies: a double combination of distributed practice and retrieval practice and a triple combination additionally including interleaved practice. We compared residents' annual In-Training Exam (ITE) scores and graduates' board certification performance between both DD curricula and a historical baseline. RESULTS: Average ITE scores rose from 149.06 in the historical baseline to 160.04 under the combined DD curricula (p < 0.001). Average ITE scores fell from 162.50 under the double combination to 155.11 under the triple combination (p = 0.03). There were no significant changes in graduates' board performance between any of the curricula. CONCLUSIONS: These results add to the evidence that DD strategies can enhance residents' learning. The drop in ITE scores under the triple DD combination may suggest that it pushed learners past beneficial desirable difficulty into detrimental overwhelming difficulty. Further research should apply this framework in larger and more diverse settings to clarify how these DD strategies can be optimally used to enhance residents' learning.

Testing the Effects of Individual Residents' Retrieval Practice on Standardized Examination Scores.

Background: Residents have limited time and much to learn. Retrieval practice-studying using exam-style review questions-is a powerful educational tool for aggregate groups of learners, but it has not been tested for individual graduate medical education (GME) trainees. Methods: We conducted a retrospective observational study examining how individual residents' self-directed retrieval practice affected their learning. We used linear regression models to compare the number of exam-style multiple-choice practice questions each resident answered per year to their scores on annual In-Training Exams (ITE). Results: We found that the higher a resident's ITE score was at the start of a year, the fewer practice questions they answered that year for both first- (p = 0.023) and second-year (p = 0.020) trainees. Then, the more questions a resident answered in a year, the more their ITE score increased over that year for both first- and second-year trainees (p = 0.026 and 0.025, respectively). Residents' prior ITE scores also independently predicted their subsequent ITE scores (p = 0.024 in the first and 0.007 in the second year), and the effect of their baseline scores was larger than the effect of the number of practice questions they answered. Conclusions: Individual residents' prior exam scores significantly predict their future exam scores, and their independent retrieval practice using exam-style review questions also significantly predicts the annual improvement in their scores. These findings add to the growing evidence supporting retrieval practice in GME, and they can inform individualized educational coaching for GME trainees.

An Interactive Workshop Reviewing Basic Biostatistics and Applying Bayes' Theorem to Diagnostic Testing and Clinical Decision-Making.

Introduction: Sensitivity, specificity, and predictive values-the basic statistics behind using and interpreting screening and diagnostic tests-are taught in all medical schools, yet studies have shown that a majority of physicians cannot correctly define and apply these concepts. Previous work has not rigorously examined this disconnect and attempted to address it. Methods: We used adult learning theory to design a case-based interactive workshop to review biostatistics and apply them to clinical decision-making using Bayes' theorem. Participants took an anonymous multiple-choice pretest, posttest, and delayed posttest on definitions and application of the concepts, and we compared the scores between the three tests. Several experiences with early iterations provided feedback to improve the workshop but were not included for analysis. Results: We conducted the finalized workshop with 54 pediatrics students, residents, and faculty. All learners completed the immediate pre- and posttests, and eight completed the delayed posttest. Average scores rose from 4.5/8 (56%) on the pretest to 6.5/8 (81%) on the posttest and 6.4/8 (80%) on the delayed posttest. Two-tailed t tests showed p < .001 for the difference between the pretest and both posttests, and post hoc power analysis showed a power of 99% to detect the observed differences. There was no significant difference (p = .8) between the posttest and delayed posttest. Discussion: Our work demonstrates that an interactive workshop reviewing basic biostatistics and teaching rational diagnostic testing using Bayes' theorem can be effective in connecting theoretical knowledge of biostatistics to evidence-based decision-making in real clinical practice.