Timing's not everything: Immediate and delayed feedback are equally beneficial for performance in formative multiple-choice testing.

INTRODUCTION: Test-enhanced learning (TEL) is an impactful teaching and learning strategy that prioritises active learner engagement through the process of regular testing and reviewing. While it is clear that meaningful feedback optimises the effects of TEL, the ideal timing of this feedback (i.e. immediate or delayed) in a medical education setting is unclear. METHOD: Forty-one second-year medical students were recruited from the University of Melbourne. Participants were given a multiple-choice question test with a mix of immediate (i.e. post-item) and delayed (i.e. post-item-block) conceptual feedback. Students were then tested on near and far transfer items during an immediate post-test, and at a one-week follow-up. RESULTS: A logistic mixed effects model was used to predict the probability of successful near and far transfer. As expected, participants in our study tended to score lower on far transfer items than they did on near transfer items. In addition, correct initial response on a parent question predicted subsequent correct responding. Contrary to our hypotheses, the feedback timing effect was non-significant-there was no discernible difference between feedback delivered immediately versus delayed feedback. DISCUSSION: The findings of this study suggest that the timing of feedback delivery (post-item versus post-item-block) does not influence the efficacy of TEL in this medical education setting. We therefore suggest that educators may consider practical factors when determining appropriate TEL feedback timing in their setting.

Functional Dysregulation of CDC42 Causes Diverse Developmental Phenotypes.

Exome sequencing has markedly enhanced the discovery of genes implicated in Mendelian disorders, particularly for individuals in whom a known clinical entity could not be assigned. This has led to the recognition that phenotypic heterogeneity resulting from allelic mutations occurs more commonly than previously appreciated. Here, we report that missense variants in CDC42, a gene encoding a small GTPase functioning as an intracellular signaling node, underlie a clinically heterogeneous group of phenotypes characterized by variable growth dysregulation, facial dysmorphism, and neurodevelopmental, immunological, and hematological anomalies, including a phenotype resembling Noonan syndrome, a developmental disorder caused by dysregulated RAS signaling. In silico, in vitro, and in vivo analyses demonstrate that mutations variably perturb CDC42 function by altering the switch between the active and inactive states of the GTPase and/or affecting CDC42 interaction with effectors, and differentially disturb cellular and developmental processes. These findings reveal the remarkably variable impact that dominantly acting CDC42 mutations have on cell function and development, creating challenges in syndrome definition, and exemplify the importance of functional profiling for syndrome recognition and delineation.

Planning Education for Long-Term Retention: The Cognitive Science and Implementation of Retrieval Practice.

Educational systems are rarely designed for long-term retention of information. Strong evidence has emerged from cognitive psychology and applied education studies that repeated retrieval of information significantly improves retention compared to repeated studying. This effect likely emerges from the processes of memory consolidation and reconsolidation. Consolidation and reconsolidation are the means by which memories are organized into associational networks or schemas that are created and recreated as memories are formed and recalled. As educators implement retrieval practice, they should consider how various test formats lead to different degrees of schema activation. Repeated acts of retrieval provide opportunities for schemas to be updated and strengthened. Spacing of retrieval allows more consolidated schemas to be reactivated. Feedback provides metacognitive monitoring to ensure retrieval accuracy and can lead to shifts from ineffective to effective retrieval strategies. By using the principles of retrieval practice, educators can improve the likelihood that learners will retain information for longer periods of time.

Tying knots: an activity theory analysis of student learning goals in clinical education.

CONTEXT: Learning goal programmes are often created to help students develop self-regulated learning skills; however, these programmes do not necessarily consider the social contexts surrounding learning goals or how they fit into daily educational practice. OBJECTIVES: We investigated a high-frequency learning goal programme in which students generated and shared weekly learning goals with their clinical teams in core Year 3 clerkships. Our study explores: (i) how learning goals were incorporated into the clinical work, and (ii) the factors that influenced the use of students' learning goals in work-based learning. METHODS: We conducted semi-structured interviews with 14 students and 14 supervisors (attending physicians and residents) sampled from all participating core clerkships. Interviews were coded for emerging themes. Using cultural historical activity theory and knotworking as theoretical lenses, we developed a model of the factors that influenced students' learning goal usage in a work-based learning context. RESULTS: Students and supervisors often faced the challenge of reconciling contradictions that arose when the desired outcomes of student skill development, grading and patient care were not aligned. Learning goals could function as tools for developing new ways of acting that overcame those contradictions by facilitating collaborative effort between students and their supervisors. However, for new collaborations to take place, both students and supervisors had to engage with the goals, and the necessary patients needed to be present. When any one part of the system did not converge around the learning goals, the impact of the learning goals programme was limited. CONCLUSIONS: Learning goals are potentially powerful tools to mediate interactions between students, supervisors and patients, and to reconcile contradictions in work-based learning environments. Learning goals provide a means to develop not only learners, but also learning systems.

High-Frequency Learning Goals: Using Self-Regulated Learning to Influence Day-to-Day Practice in Clinical Education.

PROBLEM: Although self-regulated learning (SRL) is considered a fundamental skill that must be developed in physician training, many programs of SRL utilize learning goals that are generated only at the beginning of learning experiences or are widely spaced apart in time. These goals are often not formally shared with those actually working with the learner in the clinical setting. INTERVENTION: We developed a program of written, student-generated weekly learning goals in which students focused on processes of becoming better doctors for their patients. These goals were shared at the beginning of each week with students' clinical teams for feedback and incorporation into the work. CONTEXT: The weekly learning goals program was developed and implemented as part of a required 3rd-year neurology clerkship. At the end of each 4-week clerkship, students were asked to evaluate the program through an anonymous electronic survey utilizing quantitative and open-ended qualitative questions. OUTCOME: Seventy-six of 82 students completed the evaluation survey (93% response rate). Eighty-six percent reported that the weekly learning goals increased their awareness of their thoughts and actions. Seventy-eight percent reported that the learning goals helped to improve their clinical performance to some degree, and 57% reported that the learning goals increased their focus on patient care. Students described a greater sense of focus on self-assessment and accountability from their goals. Students often commented that engagement from attendings and residents regarding their goals was a key element for successful learning from their goals. LESSONS LEARNED: Student-generated weekly learning goals on a neurology clerkship appear to be an effective method to operationalize SRL. For most students, the frequency of the goals allowed for close self-monitoring, and the act of sharing goals with the team opened a new avenue for dialogue between students and their supervisors.

Student views on the role of self-regulated learning in a surgery clerkship.

BACKGROUND: Self-regulated learning, including student-generated learning goals and flexibility in the learning structure are increasingly being used to enhance medical education. The role of these practices in surgical education of medical students has not been studied. MATERIALS AND METHODS: We administered an 18-question electronic survey to all third-year medical students at Washington University in St. Louis School of Medicine. Of the 126 students invited, 64 responded and 56 were included in the analysis. RESULTS: We found that third-year medical students develop learning goals at the beginning of the surgery clerkship. Although these learning goals theoretically can be a mechanism for enhanced student-faculty engagement, students are not aware of formal mechanisms for sharing these goals with faculty members. Furthermore, students report a lack of flexibility within the surgery clerkship and discomfort with requesting specific learning opportunities. Finally, students report that they believe increased flexibility could improve student engagement, learning, and the overall clerkship experience. CONCLUSIONS: We therefore propose that a mechanism for students to share their learning goals with faculty and an infrastructure in which student learning experiences can be tailored to fit with these individualized goals would enhance student surgical learning.

Effects of Team-Based Learning on short-term and long-term retention of factual knowledge.

PURPOSE: We investigated the effect of Team-Based Learning (TBL) on long-term retention of knowledge in comparison to a traditional curriculum. METHODS: As TBL was incorporated into our curriculum in the 2008-2009 academic year, students were compared with those who received the traditional curriculum the year prior. Students in both the groups completed multiple-choice knowledge test at four time points spanning two years. Test performance was compared at each time point to assess changes in knowledge retention as a function of time. RESULTS: Baseline knowledge did not differ significantly between the TBL and control groups [51% versus 46%; t(84) = 0.91, p = 0.37, d = 0.20]. Performance improved after the course for both the groups, but was significantly higher in the TBL group [79% versus 59%; t(84) = 4.96, p = 0.000004, d = 0.95]. However, when assessed prior to the pediatrics clerkship, learning gains from TBL had largely disappeared and the small difference in performance was not significant [57% versus 51%; t(84) = 1.51, p = 0.14, d = 0.32]. CONCLUSION: Incorporating TBL into the pre-clinical pediatrics curriculum led to large gains in knowledge over the short-term, but these gains did not persist. Further research should focus on extending the impact of TBL on long-term knowledge retention.

Comparative effects of test-enhanced learning and self-explanation on long-term retention.

CONTEXT: Educators often encourage students to engage in active learning by generating explanations for the material being learned, a method called self-explanation. Studies have also demonstrated that repeated testing improves retention. However, no studies have directly compared the two learning methods. METHODS: Forty-seven Year 1 medical students completed the study. All students participated in a teaching session that covered four clinical topics and was followed by four weekly learning sessions. In the learning sessions, students were randomised to perform one of four learning activities for each topic: testing with self-generated explanations (TE); testing without explanations (T); studying a review sheet with self-generated explanations (SE), and studying a review sheet without explanations (S). Students repeated the same activity for each topic in all four sessions. Six months later, they took a free-recall clinical application test on all four topics. RESULTS: Repeated testing led to better long-term retention and application than repeatedly studying the material (p < 0.0001, η(2)  = 0.33). Repeated generation of self-explanations also improved long-term retention and application, but the effect was smaller (p < 0.0001, η(2)  = 0.08). When data were collapsed across topics, both testing conditions produced better final test performance than studying with self-explanation (TE = 40% > SE = 29% [p = 0.001, d = 0.70]; T = 36% > SE = 29% [p = 0.02, d = 0.48]). Studying with self-explanation led to better retention and application than studying without self-explanation (SE = 29% > S = 20%; p = 0.001, d = 0.68). Our analyses showed significant interaction by topic (p = 0.001, η(2)  = 0.06), indicating some variation in the effectiveness of the interventions among topics. CONCLUSIONS: Testing and generating self-explanations are both learning activities that can be used to produce superior long-term retention and application of knowledge, but testing is generally more effective than self-explanation alone.

The importance of seeing the patient: test-enhanced learning with standardized patients and written tests improves clinical application of knowledge.

Previous research has shown that repeated retrieval with written tests produces superior long-term retention compared to repeated study. However, the degree to which this increased retention transfers to clinical application has not been investigated. In addition, increased retention obtained through written testing has not been compared to other forms of testing, such as simulation testing with a standardized patient (SP). In our study, 41 medical students learned three clinical topics through three different learning activities: testing with SPs, testing using written tests, and studying a review sheet. Students were randomized in a counter-balanced fashion to engage in one learning activity per topic. They participated in four weekly testing/studying sessions to learn the material, engaging in the same activity for a given topic in each session. Six months after initial learning, they returned to take an SP test on each topic, followed by a written test on each topic 1 week later. On both forms of final testing, we found that learning through SP testing and written testing generally produced superior long-term retention compared to studying a review sheet. SP testing led to significantly better performance on the final SP test relative to written testing, but there was no significant difference between the two testing conditions on the final written test. Overall, our study shows that repeated retrieval practice with both SPs and written testing enhances long-term retention and transfer of knowledge to a simulated clinical application.

Ictal asystole: a rare cause of cardiac arrest.

Ictal asystole, a rare cause of transient cardiac arrest, is triggered by seizure activity. Long-term seizure control and pacemaker implantation can reduce the risk of this complication.

SMARCB1/INI1-deficient pancreatic undifferentiated rhabdoid carcinoma mimicking solid pseudopapillary neoplasm: A case report and review of the literature.

BACKGROUND: SMARCB1/INI1-deficient pancreatic undifferentiated rhabdoid carcinoma is a very aggressive tumor that is rarely reported in the literature. The tumor has a predominant rhabdoid cell component and different patterns of growth have been reported. CASE SUMMARY: A 59-year-old woman presented with diffuse abdominal pain, increasing in severity and accompanied by weight loss, nausea, and vomiting. Imaging showed a pancreatic head mass. Fine needle aspiration demonstrated atypical epithelioid cells with a pseudopapillary growth pattern suggestive of solid pseudopapillary neoplasm. The excised neoplasm showed monotonous epithelioid and focally spindle cells with pseudopapillary structures, rhabdoid features, and loss of SMARCB1 protein expression with wild-type KRAS, consistent with a SMARCB1/INI1-deficient undifferentiated rhabdoid carcinoma. The patient's condition deteriorated rapidly following surgery and she expired 3 mo post operation. CONCLUSION: In this article, we report the first case of SMARCB1/INI1-deficient undifferentiated pancreatic rhabdoid carcinoma mimicking solid pseudopapillary neoplasm.

Beyond right or wrong: More effective feedback for formative multiple-choice tests.

INTRODUCTION: The role of feedback in test-enhanced learning is an understudied area that has the potential to improve student learning. This study investigates the influence of different forms of post-test feedback on retention and transfer of biomedical knowledge within a test-enhanced learning framework. METHODS: 64 participants from a Canadian and an Australian medical school sat two single-best-answer formative multiple choice tests one week apart. We compared the effects of conceptually focused, response-oriented, and simple right/wrong feedback on a learner's ability to correctly answer new (transfer) questions. On the first test occasion, participants received parent items with feedback, and then attempted items closely related (near transfer) to and more distant (far transfer) from parent items. In a repeat test at 1 week, participants were given different near and far transfer versions of parent items. Feedback type, and near and far transfer items were randomized within and across participants. RESULTS: Analysis demonstrated that response-oriented and conceptually focused feedback were superior to traditional right/wrong feedback for both types of transfer tasks and in both immediate and final retention test performance. However, there was no statistically significant difference between response-orientated and conceptually focused groups on near or far transfer problems, nor any differences in performance between our initial test occasion and the retention test 1 week later. As with most studies of transfer, participants' far transfer scores were lower than for near transfer. DISCUSSION: Right/wrong feedback appears to have limited potential to augment test-enhanced learning. Our work suggests that item-level feedback and feedback that identifies and elaborates on key conceptual knowledge are two important areas for future research on learning, retention and transfer.

Repeated testing improves long-term retention relative to repeated study: a randomised controlled trial.

CONTEXT: Laboratory studies in cognitive psychology with relatively brief final recall intervals suggest that repeated retrieval in the form of tests may result in better retention of information compared with repeated study. OBJECTIVES: Our study evaluates if repeated testing of material taught in a real-life educational setting (a didactic conference for paediatric and emergency medicine residents) replicates these findings when measured at a more educationally relevant final recall interval of 6 months. METHODS: Residents participated in an interactive teaching session on two topics: (i) status epilepticus, and (ii) myasthenia gravis. Residents were randomised to two counter-balanced groups which either took tests on status epilepticus and studied a review sheet on myasthenia gravis (SE-T/MG-S group) or took tests on myasthenia gravis and studied a review sheet on status epilepticus (MG-T/SE-S group). Testing and studying occurred immediately after teaching and then at two additional times at intervals of about 2 weeks. Residents received feedback after each test. Tests consisted of short-answer questions and the review sheets consisted of information identical to that on the answer sheets for the tests. At about 6 months residents took a final test on both topics. RESULTS: Nineteen residents in the SE-T/MG-S group and 21 residents in the MG-T/SE-S group completed the study. Collapsing across groups, repeated testing produced final test scores that were an average of 13% higher than those produced by repeated study (39% versus 26%) at > 6 months after the initial teaching session (t[78] = 3.93, standard error of the difference = 0.03, P < 0.001, d = 0.91). CONCLUSIONS: Repeated testing with feedback appears to result in significantly greater long-term retention of information taught in a didactic conference than repeated, spaced study. Testing should be considered for its potential impact on learning and not only as an assessment device.

Test-enhanced learning in medical education.

CONTEXT: In education, tests are primarily used for assessment, thus permitting teachers to assess the efficacy of their curriculum and to assign grades. However, research in cognitive psychology has shown that tests can also directly affect learning by promoting better retention of information, a phenomenon known as the testing effect. COGNITIVE PSYCHOLOGY RESEARCH: Cognitive psychology laboratory studies show that repeated testing of information produces superior retention relative to repeated study, especially when testing is spaced out over time. Tests that require effortful retrieval of information, such as short-answer tests, promote better retention than tests that require recognition, such as multiple-choice tests. The mnemonic benefits of testing are further enhanced by feedback, which helps students to correct errors and confirm correct answers. APPLICATION TO MEDICAL EDUCATION: Medical educational research has focused extensively on assessment issues. Such assessment research permits the conclusion that clinical expertise is founded on a broad fund of knowledge and effective memory networks that allow easy access to that knowledge. Test-enhanced learning can potentially strengthen clinical knowledge that will lead to improved expertise. CONCLUSIONS: Tests should be given often and spaced out in time to promote better retention of information. Questions that require effortful recall produce the greatest gains in memory. Feedback is crucial to learning from tests. Test-enhanced learning may be an effective tool for medical educators to use in promoting retention of clinical knowledge.

Commentary: Paradigms, Axiology, and Praxeology in Medical Education Research.

This supplement includes 11 research papers accepted by the 2018 Research in Medical Education Committee. In this Commentary, the authors draw the attention of medical educators and researchers to questions about the researcher's philosophical paradigm. Such questions fundamentally guide research-the choice of conceptual frameworks, methods, and methodology. The authors provide a brief overview of research paradigms and the related concept of axiology and praxeology in medical education research. The authors map the 11 accepted research articles, describing the researchers' stated or implied worldviews and the impact on chosen methods. The authors close by encouraging researchers to state the research paradigm behind their research, comment on how they ensured that the methods used displayed consistency with that paradigm, and highlight the value the research adds to everyday education.

Using reflection to influence practice: student perceptions of daily reflection in clinical education.

PURPOSE: Reflection is a key element in learning from experience, but the impact of most programmes of reflection on daily practice remains unclear. We investigated students' perceptions of adding a daily written reflection assignment to a clinical rotation. METHODS: Third-year medical students on a single two-week rotation completed daily reflections analyzing their performance. Programme evaluation used a 33-question anonymized survey. Quantitative data were summarized and qualitative responses coded for recurring themes. RESULTS: Twenty-six students completed the survey (90 % response rate). Eighty-five percent of students felt that the daily reflections had a positive impact on their learning from clinical experience. Seventy-seven percent of students reported that the programme changed their awareness of their thoughts and actions, and 80 % felt that it improved their recall of experiences. A greater sense of mindfulness and focus on self-improvement were major themes that emerge from students' descriptions of the role of daily reflections in their learning. CONCLUSION: Overall, daily reflections demonstrated a positive learning influence. This exploratory study suggests students may benefit from more frequent, short reflections as opposed to more typically spaced reflective assignments.

Student-directed retrieval practice is a predictor of medical licensing examination performance.

INTRODUCTION: A large body of evidence indicates that retrieval practice (test-enhanced learning) and spaced repetition increase long-term information retention. Implementation of these strategies in medical curricula is unfortunately limited. However, students may choose to apply them autonomously when preparing for high-stakes, cumulative assessments, such as the United States Medical Licensing Examination Step 1. We examined the prevalence of specific self-directed methods of testing, with or without spaced repetition, among preclinical students and assessed the relationship between these methods and licensing examination performance. METHODS: Seventy-two medical students at one institution completed a survey concerning their use of user-generated (Anki) or commercially-available (Firecracker) flashcards intended for spaced repetition and of boards-style multiple-choice questions (MCQs). Other information collected included Step 1 score, past academic performance (Medical College Admission Test [MCAT] score, preclinical grades), and psychological factors that may have affected exam preparation or performance (feelings of depression, burnout, and test anxiety). RESULTS: All students reported using practice MCQs (mean 3870, SD 1472). Anki and Firecracker users comprised 31 and 49 % of respondents, respectively. In a multivariate regression model, significant independent predictors of Step 1 score included MCQs completed (unstandardized beta coefficient [B] = 2.2 × 10- 3, p < 0.001), unique Anki flashcards seen (B = 5.9 × 10- 4, p = 0.024), second-year honours (B = 1.198, p = 0.002), and MCAT score (B = 1.078, p = 0.003). Test anxiety was a significant negative predictor (B= - 1.986, p < 0.001). Unique Firecracker flashcards seen did not predict Step 1 score. Each additional 445 boards-style practice questions or 1700 unique Anki flashcards was associated with an additional point on Step 1 when controlling for other academic and psychological factors. CONCLUSIONS: Medical students engage extensively in self-initiated retrieval practice, often with spaced repetition. These practices are associated with superior performance on a medical licensing examination and should be considered for formal support by educators.