Retrieval practice with short-answer, multiple-choice, and hybrid tests.

Retrieval practice improves meaningful learning, and the most frequent way of implementing retrieval practice in classrooms is to have students answer questions. In four experiments (N=372) we investigated the effects of different question formats on learning. Students read educational texts and practised retrieval by answering short-answer, multiple-choice, or hybrid questions. In hybrid conditions students first attempted to recall answers in short-answer format, then identified answers in multiple-choice format. We measured learning 1 week later using a final assessment with two types of questions: those that could be answered by recalling information verbatim from the texts and those that required inferences. Practising retrieval in all format conditions enhanced retention, relative to a study-only control condition, on both verbatim and inference questions. However, there were little or no advantages of answering short-answer or hybrid format questions over multiple-choice questions in three experiments. In Experiment 4, when retrieval success was improved under initial short-answer conditions, there was an advantage of answering short-answer or hybrid questions over multiple-choice questions. The results challenge the simple conclusion that short-answer questions always produce the best learning, due to increased retrieval effort or difficulty, and demonstrate the importance of retrieval success for retrieval-based learning activities.

Retrieval Practice and Word Learning by Children With Developmental Language Disorder: Does Expanding Retrieval Provide Additional Benefit?

PURPOSE: The word learning of preschool-age children with developmental language disorder (DLD) is improved when spaced retrieval practice is incorporated into the learning sessions. In this preregistered study, we compared two types of spacing-an expanding retrieval practice schedule and an equally spaced schedule-to determine if one of these approaches yields better word learning outcomes for the children. METHOD: Fourteen children with DLD aged 4-5 years and 14 same-age children with typical language development (TD) learned eight novel nouns over two sessions. Spacing for half of the novel words was expanded gradually during learning; for the remaining novel words, greater spacing remained at the same level throughout learning. Immediately after the second session and 1 week later, the children's recall of the words was tested. RESULTS: The children with TD recalled more novel words than the children with DLD, although this difference could be accounted for by differences in the children's standardized receptive vocabulary test scores. The two groups were similar in their ability to retain the words over 1 week. Initially, the shorter spacing in the expanding schedule resulted in greater retrieval success than the corresponding (longer spaced) retrieval trials in the equally spaced schedule. These early shorter spaced trials also seemed to benefit retrieval of the trials with greater spacing that immediately followed. However, as the learning period progressed, the accuracy levels for the two conditions converged and were likewise similar during final testing. CONCLUSION: We need a greater understanding of how and when short spacing can be helpful to children's word learning, with the recognition that early gains might give a misleading picture of the benefits that short spacing can provide to longer term retention. SUPPLEMENTAL MATERIAL: https://doi.org/10.23641/asha.25537696.

Can Retrieval Practice Facilitate Verb Learning in Children With Developmental Language Disorder and Their Peers With Typical Language Development?

PURPOSE: Children with developmental language disorder (DLD) have well-documented verb learning difficulties. In this study, we asked whether the inclusion of retrieval practice during the learning period would facilitate these children's verb learning relative to a similar procedure that provided no retrieval opportunities. METHOD: Eleven children with DLD (M age = 60.09 months) and 12 children with typical language development (TD; M age = 59.92 months) learned four novel verbs in a repeated spaced retrieval (RSR) condition and four novel verbs in a repeated study (RS) condition. The words in the two conditions were heard an equal number of times, in the context of video-recorded actors performing novel actions. RESULTS: Recall testing immediately after the learning period and 1 week later revealed greater recall for novel verbs in the RSR condition than for novel verbs in the RS condition. This was true for both groups, and for immediate as well as 1-week testing. The RSR advantage remained when children had to recall the novel verbs while watching new actors perform the novel actions. However, when tested in contexts requiring the children to inflect the novel verbs with -ing for the first time, the children with DLD were much less likely to do so than their peers with TD. Even words in the RSR condition were only inconsistently inflected. CONCLUSIONS: Retrieval practice provides benefits to verb learning-an important finding given the challenges that verbs present to children with DLD. However, these benefits do not appear to automatically translate to the process of adding inflections to newly learned verbs but rather appear to be limited to the operations of learning the verbs' phonetic forms and mapping these forms onto associated actions.

When and why do retrieval attempts enhance subsequent encoding?

Unsuccessful retrieval attempts can enhance subsequent encoding and learning. In three experiments, subjects either attempted to retrieve word pairs prior to studying them (e.g., attempting to recall tide-? before studying tide-beach) or did not attempt retrieval and retention of the studied targets was assessed on a subsequent cued recall test. Experiment 1 showed that attempting retrieval enhanced subsequent encoding and recall relative to not attempting retrieval when the word pairs were semantically related, but not when the pairs were unrelated. In Experiment 2, studying a different word pair prior to the correct pair (e.g., studying tide-wave prior to tide-beach) did not produce the same effect as attempting retrieval prior to studying. Constraining retrieval to a particular candidate word prior to study (e.g., recalling tide-wa__ before studying tide-beach) produced a negative effect on subsequent recall. Experiment 3 showed that attempting retrieval did not enhance encoding when a brief delay occurred between the retrieval attempt and the subsequent study trial. The results support the idea that a search set of candidates related to the retrieval cue is activated during retrieval and that this retrieval-specific activation can enhance subsequent encoding of those candidates.

Automatic processing influences free recall: converging evidence from the process dissociation procedure and remember-know judgments.

Dual-process theories of retrieval suggest that controlled and automatic processing contribute to memory performance. Free recall tests are often considered pure measures of recollection, assessing only the controlled process. We report two experiments demonstrating that automatic processes also influence free recall. Experiment 1 used inclusion and exclusion tasks to estimate recollection and automaticity in free recall, adopting a new variant of the process dissociation procedure. Dividing attention during study selectively reduced the recollection estimate but did not affect the automatic component. In Experiment 2, we replicated the results of Experiment 1, and subjects additionally reported remember-know-guess judgments during recall in the inclusion condition. In the latter task, dividing attention during study reduced remember judgments for studied items, but know responses were unaffected. Results from both methods indicated that free recall is partly driven by automatic processes. Thus, we conclude that retrieval in free recall tests is not driven solely by conscious recollection (or remembering) but also by automatic influences of the same sort believed to drive priming on implicit memory tests. Sometimes items come to mind without volition in free recall.

Word-learning trajectories influence long-term recall in children with developmental language disorder and typical development.

INTRODUCTION: Children with developmental language disorder (DLD) have difficulties learning words. However, the severity of these difficulties can be reduced through word learning procedures that incorporate repeated spaced retrieval (RSR). Previous studies have shown positive outcomes with RSR but we still know very little about how learning unfolds with this procedure. In this study, we focus on the process of learning under RSR. METHODS: We analyze the learning data from children with DLD and with typical development (TD) from three recent studies using RSR. We fit growth curves to the trial-by-trial learning data for immediate and spaced retrieval trials and use the characteristics of children's individual word-learning trajectories to predict their long-term recall. RESULTS: We found that children in both groups demonstrated linear growth during learning, despite breaks within and across days. Success on early immediate retrieval trials promoted success on subsequent spaced retrieval trials. Children's trial-by-trial RSR learning trajectories were related to their long-term recall: both cumulative success and growth over time on the spaced retrieval trials positively predicted long-term recall. Consecutive success was also highly correlated with growth over time. CONCLUSIONS: Repeated spaced retrieval supplemented with immediate retrieval trials during learning promotes long-term recall.

The Neural Underpinnings of Processing Newly Taught Semantic Information: The Role of Retrieval Practice.

Purpose Recent behavioral studies have demonstrated the effectiveness of implementing retrieval practice into learning tasks for children. Such approaches have revealed that repeated spaced retrieval (RSR) is particularly effective in promoting children's learning of word form and meaning information. This study further examines how retrieval practice enhances learning of word meaning information at the behavioral and neural levels. Method Twenty typically developing preschool children were taught novel words using an RSR learning schedule for some words and an immediate retrieval (IR) learning schedule for other words. In addition to the label, children were taught two arbitrary semantic features for each item. Following the teaching phase, children's learning was tested using recall tests. In addition, during the 1-week follow-up, children were presented with pictures and an auditory sentence that correctly labeled the item but stated correct or incorrect semantic information. Event-related brain potentials (ERPs) were time locked to the onset of the words noting the semantic feature. Children provided verbal judgments of whether the semantic feature was correctly paired with the item. Results Children recalled more labels and semantic features for items that had been taught in the RSR learning schedule relative to the IR learning schedule. ERPs also differentiated the learning schedules. Mismatching label-meaning pairings elicited an N400 and late positive component (LPC) for both learning conditions; however, mismatching RSR pairs elicited an N400 with an earlier onset and an LPC with a longer duration, relative to IR mismatching label-meaning pairings. These ERP timing differences indicated that the children were more efficient in processing words that were taught in the RSR schedule relative to the IR learning schedule. Conclusions Spaced retrieval practice promotes learning of both word form and meaning information. The findings lay the necessary groundwork for better understanding of processing newly learned semantic information in preschool children. Supplemental Material https://doi.org/10.23641/asha.15063060.

A multi-study examination of the role of repeated spaced retrieval in the word learning of children with developmental language disorder.

BACKGROUND: Many children with developmental language disorders (DLD) have well-documented weaknesses in vocabulary. In recent years, investigators have explored the nature of these weaknesses through the use of novel word learning paradigms. These studies have begun to uncover specific areas of difficulty and have provided hints about possible intervention strategies that might help these children learn words more accurately and efficiently. Among the studies of this type are those that incorporate repeated spaced retrieval activities in the learning procedures. METHODS: In this study, we examined the data from four of these studies that employed the same types of participants (4- and 5-year-old children with DLD and same-age children with typical language development), research design, and outcome measures. The studies differed primarily in the type of learning condition that was being compared to a spaced retrieval condition. A mixed-effects modeling framework was used, enabling the data from the four studies and different outcome measures to be aggregated. RESULTS: Across the studies, more words in the repeated spaced retrieval condition were recalled than those in the comparison conditions. This was true regardless of outcome measure. Children with typical language development recalled more words than the children with DLD. Both groups benefited from spaced retrieval, though effects were larger for the group with DLD. Children recalled words as accurately 1 week after learning as they did at the 5-min mark; the two groups were essentially identical in this respect. CONCLUSIONS: Overall, the findings support the continued refinement of these types of repeated spaced retrieval procedures, as they may have potential to serve as effective approaches to intervention.

Is expanding retrieval a superior method for learning text materials?

Expanding retrieval practice refers to the idea that gradually increasing the spacing interval between repeated tests ought to promote optimal long-term retention. Belief in the superiority of this technique is widespread, but empirical support is scarce. In addition, virtually all research on expanding retrieval has examined the learning of word pairs in paired-associate tasks. We report two experiments in which we examined the learning of text materials with expanding and equally spaced retrieval practice schedules. Subjects studied brief texts and recalled them in an initial learning phase. We manipulated the spacing of the repeated recall tests and examined final recall 1 week later. Overall we found that (1) repeated testing enhanced retention more than did taking a single test, (2) testing with feedback (restudying the passages) produced better retention than testing without feedback, but most importantly (3) there were no differences between expanding and equally spaced schedules of retrieval practice. Repeated retrieval enhanced long-term retention, but how the repeated tests were spaced did not matter.

After Initial Retrieval Practice, More Retrieval Produces Better Retention Than More Study in the Word Learning of Children With Developmental Language Disorder.

Purpose Children with developmental language disorder (DLD) often have difficulty with word learning. Recent studies have shown that incorporating retrieval practice provides a significant benefit to this learning. However, we have not yet discovered the best balance between the amount of retrieval and the amount of study (hearing the word in the presence of the referent) that is provided. In this investigation, we compared a word learning procedure using more retrieval and less study with a procedure that used more study and less retrieval. Method Participants were 13 children with DLD and 13 same-age peers with typical language development (TD). Both groups ranged in age from 4 to 6 years. The children learned two sets of novel words, with each set taught in two sessions. During an initial criterion period, the children had the opportunity to retrieve all of the words. Following this period, the words were either retrieved without further study or studied without additional retrieval. Recall and recognition testing immediately followed the second learning session and was repeated 1 week later. Testing assessed the children's retention of both the word forms and their meanings. Results Better recall both immediately after learning and after 1 week was seen for the more retrieval/less study condition. This was seen for both groups of children for word form recall and for children with DLD for meaning. Group differences were not found. Conclusion This study served as a stringent test of the benefits of retrieval to children's word learning. Continued retrieval after initial retrieval practice appeared to be helpful even when further study was discontinued and when the comparison study condition had also provided retrieval practice in the initial stages. Further refinement of retrieval procedures might lead to the development of useful clinical tools to promote word learning.

Metacognitive strategies in student learning: do students practise retrieval when they study on their own?

Basic research on human learning and memory has shown that practising retrieval of information (by testing the information) has powerful effects on learning and long-term retention. Repeated testing enhances learning more than repeated reading, which often confers limited benefit beyond that gained from the initial reading of the material. Laboratory research also suggests that students lack metacognitive awareness of the mnemonic benefits of testing. The implication is that in real-world educational settings students may not engage in retrieval practise to enhance learning. To investigate students' real-world study behaviours, we surveyed 177 college students and asked them (1) to list strategies they used when studying (an open-ended free report question) and (2) to choose whether they would reread or practise recall after studying a textbook chapter (a forced report question). The results of both questions point to the same conclusion: A majority of students repeatedly read their notes or textbook (despite the limited benefits of this strategy), but relatively few engage in self-testing or retrieval practise while studying. We propose that many students experience illusions of competence while studying and that these illusions have significant consequences for the strategies students select when they monitor and regulate their own learning.

Adjective Learning in Young Typically Developing Children and Children With Developmental Language Disorder: A Retrieval-Based Approach.

Purpose There are strong retention benefits when learners frequently test themselves during the learning period. This practice of repeated retrieval has recently been applied successfully to children's word learning. In this study, we apply a repeated retrieval procedure to the learning of novel adjectives by preschool-age children with developmental language disorder (DLD) and their typically developing (TD) peers. We ask whether the benefits of retrieval extend to children's ability to apply the novel adjectives to newly introduced objects sharing the same characteristics as the objects used during the learning period. Method Fourteen children with DLD (M age = 62.64 months) and 13 TD children (M = 62.54 months) learned novel adjectives in 2 sessions. For each child, half of the adjectives were learned in a repeated spaced retrieval condition, and half were learned in a repeated study-only condition. Recall was assessed immediately after the second learning session and 1 week later. A recognition test was also administered at the 1-week mark. Results On the recall tests, for both groups of children, recall was better for adjectives learned in the repeated spaced retrieval condition. Adjectives learned by the 2nd day were retained 1 week later. Every adjective correctly applied to an object used during the learning period was also extended accurately to new objects with the same characteristics. On these recall tests, the children with DLD did not differ from the TD group in the number of items recalled, though their phonetic accuracy was lower. On the recognition test, the DLD group showed greater accuracy for adjectives that had been learned in the repeated spaced retrieval condition than for those learned in the repeated study condition, whereas the TD group performed at high levels in both conditions. Conclusion Repeated spaced retrieval appears to provide an effective boost to word learning. Because its benefits are seen even when a word must be extended to new objects, the application of this procedure seems well suited for learning new language material rather than being limited to item-specific memorization.

Retrieval-Based Word Learning in Young Typically Developing Children and Children With Developmental Language Disorder I: The Benefits of Repeated Retrieval.

Purpose Scholars have long noted that retention improves significantly when learners frequently test themselves on the new material rather than engage in continuous study with no intermittent testing. In this study, we apply the notion of repeated testing or retrieval to the process of word learning in preschool-age children with and without developmental language disorder (DLD). Method Novel words and their meanings were taught to 10 children with DLD and 10 typically developing (TD) children matched on age (DLD, M = 63.4 months; TD, M = 63.2 months). Recall was assessed immediately after the 2nd learning session and then again 1 week later. Results Both groups showed better retention when they had attempted to retrieve the words during the learning period than when they had simply listened to and studied the words paired with their referents. Relative to their TD peers, the children with DLD seemed to be weaker in their encoding, but these children's retention over a 1-week period was indistinguishable from that of their age mates. Conclusion Word learning activities that include opportunities for repeated retrieval appear to significantly benefit retention relative to more traditional word learning activities. Supplemental Material https://doi.org/10.23641/asha.7927046.

Retrieval-Based Word Learning in Young Typically Developing Children and Children With Development Language Disorder II: A Comparison of Retrieval Schedules.

Purpose Retrieval practice has been found to be a powerful strategy to enhance long-term retention of new information; however, the utility of retrieval practice when teaching young children new words is largely unknown, and even less is known for young children with language impairments. The current study examined the effect of 2 different retrieval schedules on word learning at both the behavioral and neural levels. Method Participants included 16 typically developing children ( M TD = 61.58 months) and 16 children with developmental language disorder ( M DLD = 59.60 months). Children participated in novel word learning sessions in which the spacing of retrieval practice was manipulated: Some words were retrieved only after other words had been presented (i.e., repeated retrieval that required contextual reinstatement [RRCR]); others were taught using an immediate retrieval schedule. In Experiment 1, children's recall of the novel word labels and their meanings was tested after a 5-min delay and a 1-week delay. In Experiment 2, event-related brain potentials were obtained from a match-mismatch task utilizing the novel word stimuli. Results Experiment 1 findings revealed that children were able to label referents and to retain the novel words more successfully if the words were taught in the RRCR learning condition. Experiment 2 findings revealed that mismatching picture-word pairings elicited a robust N400 event-related brain potential only for words that were taught in the RRCR condition. In addition, children were more accurate in identifying picture-word matches and mismatches for words taught in the RRCR condition, relative to the immediate retrieval condition. Conclusions Retrieval practice that requires contextual reinstatement through spacing results in enhanced word learning and long-term retention of words. Both typically developing children and children with developmental language disorder benefit from this type of retrieval procedure. Supplemental Material https://doi.org/10.23641/asha.7927112.

Correcting a metacognitive error: feedback increases retention of low-confidence correct responses.

Previous studies investigating posttest feedback have generally conceptualized feedback as a method for correcting erroneous responses, giving virtually no consideration to how feedback might promote learning of correct responses. Here, the authors show that when correct responses are made with low confidence, feedback serves to correct this initial metacognitive error, enhancing retention of low-confidence correct responses. In 2 experiments, subjects took an initial multiple-choice test on general knowledge facts and made a confidence judgment after each response. Feedback was provided for half of the questions, and retention was assessed by a final cued-recall test. Taking the initial test improved retention relative to not testing, and feedback further enhanced performance. Consistent with prior research, feedback improved retention by allowing subjects to correct initially erroneous responses. Of more importance, feedback also doubled the retention of correct low-confidence responses, relative to providing no feedback. The function of feedback is to correct both memory errors and metacognitive errors.

Improving self-regulated learning with a retrieval practice intervention.

Repeated retrieval practice is a powerful learning tool for promoting long-term retention, but students use this tool ineffectively when regulating their learning. The current experiments evaluated the efficacy of a minimal intervention aimed at improving students' self-regulated use of repeated retrieval practice. Across 2 experiments, students made decisions about when to study, engage in retrieval practice, or stop learning a set of foreign language word pairs. Some students received direct instruction about how to use repeated retrieval practice. These instructions emphasized the mnemonic benefits of retrieval practice over a less effective strategy (restudying) and told students how to use repeated retrieval practice to maximize their performance-specifically, that they should recall a translation correctly 3 times during learning. This minimal intervention promoted more effective self-regulated use of retrieval practice and better retention of the translations compared to a control group that received no instruction. Students who experienced this intervention also showed potential for long-term changes in self-regulated learning: They spontaneously used repeated retrieval practice 1 week later to learn new materials. These results provide a promising first step for developing guidelines for teaching students how to regulate their learning more effectively using repeated retrieval practice. (PsycINFO Database Record

Reflections on the Resurgence of Interest in the Testing Effect.

We discuss the findings from our 2006 article in Psychological Science on the testing effect and describe how the project arose. The testing effect (or retrieval-practice effect) was first reported in the experimental literature about a century before our article was published, and the effect had been replicated (and sometimes discovered anew) many times over the years. Our experiments used prose materials (unlike most prior research) and produced a more powerful effect than prior research even though we used a conservative control condition for comparison. In our discussion, we drew out possible implications for educational practice. We also reported that students in the experiment could not predict the effect; this lack of metacognitive awareness represented a new finding in this context. In a companion article the same year, we provided an historical review of the testing effect. We believe the synergistic effect of the two articles accounts in part for the resurgence in interest in this phenomenon and its application in educational settings.

Expanding retrieval practice promotes short-term retention, but equally spaced retrieval enhances long-term retention.

Expanding retrieval practice (T. K. Landauer & R. A. Bjork, 1978) is regarded as a superior technique for promoting long-term retention relative to equally spaced retrieval practice. In Experiments 1 and 2, the authors found that expanding retrieval practice of vocabulary word pairs produced short-term benefits 10 min after learning, conceptually replicating Landauer and Bjork's results. However, equally spaced retrieval produced superior retention 2 days later. This pattern occurred both with and without feedback after test trials. In Experiment 3, the 1st test occurred immediately or after a brief delay, and repeated tests were expanding or equally spaced. Delaying the first test improved long-term retention, regardless of how the repeated tests were spaced. The important factor for promoting long-term retention is delaying initial retrieval to make it more difficult, as is done in equally spaced retrieval but not in expanding retrieval. Expanding the interval between repeated tests had little effect on long-term retention in 3 experiments.

The effect of type and timing of feedback on learning from multiple-choice tests.

Two experiments investigated how the type and timing of feedback influence learning from a multiple-choice test. First, participants read 12 prose passages, which covered various general knowledge topics (e.g., The Sun) and ranged between 280 and 300 words in length. Next, they took an initial six-alternative, multiple-choice test on information contained in the passages. Feedback was given immediately for some of the multiple-choice items or after delay for other items. Participants were either shown the correct answer as feedback (standard feedback) or were allowed to keep answering until the correct answer was discovered (answer-until-correct feedback). Learning from the test was assessed on a delayed cued-recall test. The results indicated that delayed feedback led to superior final test performance relative to immediate feedback. However, type of feedback did not matter: discovering the correct answer through answer-until-correct feedback produced equivalent performance relative to standard feedback. This research suggests that delaying the presentation of feedback after a test is beneficial to learning because of the spaced presentation of information.

The role of episodic context in retrieval practice effects.

The episodic context account of retrieval-based learning proposes that retrieval enhances subsequent retention because people must think back to and reinstate a prior learning context. Three experiments directly tested this central assumption of the context account. Subjects studied word lists and then either restudied the words under intentional learning conditions or made list discrimination judgments by indicating which list each word had occurred in originally. Subjects in both conditions experienced all items for the same amount of time, but subjects in the list discrimination condition were required to retrieve details about the original episodic context in which the words had occurred. Making initial list discrimination judgments consistently enhanced subsequent free recall relative to restudying the words. Analyses of recall organization and retrieval strategies on the final test showed that retrieval practice enhanced temporal organization during final recall. Semantic encoding tasks also enhanced retention relative to restudying but did so by promoting semantic organization and semantically based retrieval strategies during final recall. The results support the episodic context account of retrieval-based learning. (PsycINFO Database Record