Judgments of learning enhance recall for category-cued but not letter-cued items.

Making immediate judgments of learning (JOLs) during study can influence later memory performance, with a common outcome being that JOLs improve cued-recall performance for related word pairs (i.e., positive reactivity) and do not impact memory for unrelated pairs (i.e., no reactivity). The cue-strengthening hypothesis proposes that JOL reactivity will be observed when a criterion test is sensitive to the cues used to inform JOLs (Soderstrom et al., Journal of Experimental Psychology: Learning, Memory, and Cognition, 41 (2), 553-558, 2015). Across four experiments, we evaluated this hypothesis with category pairs (e.g., A type of gem - Jade) and letter pairs (e.g., Ja - Jade). Participants studied a list comprised of both pair types, made (or did not make) JOLs, and completed a cued-recall test (Experiments 1a/b). The cue-strengthening hypothesis predicts greater positive reactivity for category pairs than for letter pairs, because making a JOL strengthens the relationship between the cue and target, which is more beneficial for material with an a priori semantic relationship. Outcomes were consistent with this hypothesis. We also evaluated and ruled out alternative explanations for this pattern of effects: (a) that they arose due to overall differences in recall performance for the two pair types (Experiment 2); (b) that they would also occur even when the criterion test is not sensitive to the cues used to inform JOLs (Experiment 3); and (c) that JOLs only increased memory strength for the targets (Experiment 4). Thus, the current experiments rule out plausible accounts of reactivity effects and provide further, converging evidence for the cue-strengthening hypothesis.

What constrains people's ability to learn about the testing effect through task experience?

What constrains people's ability to learn from experience about the effectiveness of practice testing versus restudying for memory (i.e., the testing effect)? Across two cycles, participants studied word pairs, practiced each pair through either restudying or testing, predicted how many pairs they would recall for each strategy, then completed a critical test on the pairs. During this test, participants either received feedback about the number of pairs they had correctly recalled or made postdictions about their performance for each strategy (i.e., generated their own feedback). During both cycles, participants predicted they would recall an equivalent number of tested and restudied pairs, although they actually recalled more tested pairs. However, when participants experienced a larger testing effect, they estimated recall performance more accurately for each strategy and updated their knowledge about the testing effect. Thus, peoples' ability to learn from experience about the testing effect is primarily constrained not by a failure to initiate the metacognitive processes required to monitor and track recall performance by strategy, but by the metacognitive burden of discriminating between small differences in recall between tested versus restudied material. In summary, people can learn from experience about the testing effect when the metacognitive burden is lifted.

Investigating memory reactivity with a within-participant manipulation of judgments of learning: support for the cue-strengthening hypothesis.

When learners make judgments of learning (JOLs) for some word pairs but not others, how and why is recall performance affected? Participants studied related and unrelated word pairs and made JOLs for a randomly selected half of the pairs. We evaluated two hypotheses. The changed-goal hypothesis states that making JOLs leads learners to notice differences in pair difficulty and to change their learning goal. Because JOLs are manipulated within participants, such a goal change should influence how all (judged or non-judged) pairs are processed on the list, which should lead to no JOL reactivity. The cue-strengthening hypothesis predicts greater positive reactivity (i.e., higher recall for judged versus non-judged pairs) for related than unrelated pairs, because making a JOL strengthens the relationship between the two words in a pair, which would be more beneficial for pairs with an a priori relationship. Across experiments, we found positive reactivity for both related and unrelated pairs (albeit to a lesser degree for the latter). We also found no evidence that learners make qualitative changes in their reported strategy use when judging pairs. Making JOLs for some pairs on a list influenced memory performance and the pattern of reactivity provided support for the cue-strengthening hypothesis.

Beliefs about memory decline in aging do not impact judgments of learning (JOLs): A challenge for belief-based explanations of JOLs.

The cue-utilization framework (Koriat, 1997) and the analytic processing theory (Dunlosky, Mueller, & Tauber, 2015) identify people's beliefs about their memory as central to how judgments of learning (JOLs) are made. This assumption is supported by ample evidence. However, researchers have almost exclusively explored the impact of participants' beliefs about the materials or the learning task, and none have evaluated the impact of beliefs about a person on JOLs. Thus, to inform JOL theory, we evaluated the degree to which JOLs are related to the belief that "memory declines with aging in adulthood." In seven experiments, college-aged participants studied words, made JOLs, and took a memory test. Participants made JOLs predicting memory performance for an average younger adult (i.e., 18-21 years old) or for an average older adult (i.e., 65+ years old). Most important, beliefs about aging in adulthood were not always sufficient to produce cue effects on JOLs, which contrasts with expectations from the aforementioned theories. An important challenge for future research will be to discover factors that moderate belief effects. To guide such explorations, we discuss possible explanations for why beliefs about aging would have demonstrated little to no relationship with people's JOLs.

Why does interleaving improve math learning? The contributions of discriminative contrast and distributed practice.

Interleaved practice involves studying exemplars from different categories in a non-systematic, pseudorandom order under the constraint that no two exemplars from the same category are presented consecutively. Interleaved practice of materials has been shown to enhance test performance compared to blocked practice in which exemplars from the same category are studied together. Why does interleaved practice produce this benefit? We evaluated two non-mutually exclusive hypotheses, the discriminative-contrast hypothesis and the distributed-practice hypothesis, by testing participants' performance on calculating the volume of three-dimensional geometric shapes. In Experiment 1, participants repeatedly practiced calculating the volume of four different-sized shapes according to blocked practice, interleaved practice, or remote-interleaved practice (which involved alternating the practice of volume calculation with non-volume problems, like permutations and fraction addition). Standard interleaving enhanced performance compared to blocked practice but did not produce enhanced performance compared to remote interleaving. In Experiment 2, we replicated this pattern and extended the results to include a remote-blocked group, which involved blocking volume calculation with non-volume problems. Performance on key measures was better for remote-interleaved groups compared to remote-blocked groups, a finding that supports the distributed-practice hypothesis.

Note-taking habits of 21st Century college students: implications for student learning, memory, and achievement.

Students rely on their notes to memorise and learn critical course content, and recent studies of note-taking state that most students take notes, citing a survey published in 1974. Over the past four decades, classrooms and note-taking technologies have evolved: students can take notes on electronic devices, and some classes are entirely online. Despite these changes, few studies have investigated their impact on note-taking habits. Given that note-taking is critical for student learning, the purpose of the present study was to update the literature on students' general note-taking behaviour and investigate the impact of these newer technological advances on note-taking. We surveyed 577 college students, using questions from prior surveys and new questions relevant to technology. As per prior studies, students still reported taking notes in classrooms, and as important, students reported flexibility in their note-taking by (a) using a notebook or a laptop depending on course demands and (b) often deciding not to take notes in online courses. Thus, students are flexible in their note-taking behaviour, but may not always make the best decisions about how and when to take notes.

An evidence-based approach to learning clinical anatomy: A guide for medical students, educators, and administrators.

The amount of information that medical students learn is voluminous and those who do not use evidence-based learning strategies may struggle. Research from cognitive and educational psychology provides a blueprint on how best to learn science subjects, including clinical anatomy. Students should aim for high-cognitive learning levels as defined in the SOLO taxonomy. Using a real-world example from a modern clinical anatomy textbook, we describe how to learn information using strategies that have been experimentally validated as effective. Students should avoid highlighting and rereading text because they do not result in robust learning as defined in the SOLO taxonomy. We recommend that students use (1) practice testing, (2) distributed practice, and (3) successive relearning. Practice testing refers to nonsummative assessments that contain questions used to facilitate retrieval (e.g., flashcards and practice questions). Practice questions can be fill-in, short-answer, and multiple-choice types, and students should receive explanatory feedback. Distributed practice, the technique of distributing learning of the same content within a single study session or across sessions, has been found to facilitate long-term retention. Finally, successive relearning combines both practice testing and distributed practice. For this strategy, students use practice questions to continue learning until they can answer all of the practice questions correctly. Students who continuously use practice testing, distributed practice, and successive relearning will become more efficient and effective learners. Our hope is that the real-world clinical anatomy example presented in this article makes it easier for students to implement these evidence-based strategies and ultimately improve their learning. Clin. Anat., 2018. © 2018 The Authors. Clinical Anatomy published by Wiley Periodicals, Inc. on behalf of American Association of Clinical Anatomists.

Does mediator use contribute to the spacing effect for cued recall? Critical tests of the mediator hypothesis.

When study is spaced across sessions (versus massed within a single session), final performance is greater after spacing. This spacing effect may have multiple causes, and according to the mediator hypothesis, part of the effect can be explained by the use of mediator-based strategies. This hypothesis proposes that when study is spaced across sessions, rather than massed within a session, more mediators will be generated that are longer lasting and hence more mediators will be available to support criterion recall. In two experiments, participants were randomly assigned to study paired associates using either a spaced or massed schedule. They reported strategy use for each item during study trials and during the final test. Consistent with the mediator hypothesis, participants who had spaced (as compared to massed) practice reported using more mediators on the final test. This use of effective mediators also statistically accounted for some - but not all of - the spacing effect on final performance.

Monitoring and regulation of learning in medical education: the need for predictive cues.

CONTEXT: Being able to accurately monitor learning activities is a key element in self-regulated learning in all settings, including medical schools. Yet students' ability to monitor their progress is often limited, leading to inefficient use of study time. Interventions that improve the accuracy of students' monitoring can optimise self-regulated learning, leading to higher achievement. This paper reviews findings from cognitive psychology and explores potential applications in medical education, as well as areas for future research. COGNITIVE PSYCHOLOGY: Effective monitoring depends on students' ability to generate information ('cues') that accurately reflects their knowledge and skills. The ability of these 'cues' to predict achievement is referred to as 'cue diagnosticity'. Interventions that improve the ability of students to elicit predictive cues typically fall into two categories: (i) self-generation of cues and (ii) generation of cues that is delayed after self-study. Providing feedback and support is useful when cues are predictive but may be too complex to be readily used. APPLICATION TO MEDICAL EDUCATION: Limited evidence exists about interventions to improve the accuracy of self-monitoring among medical students or trainees. Developing interventions that foster use of predictive cues can enhance the accuracy of self-monitoring, thereby improving self-study and clinical reasoning. First, insight should be gained into the characteristics of predictive cues used by medical students and trainees. Next, predictive cue prompts should be designed and tested to improve monitoring and regulation of learning. Finally, the use of predictive cues should be explored in relation to teaching and learning clinical reasoning. CONCLUSIONS: Improving self-regulated learning is important to help medical students and trainees efficiently acquire knowledge and skills necessary for clinical practice. Interventions that help students generate and use predictive cues hold the promise of improved self-regulated learning and achievement. This framework is applicable to learning in several areas, including the development of clinical reasoning.

Children's agenda-based regulation: The effects of prior performance and reward on elementary school children's study choices.

The main goal of the current experiments was to examine the influence of monitoring and reward on elementary school children's study decisions. First and third graders studied names for 10 animals (e.g., "The elephant's name is Suzy") and then were given a cued recall test on which they were shown the animal and needed to recall the name. Next, they were given an opportunity to restudy the animal-name pairs, and some of these pairs were slated to earn a reward (a sticker) if correctly recalled. In Experiment 1, both groups of children were (a) more likely to restudy previously unrecalled pairs than previously recalled pairs and (b) more likely to restudy pairs that were slated to receive a reward. In Experiment 2, we further explored children's use of reward using a forced-choice selection task. Namely, during selection, pairs were presented in dyads where one pair was slated for a reward and the other pair was not, and the children could choose only one pair from each dyad for restudy. Both first and third graders chose to restudy pairs slated for a reward. Thus, even young elementary school children consider both rewards and performance monitoring when regulating their learning.

Do people use category-learning judgments to regulate their learning of natural categories?

Although research has established that people can accurately judge how well they have learned categories, no research has examined whether people use their category-learning judgments (CLJs) to regulate their restudy of natural categories. Thus, in five experiments we investigated the relationship between people's CLJs and selections of categories for restudy. Participants first attempted to learn natural categories (bird families; e.g., finches, grosbeaks, and warblers) so that they could categorize new exemplars on a final test. After this initial study phase, participants made a CLJ for each category and then selected a subset of the categories for restudy. Across experiments, we also manipulated several variables (e.g., selecting either three or nine categories, or obtaining 30% vs. 80% performance on the final test) that were expected to influence restudy selections. However, the manipulations typically had minimal impact. More important, in all experiments we found an unexpected outcome: Some participants tended to select the categories they judged to be most well learned for restudy, and others tended to select those judged to be least well learned. We discovered these qualitative differences in the use of CLJs to make restudy selections by using post-hoc analyses in Experiments 1a and 1b, and hence we sought to (a) replicate them in Experiments 2, 3, and 4 and (b) provide preliminary evidence regarding factors that can (vs. cannot) account for them. Most important, evidence across all of the experiments supported the conclusion that people do use their CLJs to select categories for restudy.

The what, how much, and when of study strategies: comparing intended versus actual study behaviour.

The study behaviours of students can be assessed from several perspectives, such as what study strategies are used, the total number of hours of study, and the distribution of studying over time. Here, we present the results of a survey study that considered each of these perspectives by asking students to report the what, how much, and when of their study behaviours over the course of a semester. As important, to better understand students' use of study strategies, we also had students report at the beginning of the semester how they intended to study and their beliefs about the effectiveness of a variety of common strategies. Our results indicate that during the semester, students rely on relatively ineffective strategies and mass their studying the day or two before an exam. However, students intended to begin studying earlier and to use a mix of effective and ineffective study habits. Despite their use of some ineffective strategies, they did have a relatively accurate assessment of which strategies were less versus more effective. Taken together, our results suggest that students have some excellent intentions but may falter because massing study the evening before an exam limits their use of more effective study strategies.

Effects of successive relearning on recall: Does relearning override the effects of initial learning criterion?

Retrieval practice improves memory for many kinds of materials, and numerous factors moderate the benefits of retrieval practice, including the amount of successful retrieval practice (referred to as the learning criterion). In general, the benefits of retrieval practice are greater with more than with less successful retrieval practice; however, learning items to a higher (vs. lower) criterion requires more time and effort. If students plan on relearning material in a subsequent study session, does the benefit of learning to a higher criterion during an initial session persist? In Session 1, participants studied and successfully recalled Swahili-English word pairs one, two, three, four, five, six, or seven times. In subsequent sessions, all of the pairs were relearned to a criterion of one correct recall at one-week intervals across four or five successive relearning sessions. Experiments 1 and 2 revealed that the substantial benefits of learning to a higher initial criterion during the first session do not persist across relearning sessions. This relearning-override effect was also demonstrated in Experiment 2 after a one-month retention interval. The implications of relearning-override effects are important for theory and for education. For theories of test-enhanced learning, they support the predictions of one theory and appear inconsistent with the predictions of another. For education, if relearning is to occur, using extra time to learn to a higher initial learning criterion is not efficient. Instead, students should devote their time to subsequent spaced relearning sessions, which produce substantial gains in recall performance.

Why is knowledge updating after task experience incomplete? Contributions of encoding experience, scaling artifact, and inferential deficit.

Knowledge updating occurs when people learn about the impacts of variables on memory after experiencing their effects. For instance, judgments of learning (JOLs) for encoding strategies (e.g., imagery and repetition) show no difference during a first study-test trial; however, during a second trial, JOLs better reflect the benefits of the more effective strategy. Although this outcome indicates some knowledge updating, JOLs on a second trial rarely update to reflect the full impact of a given variable. We investigated several explanations for this incomplete updating. Evidence using prestudy JOLs from Experiments 1 and 2 disconfirmed the encoding-disrupts-updating (EDU) hypothesis, which is that the experience of encoding items on the second trial disrupts the use of new knowledge in making JOLs. In Experiment 3, we used binary JOLs to evaluate whether the lack of updating is an artifact of people not wanting to use extreme ratings, which accounted for some-but not all-of the incomplete updating. Finally, in Experiment 4, immediately after the test on the initial trial, participants received feedback about how many items they had recalled for each level of the focal variable, and their JOLs on the second trial still showed incomplete updating. Taken together, the evidence suggests that incomplete knowledge updating on JOLs arises from multiple factors, including a scaling artifact and the deficient use of accurate knowledge when making JOLs.

How do students improve their value-based learning with task experience?

When learning items that vary in reward, students improve their scores (i.e., earned reward) with task experience. In four experiments, we examined whether such improvements arise from better selective encoding of items that would earn more (vs. less) reward. Participants studied and recalled words across multiple study-test trials. On each trial, 12 words were slated with different values (typically from 1 to 12), and participants earned the point value assigned to a given word if it was correctly recalled. In all experiments, participants earned more points across the first two trials. In Experiment 1, participants either self-paced their study or had experimenter-paced study and in Experiment 2, some participants were penalised for each second spent during study. Improvements in points earned were related to increases in overall recall but not to selective encoding. In Experiment 3, some participants were given value-emphasised instructions, yet they did not demonstrate selective encoding. In Experiment 4, we used a larger range of point values, but selective encoding still did not account for the improvement in point scores across lists. These results suggest that metacognitively-driven selective encoding is not necessary to observe improvements in value-based learning.

Self-regulated learning: beliefs, techniques, and illusions.

Knowing how to manage one's own learning has become increasingly important in recent years, as both the need and the opportunities for individuals to learn on their own outside of formal classroom settings have grown. During that same period, however, research on learning, memory, and metacognitive processes has provided evidence that people often have a faulty mental model of how they learn and remember, making them prone to both misassessing and mismanaging their own learning. After a discussion of what learners need to understand in order to become effective stewards of their own learning, we first review research on what people believe about how they learn and then review research on how people's ongoing assessments of their own learning are influenced by current performance and the subjective sense of fluency. We conclude with a discussion of societal assumptions and attitudes that can be counterproductive in terms of individuals becoming maximally effective learners.

Does strategy training reduce age-related deficits in working memory?

BACKGROUND: Older adults typically perform worse on measures of working memory (WM) than do young adults; however, age-related differences in WM performance might be reduced if older adults use effective encoding strategies. OBJECTIVE: The purpose of the current experiment was to evaluate WM performance after training individuals to use effective encoding strategies. METHODS: Participants in the training group (older adults: n = 39; young adults: n = 41) were taught about various verbal encoding strategies and their differential effectiveness and were trained to use interactive imagery and sentence generation on a list-learning task. Participants in the control group (older: n = 37; young: n = 38) completed an equally engaging filler task. All participants completed a pre- and post-training reading span task, which included self-reported strategy use, as well as two transfer tasks that differed in the affordance to use the trained strategies - a paired-associate recall task and the self-ordered pointing task. RESULTS: Both young and older adults were able to use the target strategies on the WM task and showed gains in WM performance after training. The age-related WM deficit was not greatly affected, however, and the training gains did not transfer to the other cognitive tasks. In fact, participants attempted to adapt the trained strategies for a paired-associate recall task, but the increased strategy use did not benefit their performance. CONCLUSIONS: Strategy training can boost WM performance, and its benefits appear to arise from strategy-specific effects and not from domain-general gains in cognitive ability.

The contribution of judgment scale to the unskilled-and-unaware phenomenon: how evaluating others can exaggerate over- (and under-) confidence.

The unskilled-and-unaware phenomenon occurs when low performers tend to overestimate their performance on a task, whereas high performers judge their performance more accurately (and sometimes underestimate it). In previous research, this phenomenon has been observed for a variety of cognitive tasks and judgment scales. However, the role of judgment scale in producing the unskilled-and-unaware phenomenon has not been systematically investigated. Thus, we present four studies in which all participants judged their performance on both a relative scale (percentile rank) and an absolute scale (number correct). The studies included a variety of performance tasks (general knowledge questions, math problems, introductory psychology questions, and logic questions) and test formats (multiple-choice, recall). Across all tasks and formats, the percentile-rank judgments were less accurate than the absolute judgments, particularly for low and high performers. Furthermore, in Studies 1-3, the absolute judgments were highly accurate, even when the percentile-rank judgments were not. Thus, differences in the accuracy of percentile-rank judgments across skill levels do not always represent differences in self-awareness, but rather they may arise from difficulties that performers have at evaluating how well others are performing. Most importantly, the unskilled-and-unaware phenomenon on a relative scale does not guarantee inaccurate self-evaluations of absolute performance.

Recalled aspects of original encoding strategies influence episodic feelings of knowing.

We tested the hypothesis that the feeling of knowing (FOK) after a failed recall attempt is influenced by recalling aspects of the original encoding strategy. Individuals were instructed to use interactive imagery to encode unrelated word pairs. We manipulated item concreteness (abstract vs. concrete) and item repetitions at study (one vs. three). Participants orally described the mediator produced immediately after studying each item, if any. After a delay, they were given cued recall, made FOK ratings, and attempted to recall their original mediator. Concreteness and item repetition enhanced strategy recall, which had a large effect on FOKs. Controlling on strategy recall reduced the predictive validity of FOKs for recognition memory, indicating that access to the original aspects of encoding influenced FOK accuracy. Confidence judgments (CJs) for correctly recognized items covaried with FOKs, but FOKs did not fully track the strategy recall associations with CJs, suggesting emergent effects of strategy cues that were elicited by recognition tests but not accessed at the time of the FOK judgment. In summary, cue-generated access to aspects of the original encoding strategy strongly influenced episodic FOKs, although other influences were also implicated.

When do learners shift from habitual to agenda-based processes when selecting items for study?

Learners presumably attempt to allocate their study time to maximize reward, yet in some contexts, their study choices are driven by reading biases that would not maximize reward. For instance, when presented with items in a horizontal array that are worth different values if correctly recalled, learners will often first select the leftmost item (i.e., a reading bias), even when it is associated with the lowest value. In four experiments, we investigated the degrees to which various factors cause learners to shift to agenda-based regulation. On each trial, participants were presented with three cues and a point value (1, 3, or 5) for each. The participants could select any cue for study (in which case, its target would be presented) in any order. In Experiment 1, participants either selected items for study under time pressure or were given unlimited time to select items. Not limiting selection time increased the likelihood that higher-valued items would be prioritized for study, but reading biases still influenced item selection. In Experiment 2, participants could select only one item per trial, and higher-valued items were prioritized even more for study, but not exclusively so. In Experiments 3 and 4, we ruled out a lack of motivation and inaccurate task beliefs as explanations for why participants would sometimes choose lower-valued items. The results demonstrate the influence of a pervasive reading bias on learners' item selections, but as importantly, they show that a shift toward agenda use occurs when habitual responding cannot maximize reward.