A metacognitive illusion in monkeys.

Like humans, monkeys can make accurate judgements about their own memory by reporting their confidence during cognitive tasks. Some have suggested that animals use associative learning to make accurate confidence judgements, while others have suggested animals directly access and estimate the strength of their memories. Here we test a third, non-exclusive possibility: perhaps monkeys, like humans, base metacognitive inferences on heuristic cues. Humans are known to use cues like perceptual fluency (e.g. how easy something is to see) when making metacognitive judgements. We tested monkeys using a match-to-sample task in which the perceptual fluency of the stimuli was manipulated. The monkeys made confidence wagers on their accuracy before or after each trial. We found that monkeys' wagers were affected by perceptual fluency even when their accuracy was not. This is novel evidence that animals are susceptible to metacognitive illusions similar to those experienced by humans.

Performance bias: Why judgments of learning are not affected by learning.

Past research has shown a performance bias: People expect their future performance level on a task to match their current performance level, even when there are good reasons to expect future performance to differ from current performance. One explanation of this bias is that judgments are controlled by what learners can observe, and while current performance is usually observable, changes in performance (i.e., learning or forgetting) are not. This explanation makes a prediction that we tested here: If learning becomes observable, it should begin to affect judgments. In three experiments, after practicing a skill, participants estimated how they performed in the past and how they expected to perform in the future. In Experiments 1 and 2, participants knew they had been improving, as shown by their responses, yet they did not predict that they would improve in the future. This finding was particularly striking because (a) they did improve in the future and (b) as Experiment 3 showed, they did hold the conscious belief that past improvement predicted future improvement. In short, when learning and performance are both observable, judgments of learning seem to be guided by performance and not learning.

Retrieval attempts enhance learning regardless of time spent trying to retrieve.

Attempting to retrieve information from memory is an engaging cognitive activity. We predicted that people would learn more when they had spent more time attempting to retrieve. In experiments 1a and 1b, participants were shown trivia questions for 0, 5, 10, or 30 seconds and then the answer was revealed. They took a final test immediately or after 48 hours. Retrieval enhanced learning, but the length of the retrieval attempt had no effect (i.e., final test performance was equivalent in the 5-, 10-, and 30-second conditions and worse in the 0-second condition). During the initial retrieval attempt, more time did increase recall, suggesting that participants continued to engage in productive retrieval activities when given more time. Showing the answer for longer (7 versus 2 seconds) increased learning in Experiments 2a and 2b. Experiment 3 examined the effect of retrieval success and Experiment 4 replicated the results using different materials. These results have direct implications for current theories of retrieval.

The influence of feedback on predictions of future memory performance.

Kornell and Rhodes (Journal of Experimental Psychology: Applied, 19, 1-13, 2013) reported that correct answer feedback impairs the accuracy of prospective memory judgments. The current experiments explored the boundaries of this effect. In Experiment 1, participants studied Lithuanian-English word pairs, took an initial test, and were either given correct answer feedback or no feedback at all. They then made a judgment of learning (JOL) regarding the likelihood of correctly recalling the English translation on a later test. Presenting the correct answer as feedback increased average JOLs but impaired relative accuracy on a final test. Therefore, Experiments 2-4 aimed to specifically ameliorate impairments in relative accuracy following feedback. Participants in Experiment 2 were exposed to right/wrong feedback, no feedback, and correct answer feedback while making JOLs. Using such a within-subjects design did not improve relative accuracy following correct answer feedback. Experiment 3 showed that previous exposure to a test-feedback-test cycle did not improve relative accuracy. In Experiment 4, feedback was scaffolded such that the correct answer was progressively revealed. Participants corrected more errors if they could generate the correct response with fewer letter cues. However, relative accuracy did not improve in comparison to the previous experiments. Accordingly, the current experiments suggest that participants may understand that feedback is beneficial, but receiving feedback diminishes prediction accuracy for specific items and participants do not appreciate the magnitude of the benefits of feedback.

Young children bet on their numerical skills: metacognition in the numerical domain.

Metacognition, the ability to assess one's own knowledge, has been targeted as a critical learning mechanism in mathematics education. Yet the early childhood origins of metacognition have proven difficult to study. Using a novel nonverbal task and a comprehensive set of metacognitive measures, we provided the strongest evidence to date that young children are metacognitive. We showed that children as young as 5 years made metacognitive "bets" on their numerical discriminations in a wagering task. However, contrary to previous reports from adults, our results showed that children's metacognition is domain specific: Their metacognition in the numerical domain was unrelated to their metacognition in another domain (emotion discrimination). Moreover, children's metacognitive ability in only the numerical domain predicted their school-based mathematics knowledge. The data provide novel evidence that metacognition is a fundamental, domain-dependent cognitive ability in children. The findings have implications for theories of uncertainty and reveal new avenues for training metacognition in children.

Retrospective and prospective metacognitive judgments in rhesus macaques (Macaca mulatta).

A growing body of research suggests that some non-human animals are capable of making accurate metacognitive judgments. In previous studies, non-human animals have made either retrospective or prospective judgments (about how they did on a test or how they will do on a test, respectively). These two types of judgments are dissociable in humans. The current study tested the abilities of two rhesus macaque monkeys to make both retrospective and prospective judgments about their performance on the same memory task. Both monkeys had been trained previously to make retrospective confidence judgments. Both monkeys successfully demonstrated transfer of retrospective metacognitive judgments to the new memory task. Furthermore, both monkeys transferred their retrospective judgments to the prospective task (one, immediately, and one, following the elimination of a response bias). This study is the first to demonstrate both retrospective and prospective monitoring abilities in the same monkeys and on the same task, suggesting a greater level of flexibility in animals' metacognitive monitoring abilities than has been reported previously.

Self-regulated learning: beliefs, techniques, and illusions.

Knowing how to manage one's own learning has become increasingly important in recent years, as both the need and the opportunities for individuals to learn on their own outside of formal classroom settings have grown. During that same period, however, research on learning, memory, and metacognitive processes has provided evidence that people often have a faulty mental model of how they learn and remember, making them prone to both misassessing and mismanaging their own learning. After a discussion of what learners need to understand in order to become effective stewards of their own learning, we first review research on what people believe about how they learn and then review research on how people's ongoing assessments of their own learning are influenced by current performance and the subjective sense of fluency. We conclude with a discussion of societal assumptions and attitudes that can be counterproductive in terms of individuals becoming maximally effective learners.

Expecting to teach enhances learning and organization of knowledge in free recall of text passages.

The present research assessed the potential effects of expecting to teach on learning. In two experiments, participants studied passages either in preparation for a later test or in preparation for teaching the passage to another student who would then be tested. In reality, all participants were tested, and no one actually engaged in teaching. Participants expecting to teach produced more complete and better organized free recall of the passage (Experiment 1) and, in general, correctly answered more questions about the passage than did participants expecting a test (Experiment 1), particularly questions covering main points (Experiment 2), consistent with their having engaged in more effective learning strategies. Instilling an expectation to teach thus seems to be a simple, inexpensive intervention with the potential to increase learning efficiency at home and in the classroom.

Why interleaving enhances inductive learning: the roles of discrimination and retrieval.

Kornell and Bjork (Psychological science 19:585-592, 2008) found that interleaving exemplars of different categories enhanced inductive learning of the concepts based on those exemplars. They hypothesized that the benefit of mixing exemplars from different categories is that doing so highlights differences between the categories. Kang and Pashler (Applied cognitive psychology 26:97-103, 2012) obtained results consistent with this discriminative-contrast hypothesis: Interleaving enhanced inductive learning, but temporal spacing, which does not highlight category differences, did not. We further tested the discriminative-contrast hypothesis by examining the effects of interleaving and spacing, as well as their combined effects. In three experiments, using photographs of butterflies and birds as the stimuli, temporal spacing was harmful when it interrupted the juxtaposition of interleaved categories, even when total spacing was held constant, supporting the discriminative-contrast hypothesis. Temporal spacing also had value, however, when it did not interrupt discrimination processing.

The ease-of-processing heuristic and the stability bias: dissociating memory, memory beliefs, and memory judgments.

Judgments about memory are essential in promoting knowledge; they help identify trustworthy memories and predict what information will be retained in the future. In the three experiments reported here, we investigated the mechanisms underlying predictions about memory. In Experiments 1 and 2, single words were presented once or multiple times, in large or small type. There was a double dissociation between actual memory and predicted memory: Type size affected predicted but not actual memory, and future study opportunities affected actual memory but scarcely affected predicted memory. The results of Experiment 3 suggest that beliefs and judgments are largely independent, and neither consistently resembles actual memory. Participants' underestimation of future learning-a stability bias-stemmed from an overreliance on their current memory state in making predictions about future memory states. The overreliance on type size highlights the fundamental importance of the ease-of-processing heuristic: Information that is easy to process is judged to have been learned well.

How the wisdom of crowds, and of the crowd within, are affected by expertise.

We investigated the effect of expertise on the wisdom of crowds. Participants completed 60 trials of a numerical estimation task, during which they saw 50-100 asterisks and were asked to estimate how many stars they had just seen. Experiment 1 established that both inner- and outer-crowd wisdom extended to our novel task: Single responses alone were less accurate than responses aggregated across a single participant (showing inner-crowd wisdom) and responses aggregated across different participants were even more accurate (showing outer-crowd wisdom). In Experiment 2, prior to beginning the critical trials, participants did 12 practice trials with feedback, which greatly increased their accuracy. There was a benefit of outer-crowd wisdom relative to a single estimate. There was no inner-crowd wisdom effect, however; with high accuracy came highly restricted variance, and aggregating insufficiently varying responses is not beneficial. Our data suggest that experts give almost the same answer every time they are asked and so they should consult the outer crowd rather than solicit multiple estimates from themselves.

Answering a factual question today increases one's confidence in the same answer tomorrow - independent of fluency.

We investigated the cognitive processes that cause confidence to increase. Participants were asked 48 general-knowledge questions either once or three times, without feedback. After 2 min (Experiment 1) or 48 h (Experiment 2) they were asked the same questions again, and rated their confidence. Repeated questioning increased confidence but not accuracy. This increase, which replicated research on episodic memory in the eyewitness literature (e.g., Shaw, Journal of Experimental Psychology: Applied 2: 126-146, 1996), occurred even though accuracy was only around 25%. A mediation analysis identified response repetition, but not fluency, as a mechanism underlying growth in confidence. Thus, the basis for confidence judgments appears to be whether one's current response has been generated previously. In sum, answering a factual question increases confidence, but not accuracy, and this happens because learners use response repetition as a cue for confidence judgments.

Unsuccessful retrieval attempts enhance subsequent learning.

Taking tests enhances learning. But what happens when one cannot answer a test question-does an unsuccessful retrieval attempt impede future learning or enhance it? The authors examined this question using materials that ensured that retrieval attempts would be unsuccessful. In Experiments 1 and 2, participants were asked fictional general-knowledge questions (e.g., "What peace treaty ended the Calumet War?"). In Experiments 3-6, participants were shown a cue word (e.g., whale) and were asked to guess a weak associate (e.g., mammal); the rare trials on which participants guessed the correct response were excluded from the analyses. In the test condition, participants attempted to answer the question before being shown the answer; in the read-only condition, the question and answer were presented together. Unsuccessful retrieval attempts enhanced learning with both types of materials. These results demonstrate that retrieval attempts enhance future learning; they also suggest that taking challenging tests-instead of avoiding errors-may be one key to effective learning.

Delayed versus immediate feedback in children's and adults' vocabulary learning.

We investigated whether the superior memory performance sometimes seen with delayed rather than immediate feedback was attributable to the shorter retention interval (or lag to test) from the last presentation of the correct information in the delayed condition. Whether lag to test was controlled or not, delayed feedback produced better final test performance than did immediate feedback, which in turn produced better performance than did no feedback at all, when we tested Grade 6 children learning school-relevant vocabulary. With college students learning GRE-level words, however, delayed feedback produced better performance than did immediate feedback (and both were better than no feedback) when lag to test was uncontrolled, but there was no difference between the delayed and immediate feedback conditions when the lag to test was controlled.

Learners' choices and beliefs about self-testing.

Students have to make scores of practical decisions when they study. We investigated the effectiveness of, and beliefs underlying, one such practical decision: the decision to test oneself while studying. Using a flashcards-like procedure, participants studied lists of word pairs. On the second of two study trials, participants either saw the entire pair again (pair mode) or saw the cue and attempted to generate the target (test mode). Participants were asked either to rate the effectiveness of each study mode (Experiment 1) or to choose between the two modes (Experiment 2). The results demonstrated a mismatch between metacognitive beliefs and study choices: Participants (incorrectly) judged that the pair mode resulted in the most learning, but chose the test mode most frequently. A post-experimental questionnaire suggested that self-testing was motivated by a desire to diagnose learning rather than a desire to improve learning.

How to activate students' natural desire to test themselves.

Testing oneself (i.e., doing retrieval practice) is an effective way to study. We attempted to make learners choose to test themselves more often. In Experiment 1, participants were asked how they wanted to study and were given four options: retrieval with no hint (e.g., idea: ______), a two-letter hint (e.g., idea: s____r), a four-letter hint (e.g., idea: se__er), or a presentation trial (e.g., idea: seeker). They tested themselves on the majority of trials. In Experiment 2, when the hint options were removed, they chose restudy rather than pure test on the majority of trials. These findings show that people prefer self-testing over restudy as long as they can get the answer right on the test. However, we would not recommend hints if they impaired learning compared to pure testing. Experiment 3 showed that this was not the case; the three retrieval conditions from Experiment 1 led to equivalent amounts of learning, and all three outperformed the pure presentation condition. We used different materials in Experiment 4 and found that the hints made retrieval slightly less beneficial when the hints made it possible to guess the answers without thinking back to the study phase (e.g., whip: pu__sh). In summary, hints catalyzed people's intuitive desire to self-test, without any downside for learning, thus making their self-regulated study more enjoyable and effective.

The spacing effect in children's memory and category induction.

The spacing effect describes the robust phenomenon whereby memory is enhanced when learning events are distributed, instead of being presented in succession. We investigated the effect of spacing on children's memory and category induction. Three-year-old children were presented with two tasks, a memory task and a category induction task. In the memory task, identical instances of an object were presented and then tested in a multiple choice test. In the category induction task, different instances of a category were presented and tested in a multiple choice test. In both tasks, presenting the instances in a spaced sequence resulted in more learning than presenting the instances in a massed sequence, despite the difficulty created by the spaced sequence. The spaced sequence increased the difficulty of the task by allowing children time to forget the previous instance during the spaced interval.

Audiovisual quality impacts assessments of job candidates in video interviews: Evidence for an AV quality bias.

Video job interviews have become a common hiring practice, allowing employers to save money and recruit from a wider applicant pool. But differences in job candidates' internet connections mean that some interviews will have higher audiovisual (AV) quality than others. We hypothesized that interviewers would be impacted by AV quality when they rated job candidates. In two experiments, participants viewed two-minute long simulated Skype interviews that were either unedited (fluent videos) or edited to mimic the effects of a poor internet connection (disfluent videos). Participants in both experiments rated job candidates from fluent videos as more hirable, even after being explicitly told to disregard AV quality (experiment 2). Our findings suggest that video interviews may favor job candidates with better internet connections and that being aware of this bias does not make it go away.

Principles of cognitive science in education: the effects of generation, errors, and feedback.

Principles of cognitive science hold the promise of helping children to study more effectively, yet they do not always make successful transitions from the laboratory to applied settings and have rarely been tested in such settings. For example, self-generation of answers to questions should help children to remember. But what if children cannot generate anything? And what if they make an error? Do these deviations from the laboratory norm of perfect generation hurt, and, if so, do they hurt enough that one should, in practice, spurn generation? Can feedback compensate, or are errors catastrophic? The studies reviewed here address three interlocking questions in an effort to better implement a computer-based study program to help children learn: (1) Does generation help? (2) Do errors hurt if they are corrected? And (3) what is the effect of feedback? The answers to these questions are: Yes, generation helps; no, surprisingly, errors that are corrected do not hurt; and, finally, feedback is beneficial in verbal learning. These answers may help put cognitive scientists in a better position to put their well-established principles in the service of children's learning.

The promise and perils of self-regulated study.

Self-regulated study involves many decisions, some of which people make confidently and easily (if not always optimally) and others of which are involved and difficult. Good study decisions rest on accurate monitoring of ongoing learning, a realistic mental model of how learning happens, and appropriate use of study strategies. We review our research on the decisions people make, for better or worse, when deciding what to study, how long to study, and how to study.