Terracotta: A tool for conducting experimental research on student learning.

For researchers seeking to improve education, a common goal is to identify teaching practices that have causal benefits in classroom settings. To test whether an instructional practice exerts a causal influence on an outcome measure, the most straightforward and compelling method is to conduct an experiment. While experimentation is common in laboratory studies of learning, experimentation is increasingly rare in classroom settings, and to date, researchers have argued it is prohibitively expensive and difficult to conduct experiments on education in situ. To address this challenge, we present Terracotta (Tool for Education Research with RAndomized COnTrolled TriAls), an open-source web application that integrates with a learning management system to provide a comprehensive experimental research platform within an online class site. Terracotta automates randomization, informed consent, experimental manipulation of different versions of learning activities, and export of de-identified research data. Here we describe these features, and the results of a live classroom demonstration study using Terracotta, a preregistered replication of McDaniel et al. (Journal of Applied Research in Memory and Cognition, 1(1), 18-26, 2012). Using Terracotta, we experimentally manipulated online review assignments so that consenting students alternated, on a weekly basis, between taking multiple-choice quizzes (retrieval practice) and reading answers to these quizzes (restudy). Students' performance on subsequent exams was significantly improved for items that had been in retrieval practice review assignments. This successful replication demonstrates that Terracotta can be used to experimentally manipulate consequential aspects of students' experiences in education settings.

Adding the keyword mnemonic to retrieval practice: A potent combination for foreign language vocabulary learning?

The keyword mnemonic and retrieval practice are two cognitive techniques that have each been identified to enhance foreign language vocabulary learning. However, little is known about the use of these techniques in combination. Previous demonstrations of retrieval-practice effects in foreign language vocabulary learning have tended to use several rounds of retrieval practice. In contrast, we focused on a situation in which retrieval practice was limited to twice per item. For this situation, it is unclear whether retrieval practice will be effective relative to restudying. We advance the view that the keyword mnemonic catalyzes the effectiveness of retrieval practice in this learning context. Experiment 1 (48-h delay) partially supported this view, such that there was no testing effect with retrieval practice alone, but the keyword-retrieval combination did not promote better retention than keyword alone. Experiments 2 and 3 (1-week delay) supported the catalytic view by showing that the keyword-retrieval combination was better than keyword alone, but in the absence of keyword encoding there was no retrieval practice effect (replicating Experiment 1). However, with four rounds of retrieval practice, a marginally significant testing effect emerged (Experiment 3). Moreover, the routes through which participants reached each answer were identified by asking retrieval-route questions in Experiments 2 and 3. Keyword-mediated retrieval, which was observed sometimes even in no-keyword instructed conditions, was shown to be more effective than unmediated retrieval. Our findings suggest that incorporating effective encoding techniques prior to retrieval practice could augment the effectiveness of retrieval practice, at least for vocabulary learning.

Hey buddy, why don't we take it outside: An experience sampling study of prospective memory.

Relatively little research has focused on how prospective memory (PM) operates outside of the laboratory, partially due to the methodological problems presented by naturalistic memory research in general and by the unique challenges of PM in particular. Experience sampling methods (ESM) offer a fruitful avenue for this type of research, as recent work from Gardner and Ascoli (Psychology and Aging, 30, 209-219, 2015) has shown. They found that people thought about PM around 15% of the time, and that future thinking was more common than past thinking. In two studies, we replicated our own findings and those reported by Gardner and Ascoli. To summarize, people think about the future more often than the past (30% compared to 13%), and PM occupies our thoughts approximately 13-15% of the time, supporting claims made by some researchers that our episodic memory systems are forward-looking (Klein in Journal of Applied Research in Memory and Cognition, 2, 222-234, 2013). Of those PM thoughts, participants reported that 61% were internally cued, rather than externally triggered. Through the use of multi-level modeling, we additionally found that PM thoughts were more likely when the respondant was alone than with people, and earlier in the day. Finally, we found that participants higher in neuroticism were more likely to report thinking of PM, and that this was driven entirely by the anxiety facet. Most generally, we hope to have demonstrated the value of ESM to help researchers investigate and understand naturalistic PM.

Age-related changes in neural mechanisms of prospective memory.

The capability to remember and execute intentions in the future - termed prospective memory (PM) - may be of special significance for older adults to enable successful completion of important activities of daily living. Despite the importance of this cognitive function, mixed findings have been obtained regarding age-related decline in PM, and, currently, there is limited understanding of potential contributing mechanisms. In the current study, older (N=41) and younger adults (N=47) underwent task-functional MRI during performance of PM conditions that encouraged either spontaneous retrieval (Focal) or sustained attentional monitoring (Non-focal) to detect PM targets. Older adults exhibited a reduction in PM-related sustained activity within the anterior prefrontal cortex (aPFC) and associated dorsal frontoparietal cognitive control network, due to an increase in non-specific sustained activation in (no-PM) control blocks (i.e., an age-related compensatory shift). Transient PM-trial specific activity was observed in both age groups within a ventral parietal memory network that included the precuneus. However, within a left posterior inferior parietal node of this network, transient PM-related activity was selectively reduced in older adults during the non-focal condition. These age differences in sustained and transient brain activity statistically mediated age-related declines in PM performance, and were potentially linked via age-related changes in functional connectivity between the aPFC and precuneus. Together, they support an account consistent with the Dual Mechanisms of Control framework, in which age-related PM declines are due to neural mechanisms that support proactive cognitive control processes, such as sustained attentional monitoring, while leaving reactive control mechanisms relatively spared.

On Learning Natural-Science Categories That Violate the Family-Resemblance Principle.

The general view in psychological science is that natural categories obey a coherent, family-resemblance principle. In this investigation, we documented an example of an important exception to this principle: Results of a multidimensional-scaling study of igneous, metamorphic, and sedimentary rocks (Experiment 1) suggested that the structure of these categories is disorganized and dispersed. This finding motivated us to explore what might be the optimal procedures for teaching dispersed categories, a goal that is likely critical to science education in general. Subjects in Experiment 2 learned to classify pictures of rocks into compact or dispersed high-level categories. One group learned the categories through focused high-level training, whereas a second group was required to simultaneously learn classifications at a subtype level. Although high-level training led to enhanced performance when the categories were compact, subtype training was better when the categories were dispersed. We provide an interpretation of the results in terms of an exemplar-memory model of category learning.

An implementation intention strategy can improve prospective memory in older adults with very mild Alzheimer's disease.

OBJECTIVES: This study tested whether (1) very mild Alzheimer's disease (AD) is associated with impaired prospective memory (PM) for tasks that are supported by either spontaneous retrieval (focal PM) or strategic monitoring (non-focal PM) and (2) implementation intention (II) encoding could improve PM performance in very mild AD. DESIGN: Thirty-eight healthy older adults and 34 with very mild AD were randomly assigned to perform two PM tasks in either the standard or the II encoding condition. METHOD: All participants performed blocks of category decision in which they were asked to respond to a focal PM target (e.g., the word 'orange') and a non-focal PM target (e.g., words that begin with the letter 'o'). Half of the participants encoded PM instructions in the standard manner, while the other half had a stronger encoding by forming IIs. PM accuracy and category decision accuracy and reaction times were measured. RESULTS: Participants with very mild AD showed deficits in both focal and non-focal PM performance compared to the healthy controls, reflecting deficits in both spontaneous retrieval and strategic monitoring. Participants with very mild AD in the II encoding condition showed better focal PM performance relative to those in the standard encoding condition. CONCLUSIONS: Deficits in both focal and non-focal PM are associated with very mild AD and IIs may be a helpful behavioural intervention for the focal PM deficits. PRACTITIONER POINTS: Multiple deficits in PM are observable in very mild AD. Implementation intentions may enhance focal PM in very mild AD. Future research using larger samples is needed to better understand the effect of II on non-focal PM tasks in healthy older adults and those with very mild AD. The use of simple laboratory PM tasks may limit the generality of our findings. Future research is needed to investigate whether IIs improve PM over a range of more realistic tasks.

Metamemory monitoring and control following retrieval practice for text.

Test-taking is assumed to help learners diagnose what they do and do not know, and by so doing improve the effectiveness of their subsequent study. Previous work has examined metamemory monitoring (e.g., predictions of future performance) and control (e.g., restudy decisions) following testing or retrieval practice with relatively simple materials (e.g., word pairs). There is reason to believe, however, that such monitoring and control decisions might be more difficult with text materials, even after retrieval practice, owing perhaps to difficulty in accurately assessing one's performance on the retrieval-practice test. In two experiments, participants read texts about world regions, then engaged in retrieval practice or rereading of the information in those texts, made estimates about future performance, and then received an opportunity to restudy the texts before taking a final recall test, with self-paced restudy enabling an examination of control processes. Memory predictions were more accurate in the retrieval-practice than in the rereading condition, and learners in both conditions allocated restudy time on the basis of their predictions. Additionally, restudy provided a greater benefit following retrieval practice than following rereading. The present study has implications for how students can use retrieval practice with text to foster subsequent learning.

Individual differences in category learning: memorization versus rule abstraction.

Although individual differences in category-learning tasks have been explored, the observed differences have tended to represent different instantiations of general processes (e.g., learners rely upon different cues to develop a rule) and their consequent representations. Additionally, studies have focused largely on participants' categorizations of transfer items to determine the representations that they formed. In the present studies, we used a convergent-measures approach to examine participants' categorizations of transfer items in addition to their self-reported learning orientations and response times on transfer items, and in doing so, we garnered evidence that qualitatively distinct approaches in explicit strategies for category learning (i.e., memorization vs. abstracting an articulable rule) and consequent representations might emerge in a single task. Participants categorized instances that followed a categorization rule (in Study 1, we used a relational rule; in Study 2, an additional task with a single-feature rule). Critically, for both tasks, some transfer items differed from trained instances on only one attribute (but otherwise were perceptually similar), rendering the item a member of the opposing category on the basis of the rule (i.e., termed ambiguous items). Some learners categorized ambiguous items on the basis of perceptual similarity, whereas others categorized them on the basis of an abstracted rule. Self-reported learning orientation (i.e., memorization vs. rule abstraction) predicted categorizations and response times on transfer items. Differences in learning orientations were not associated with performance on other cognitive measures (i.e., working memory capacity and Raven's Advanced Progressive Matrices). This work suggests that individuals may have different predispositions toward memorization versus rule abstraction in a single categorization task.

Relational and item-specific influences on generate-recognize processes in recall.

The generate-recognize model and the relational-item-specific distinction are two approaches to explaining recall. In this study, we consider the two approaches in concert. Following Jacoby and Hollingshead (Journal of Memory and Language 29:433-454, 1990), we implemented a production task and a recognition task following production (1) to evaluate whether generation and recognition components were evident in cued recall and (2) to gauge the effects of relational and item-specific processing on these components. An encoding task designed to augment item-specific processing (anagram-transposition) produced a benefit on the recognition component (Experiments 1-3) but no significant benefit on the generation component (Experiments 1-3), in the context of a significant benefit to cued recall. By contrast, an encoding task designed to augment relational processing (category-sorting) did produce a benefit on the generation component (Experiment 3). These results converge on the idea that in recall, item-specific processing impacts a recognition component, whereas relational processing impacts a generation component.

Evidence for a detrimental relationship between hypertension history, prospective memory, and prefrontal cortex white matter in cognitively normal older adults.

Hypertension affects many older adults and is associated with impaired neural and cognitive functioning. We investigated whether a history of hypertension was associated with impairments to prospective memory, which refers to the ability to remember to perform delayed intentions, such as remembering to take medication. Thirty-two cognitively normal older adult participants with or without a history of hypertension (self-reported) performed two laboratory prospective memory tasks, one that relied more strongly on executive control (nonfocal prospective memory) and one that relied more strongly on spontaneous memory retrieval processes (focal prospective memory). We observed hypertension-related impairments for nonfocal, but not focal, prospective memory. To complement our behavioral approach, we conducted a retrospective analysis of available structural magnetic resonance imaging data. Lower white matter volume estimates in the anterior prefrontal cortex were associated with lower nonfocal prospective memory and with a history of hypertension. A history of hypertension may be associated with worsened executive control and lower prefrontal white matter volume. The translational implication is that individuals who must remember to take antihypertensive medications and to monitor their blood pressure at home may be impaired in the executive control process that helps to support these prospective memory behaviors.

Dissociable neural routes to successful prospective memory.

Identifying the processes by which people remember to execute an intention at an appropriate moment (prospective memory) remains a fundamental theoretical challenge. According to one account, top-down attentional control is required to maintain activation of the intention, initiate intention retrieval, or support monitoring. A diverging account suggests that bottom-up, spontaneous retrieval can be triggered by cues that have been associated with the intention and that sustained attentional processes are not required. We used a specialized experimental design and functional MRI methods to selectively marshal and identify each process. Results revealed a clear dissociation. One prospective-memory task recruited sustained activity in attentional-control areas, such as the anterior prefrontal cortex; the other engaged purely transient activity in parietal and ventral brain regions associated with attentional capture, target detection, and episodic retrieval. These patterns provide critical evidence that there are two neural routes to prospective memory, with each route emerging under different circumstances.

The Dynamic Multiprocess Framework: evidence from prospective memory with contextual variability.

The ability to remember to execute delayed intentions is referred to as prospective memory. Previous theoretical and empirical work has focused on isolating whether a particular prospective memory task is supported either by effortful monitoring processes or by cue-driven spontaneous processes. In the present work, we advance the Dynamic Multiprocess Framework, which contends that both monitoring and spontaneous retrieval may be utilized dynamically to support prospective remembering. To capture the dynamic interplay between monitoring and spontaneous retrieval, we had participants perform many ongoing tasks and told them that their prospective memory cue may occur in any context. Following either a 20-min or a 12-h retention interval, the prospective memory cues were presented infrequently across three separate ongoing tasks. The monitoring patterns (measured as ongoing task cost relative to a between-subjects control condition) were consistent and robust across the three contexts. There was no evidence for monitoring prior to the initial prospective memory cue; however, individuals who successfully spontaneously retrieved the prospective memory intention, thereby realizing that prospective memory cues could be expected within that context, subsequently monitored. These data support the Dynamic Multiprocess Framework, which contends that individuals will engage monitoring when prospective memory cues are expected, disengage monitoring when cues are not expected, and that when monitoring is disengaged, a probabilistic spontaneous retrieval mechanism can support prospective remembering.

Resource depletion does not influence prospective memory in college students.

This paper reports an experiment designed to investigate the potential influence of prior acts of self-control on subsequent prospective memory performance. College undergraduates (n=146) performed either a cognitively depleting initial task (e.g., mostly incongruent Stroop task) or a less resource-consuming version of that task (e.g., all congruent Stroop task). Subsequently, participants completed a prospective memory task that required attentionally demanding monitoring processes. The results demonstrated that prior acts of self-control do not impair the ability to execute a future intention in college-aged adults. We conceptually replicated these results in three additional depletion and prospective memory experiments. This research extends a growing number of studies demonstrating the boundary conditions of the resource depletion effect in cognitive tasks.

Discrepancy-plus-search processes in prospective memory retrieval.

In the present study, we investigated the processes underlying prospective memory (PM) retrieval, focusing specifically on two possible spontaneous processes: discrepancy-plus-search and familiarity. Discrepancy was elicited by orthogonally manipulating the processing difficulties of the PM targets and the nontargets. Participants performed a PM task while solving anagrams with two levels of difficulty (easy or difficult). Assuming that the ease of processing easy anagrams would heighten a sense of familiarity, the familiarity view predicted better PM performance with easy anagrams as the PM targets. In contrast, the discrepancy-plus-search view predicted higher PM performance for the PM targets that were anagrams whose difficulty level mismatched that of the surrounding nontargets, as compared to PM targets whose difficulty matched that of the surrounding nontargets. This prediction was based on the idea that mismatching rather than matching difficulty levels would create discrepancy, thereby signaling significance for the target. Participants were more likely to perform the PM task for PM targets that were discrepant, supporting the discrepancy-plus-search view.

The bizarreness effect: evidence for the critical influence of retrieval processes.

People show better memory for bizarre sentences relative to common sentences, a finding referred to as the bizarrness effect. Interestingly, this effect is typically only obtained using a mixed-list design, in which participants study common and bizarre sentences in the same list. This bizarreness effect in mixed-list designs has been explained as the result of both enhanced encoding processes and efficient retrieval processes. The present experiment was designed to isolate the unique contributions of the retrieval context to the bizarreness effect. Participants studied common sentences in one room under one set of instructions, and bizarre sentences in another room under another set of instructions. At test, participants recalled the common and bizarre sentences either together or separately. The results showed that the bizarreness effect was only obtained when participants recalled the common and bizarre items together; no bizarreness advantage emerged when participants were required to recall the common and bizarre items separately. These results suggest that differential encoding processes are not necessary for explaining the bizarreness effect in memory. Rather, retrieval of the mixed-list context appears to be critical for obtaining the effect.

Individual Differences in Structure Building: Impacts on Comprehension and Learning, Theoretical Underpinnings, and Support for Less Able Structure Builders.

In this article, we highlight an underappreciated individual difference: structure building. Structure building is integral to many everyday activities and involves creating coherent mental representations of conversations, texts, pictorial stories, and other events. People vary in this ability in a way not generally captured by other better known concepts and individual difference measures. Individuals with lower structure-building ability consistently perform worse on a range of comprehension and learning measures than do individuals with higher structure-building ability, both in the laboratory and in the classroom. Problems include a range of comprehension processes, including encoding factual content, inhibiting irrelevant information, and constructing a cohesive situation model of a text or conversation. Despite these problems, recent research is encouraging in that techniques to improve the learning outcomes for low-ability structure builders have been identified. We argue that the accumulated research warrants the recognition of structure building as an important individual difference in cognitive functioning and that additional theoretical work is needed to understand the underpinnings of structure-building deficits.

Effects of feature highlighting and causal explanations on category learning in a natural-science domain.

Teaching natural-science categories is highly challenging because the objects in such categories are composed of numerous complex dimensions that need to be perceived, evaluated, and integrated. Furthermore, the boundaries separating such categories are often fuzzy. A technique that has been proposed and investigated for enhancing the teaching of natural-science categories is feature highlighting, in which diagnostic features for identifying category members are explicitly described and illustrated. Using rock classification in geology as an example target domain, the present study further investigated the potential benefits of feature highlighting and also of providing causal explanations for the highlighted features. The authors found that feature highlighting did not always lead to improved generalization to novel members of the taught categories. However, robust beneficial effects were seen when the categories were relatively confusable ones and the stated diagnostic features were highly valid for distinguishing among the categories. Finally, at least under the present conditions, supplementing the highlighted features with causal explanations of the reasons for their occurrence did not further enhance the participants' rock-classification learning and generalization. Although the teaching of causal explanations is fundamental to science education, clear evidence that causal explanations enhance classification-learning per se in this domain remains to be demonstrated. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Learning Introductory Biology: Students' Concept-Building Approaches Predict Transfer on Biology Exams.

Previous studies have found that students' concept-building approaches, identified a priori with a cognitive psychology laboratory task, are associated with student exam performances in chemistry classes. Abstraction learners (those who extract the principles underlying related examples) performed better than exemplar learners (those who focus on memorizing the training exemplars and responses) on transfer exam questions but not retention questions, after accounting for general ability. We extended these findings to introductory biology courses in which active-learning techniques were used to try to foster deep conceptual learning. Exams were constructed to contain both transfer and retention questions. Abstraction learners demonstrated better performance than exemplar learners on the transfer questions but not on the retention questions. These results were not moderated by indices of crystallized or fluid intelligence. Our central interpretation is that students identified as abstraction learners appear to construct a deep understanding of the concepts (presumably based on abstract underpinnings), thereby enabling them to apply and generalize the concepts to scenarios and instantiations not seen during instruction (transfer questions). By contrast, other students appear to base their representations on memorized instructed examples, leading to good performance on retention questions but not transfer questions.

Dissociative effects of orthographic distinctiveness in pure and mixed lists: an item-order account.

We apply the item-order theory of list composition effects in free recall to the orthographic distinctiveness effect. The item-order account assumes that orthographically distinct items advantage item-specific encoding in both mixed and pure lists, but at the expense of exploiting relational information present in the list. Experiment 1 replicated the typical free recall advantage of orthographically distinct items in mixed lists and the elimination of that advantage in pure lists. Supporting the item-order account, recognition performances indicated that orthographically distinct items received greater item-specific encoding than did orthographically common items in mixed and pure lists (Experiments 1 and 2). Furthermore, order memory (input-output correspondence and sequential contiguity effects) was evident in recall of pure unstructured common lists, but not in recall of unstructured distinct lists (Experiment 1). These combined patterns, although not anticipated by prevailing views, are consistent with an item-order account.

Prospective memory and aging: preserved spontaneous retrieval, but impaired deactivation, in older adults.

Prospective remembering is partially supported by cue-driven spontaneous retrieval processes. We investigated spontaneous retrieval processes in younger and older adults by presenting prospective memory target cues during a lexical decision task following instructions that the prospective memory task was finished. Spontaneous retrieval was inferred from slowed lexical decision responses to target cues (i.e., intention interference). When the intention was finished, younger adults efficiently deactivated their intention, but the older adults continued to retrieve their intentions. Levels of inhibitory functioning were negatively associated with intention interference in the older adult group, but not in the younger adult group. These results indicate that normal aging might not compromise spontaneous retrieval processes but that the ability to deactivate completed intentions is impaired.