Eleven strategies for making reproducible research and open science training the norm at research institutions.

Reproducible research and open science practices have the potential to accelerate scientific progress by allowing others to reuse research outputs, and by promoting rigorous research that is more likely to yield trustworthy results. However, these practices are uncommon in many fields, so there is a clear need for training that helps and encourages researchers to integrate reproducible research and open science practices into their daily work. Here, we outline eleven strategies for making training in these practices the norm at research institutions. The strategies, which emerged from a virtual brainstorming event organized in collaboration with the German Reproducibility Network, are concentrated in three areas: (i) adapting research assessment criteria and program requirements; (ii) training; (iii) building communities. We provide a brief overview of each strategy, offer tips for implementation, and provide links to resources. We also highlight the importance of allocating resources and monitoring impact. Our goal is to encourage researchers - in their roles as scientists, supervisors, mentors, instructors, and members of curriculum, hiring or evaluation committees - to think creatively about the many ways they can promote reproducible research and open science practices in their institutions.

Anticipatory Saccades Towards the Future Consequences of One's Actions - an Online Eye Tracking Study.

When an action contingently yields a predictable effect, we form bi-directional action-effect associations that allow us to anticipate both the location and timing of our actions' effects. This is evident in anticipatory eye movements towards the future effect's location which are performed earlier when the effect's delay is short rather than long. Such anticipatory eye movements reflect a proactive process of effect monitoring which prepares a comparison of expected and actual effects. Here, in two online eye tracking experiments, we manipulated effect locations (spatially compatible vs. incompatible in one half) and effect delays (short vs. long) to determine whether in-laboratory effects could be reliably replicated online using participants' individual webcams. Extending prior research, we further compared irrelevant (Experiment 1) to relevant effects (response to effect feature; Experiment 2). In contrast to prior in-laboratory studies, participants anticipatorily looked towards future effects above chance only when effects were relevant. Post-experiment questions suggested that online-participants intentionally ignore irrelevant information to optimize task performance. Nevertheless, replicating in-laboratory experiments, both for relevant and irrelevant effects, participants' first saccade towards a future effect occurred earlier for the short rather than the long effect delay. Thus, we demonstrate that anticipatory eye movements reflecting a time-sensitive proactive effect monitoring process can reliably be assessed both in-laboratory as well as online. However, when investigating anticipatory saccade frequencies online, additional aspects like effect relevance have to be considered.

Similar proactive effect monitoring in free and forced choice action modes.

When our actions yield predictable consequences in the environment, our eyes often already saccade towards the locations we expect these consequences to appear at. Such spontaneous anticipatory saccades occur based on bi-directional associations between action and effect formed by prior experience. That is, our eye movements are guided by expectations derived from prior learning history. Anticipatory saccades presumably reflect a proactive effect monitoring process that prepares a later comparison of expected and actual effect. Here, we examined whether anticipatory saccades emerged under forced choice conditions when only actions but not target stimuli were predictive of future effects and whether action mode (forced choice vs. free choice, i.e., stimulus-based vs. stimulus-independent choice) affected proactive effect monitoring. Participants produced predictable visual effects on the left/right side via forced choice and free choice left/right key presses. Action and visual effect were spatially compatible in one half of the experiment and spatially incompatible in the other half. Irrespective of whether effects were predicted by target stimuli in addition to participants' actions, in both action modes, we observed anticipatory saccades towards the location of future effects. Importantly, neither the frequency, nor latency or amplitude of these anticipatory saccades significantly differed between forced choice and free choice action modes. Overall, our findings suggest that proactive effect monitoring of future action consequences, as reflected in anticipatory saccades, is comparable between forced choice and free choice action modes.

Assessing the Durability of One-Shot Stimulus-Control Bindings.

It has been proposed that cognitive control processes may be implemented in a contextually appropriate manner through the encoding, and cued retrieval, of associations between stimuli and the control processes that were active during their encoding, forming "stimulus-control bindings" as part of episodic event files. Prior work has found strong evidence for such a mechanism by observing behavioral effects of stimulus-control bindings based on a single pairing (one-shot learning). Here, we addressed the important question of how durable these one-shot stimulus-control bindings are. Over three experiments, we investigated the durability of one-shot stimulus-control bindings in relation to both the passage of time and the number of intervening events between the encoding (prime) and retrieval (probe) of the stimulus-control bindings. We found that stimulus-control bindings are quite robust to temporal decay, lasting at least up to 5 minutes in the absence of similar intervening events. By contrast, binding effects were more short-lived in the face of interference from the encoding of similar events between the prime and probe, with a maximum duration of ~2 minutes. Together, these results shed new light on the characteristics of the binding mechanisms underlying the integration of internal control processes in episodic event files and highlight that interference, rather than temporal decay, may be the main limiting factor on long-term effects of item-specific one-shot control learning.

Evidence for a Selective Influence of Short-Term Experiences on the Retrieval of Item-Specific Long-Term Bindings.

Human behavior is guided by prior experience such as bindings between stimuli and responses. Experimentally, this is evident in performance changes when features of the stimulus-response episode reoccur either in the short-term or in the long-term. So far, effects of short-term and long-term bindings are assumed to be independent from one another. In a large-scale re-analysis of eight item-specific stimulus-response priming experiments that orthogonally varied task-specific classifications and actions in the short-term (trial N-1 to trial N) and, item-specifically, in the long-term (lag of several trials), we tested this independence assumption. In detail, we tested whether short-term experiences (repetitions of classification and action features in two consecutive trials) affected the retrieval of item-specific long-term stimulus-classification (S-C) and stimulus-action (S-A) bindings as well as potential long-term C-A bindings. The retrieval of item-specific long-term S-C bindings (i.e., the size of item-specific S-C priming effects) was affected by the persisting activation of classifications from trial N-1 (short-term priming). There were no further interactions between short-term experiences and long-term bindings. These results suggest a feature-specific, selective influence of short-term priming on long-term binding retrieval (e.g., based on shared feature representations). In contrast, however, we found evidence against an influence of short-term C-A bindings on long-term binding retrieval. This finding suggests that the processes contributing to short-term priming and long-term binding retrieval are dissociable from short-term binding and retrieval processes. Our results thus inform current theories on how short-term and long-term bindings are bound and retrieved (e.g., the BRAC framework).

Instructing item-specific switch probability: expectations modulate stimulus-action priming.

Both active response execution and passive listening to verbal codes (a form of instruction) in single prime trials lead to item-specific repetition priming effects when stimuli re-occur in single probe trials. This holds for task-specific classification (stimulus-classification, SC priming, e.g., apple-small) and action (stimulus-action, SA priming, e.g., apple-right key press). To address the influence of expectation on item-specific SC and SA associations, we tested if item-specific SC and SA priming effects were modulated by the instructed probability of re-encountering individual SC or SA mappings (25% vs. 75% instructed switch probability). Importantly, the experienced item-specific switch probability was always 50%. In Experiment 1 (N = 78), item-specific SA/SC switch  expectations affected SA, but not SC priming effects exclusively following active response execution. Experiment 2 (N = 40) was designed to emphasize SA priming by only including item-specific SC repetitions. This yielded stronger SA priming for 25% vs. 75% expected switch probability, both following response execution as in Experiment 1 and also following verbally coded SA associations. Together, these results suggest that SA priming effects, that is, the encoding and retrieval of SA associations, is modulated by item-specific switch expectation. Importantly, this expectation effect cannot be explained by item-specific associative learning mechanisms, as stimuli were primed and probed only once and participants experienced item-specific repetitions/switches equally often across stimuli independent of instructed switch probabilities. This corroborates and extends previous results by showing that SA priming effects are modulated by  expectation not only based on experienced item-specific switch probabilities, but also on mere instruction.

On the interdependence of post-error control and memory.

Post-error cognitive control processes are evident in post-error slowing (PES) and post-error increased accuracy (PIA). A recent theory (Wessel, 2018) proposes that post-error control disrupts not only ongoing motor activity but also current task-set representations, suggesting an interdependence of post-error control and memory. In 2 experiments, we directly tested this interdependence using response repeat/switch targets. As participants' memory of the previously-given response determined their next response, we predicted that participants would not show post-error performance improvements, or may even exhibit decrements (memory obstruction hypothesis). In line with a weak version of the memory obstruction hypothesis, participants' performance did not improve post-error regardless of preparation time across several measures (intertrial interval 500 ms-3000 ms). This could not alternatively be explained by a lack in post-error processing or error awareness. Thus, our results suggest an interdependence of memory and cognitive control in post-error processing, and we offer a novel way of directly assessing this interdependence using response repeat/switch targets. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Learning to expect and monitor the future: How fast do anticipatory saccades toward future action consequences emerge?

When an action contingently yields the same effect, we form bidirectional action-effect associations that allow us to anticipate the effects of our actions. Importantly, our eyes already move toward the expected future location of our actions' effects in anticipation of them, that is, we perform anticipatory saccades. These anticipatory saccades are linked to a proactive effect-monitoring process that prepares the comparison of expected and actual effect. However, how fast such anticipatory saccades emerge (i.e., how fast learning leads to monitoring) is unknown. To address this question, correct left/right responses were followed by a visual effect either on the same side (response-effect compatible) or on the opposite side (response-effect incompatible). In Experiment 1, action-effect compatibility switched after sequences of four, eight, or 12 trials (randomly allocated; partly predictable environment). In Experiment 2, random trials (two to seven) separated sequences of three, five, or seven experimental trials. Again, action-effect compatibility switched after a sequence of trials and sequences were randomly allocated (unpredictable environment). In both experiments, participants started to perform anticipatory saccades toward future effects after experiencing a new action-effect mapping once/twice (response-effect compatible/incompatible). That is, a single to two action-effect (re-)learning instances were sufficient to develop action-effect associations that trigger attentional shifts toward the expected future consequences of our actions (i.e., monitoring processes), whereas influences on action selection are only observed after a substantially larger number of (re-)learning instances. These results suggest that monitoring processes modulate the expression of action-effect associations in action planning based on observed action-effect contingencies. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Item-Specificity and Intention in Episodic Memory.

Schmidt et al.'s (2020) PEP model accurately reflects the complexity of task switching based on bottom-up assumptions and episodic memory, re-evaluating the contribution of commonly presumed top-down processes. Extending it to long-term bindings and their item-specific effects could eludicate puzzling findings regarding the independence of long-term bindings between stimuli, responses, and task-specific categorizations as well as the relation between short-term and long-term bindings. Moreover, ideomotor theories of action control provide a bottom-up basis of incorporating volition and intentional action into the PEP model which is currently restricted to stimulus-based action.

Only time will tell the future: Anticipatory saccades reveal the temporal dynamics of time-based location and task expectancy.

Humans form associations between time intervals and subsequent events and thus develop time-based expectancies that enable time-based action preparation. For instance, when each of two foreperiods (short vs. long) is frequently paired with one specific task (e.g., number magnitude judgment vs. number parity judgment) and infrequently with the alternative task, participants are faster to respond to frequent rather than infrequent foreperiod-task combinations. Here, we investigated the time course of time-based task expectancy by measuring eye movements toward a left and right target location. Foreperiods predicted target locations with 100% validity and tasks with 90% validity. In 2 experiments, without having any explicit knowledge about the foreperiod-location or foreperiod-task contingencies, participants first moved their eyes to the location associated with the short foreperiod and then looked toward the location of the long foreperiod (if no stimulus had been presented after the short foreperiod had passed). That is, they proactively moved their eyes to optimize perception in line with the specific time and location they expected an event to occur at. Importantly, these eye movements reflected not only time-based location expectations, but also time-based task expectations. We discuss new insights in time-based expectancy and its temporal dynamics obtained from anticipatory eye movements and highlight spontaneous eye movements as a window into cognitive processes that cannot be assessed via behavioral response measures. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Memories of control: One-shot episodic learning of item-specific stimulus-control associations.

The repeated pairing of a particular stimulus with a specific cognitive control process, such as task switching, can bind the two together in memory, resulting in the formation of stimulus-control associations. These bindings are thought to guide the context-sensitive application of cognitive control, but it is not presently known whether such stimulus-control associations are only acquired through slow, incremental learning or could also be mediated by episodic memories of a single experience, so-called one-shot learning. Here, we tested this episodic control-binding hypothesis by probing whether a single co-occurrence of a stimulus and the control process of task switching would lead to significant performance benefits (reduced task switch cost) when that stimulus later re-occurred under the same as opposed to different control demands. Across three experiments, we demonstrate that item-specific stimulus-control associations can be formed based on a single exposure, providing the first strong evidence for episodic memory guidance of cognitive control.

Execution-based and verbal code-based stimulus-response associations: proportion manipulations reveal conflict adaptation processes in item-specific priming.

Stimulus-response (S-R) associations consist of two independent components: Stimulus-classification (S-C) and stimulus-action (S-A) associations. Here, we examined whether these S-C and S-A associations were modulated by cognitive control operations. In two item-specific priming experiments, we systematically manipulated the proportion of trials in which item-specific S-C and/or S-A mappings repeated or switched between the single encoding (prime) and single retrieval (probe) instance of each stimulus (i.e., each stimulus appeared only twice). Thus, we assessed the influence of a list-level proportion switch manipulation on the strength of item-specific S-C and S-A associations. Participants responded slower and committed more errors when item-specific S-C or S-A mappings switched rather than repeated between prime and probe (i.e., S-C/S-A switch effects). S-C switch effects were larger when S-C repetitions rather than switches were frequent on the list-level. Similarly, S-A switch effects were modulated by S-A switch proportion. Most importantly, our findings rule out contingency learning and temporal learning as explanations of the observed results and point towards a conflict adaptation mechanism that selectively adapts the encoding and/or retrieval for each S-R component. Finally, we outline how cognitive control over S-R associations operates in the context of item-specific priming.

Binding lies: Flexible retrieval of honest and dishonest behavior.

Telling a consistent lie across multiple occasions poses severe demands on memory. Two cognitive mechanisms aid with overcoming this difficulty: associations between a question and its corresponding response and associations between a question and its previous intentional context (in this case: honest vs. dishonest responding). Here, we assessed whether intentional contexts such as an honest versus dishonest mindset modulate the retrieval of stimulus-response associations. In an item-specific priming paradigm, participants classified stimuli either honestly or dishonestly during a prime and a later probe. The results of three experiments yielded automatic retrieval of the previously primed motor responses (for both honest and dishonest responses) only when the intentional context repeated but not when it switched. These findings indicate interdependent associations between a stimulus, its intentional context, and the corresponding response, allowing for flexible, context-specific retrieval. Thus, humans benefit from prior learning history without incurring costs when the intentional context changes. This finding implies top-down control over the retrieval of stimulus-response associations and provides new insights into the mechanisms of associative learning. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Defining stimulus representation in stimulus-response associations formed on the basis of task execution and verbal codes.

Responding to stimuli leads to the formation of stimulus-response (S-R) associations that allow stimuli to subsequently automatically trigger associated responses. A recent study has shown that S-R associations are established not only by active task execution, but also by the simultaneous presentation of stimuli and verbal codes denoting responses in the absence of own action [Pfeuffer et al. (Journal of Experimental Psychology: Human Perception and Performance 43:328-347, 2017)]. Here, we used an item-specific priming paradigm to investigate whether the stimulus part of S-R associations formed based on task execution and verbal codes is represented in abstract or specific format by examining whether S-R associations are retrieved for perceptually different forms of the same stimulus or not. Between the prime and probe instance of a stimulus, its format switched from image to word or vice versa. We found that, irrespective of whether stimuli were primed by task execution or verbal coding, performance was impaired when S-R mappings switched rather than repeated between the prime and probe instance of a stimulus. The finding that prime S-R mappings affected probe performance even when stimulus format switched indicates that stimuli were represented in abstract form in S-R association based on both task execution and verbal coding. Furthermore, we found no performance benefits for stimuli primed and probed in the same format rather than different formats, suggesting that stimuli were not additionally represented in specific format. Overall, our findings demonstrate the adaptability of automatized behaviors and indicate that abstract stimulus representations allow S-R associations to generalize across perceptually different stimulus formats.

Multiple priming instances increase the impact of practice-based but not verbal code-based stimulus-response associations.

Stimulus-response (S-R) associations, the basis of learning and behavioral automaticity, are formed by the (repeated) co-occurrence of stimuli and responses and render stimuli able to automatically trigger associated responses. The strength and behavioral impact of these S-R associations increases with the number of priming instances (i.e., practice). Here we investigated whether multiple priming instances of a special form of instruction, verbal coding, also lead to the formation of stronger S-R associations in comparison to a single instance of priming. Participants either actively classified stimuli or passively attended to verbal codes denoting responses once or four times before S-R associations were probed. We found that whereas S-R associations formed on the basis of active task execution (i.e., practice) were strengthened by multiple priming instances, S-R associations formed on the basis of verbal codes (i.e., instruction) did not benefit from additional priming instances. These findings indicate difference in the mechanisms underlying the encoding and/or retrieval of previously executed and verbally coded S-R associations.

The power of words: On item-specific stimulus-response associations formed in the absence of action.

Research on stimulus-response (S-R) associations as the basis of behavioral automaticity has a long history. Traditionally, it was assumed that S-R associations are formed as a consequence of the (repeated) co-occurrence of stimulus and response, that is, when participants act upon stimuli. Here, we demonstrate that S-R associations can also be established in the absence of action. In an item-specific priming paradigm, participants either classified everyday objects by performing a left or right key press (task-set execution) or they were verbally presented with information regarding an object's class and associated action while they passively viewed the object (verbal coding). Both S-R associations created by task-set execution and by verbal coding led to the later retrieval of both the stimulus-action component and the stimulus-classification component of S-R associations. Furthermore, our data indicate that both associations created by execution and by verbal coding are temporally stable and rather resilient against overwriting. The automaticity of S-R associations formed in the absence of action reveals the striking adaptability of human action control. (PsycINFO Database Record

A look into the future: Spontaneous anticipatory saccades reflect processes of anticipatory action control.

According to ideomotor theory, human action control uses anticipations of one's own actions' future consequences, that is, action effect anticipations, as a means of triggering actions that will produce desired outcomes (e.g., Hommel, Müsseler, Aschersleben, & Prinz, 2001). Using the response-effect compatibility paradigm (Kunde, 2001), we demonstrate that the anticipation of one's own manual actions' future consequences not only triggers appropriate (i.e., instructed) actions, but simultaneously induces spontaneous (uninstructed) anticipatory saccades to the location of future action consequences. In contrast to behavioral response-effect compatibility effects that have been linked to processes of action selection and action planning, our results suggest that these anticipatory saccades serve the function of outcome evaluation, that is, the comparison of expected/intended and observed action outcomes. Overall, our results demonstrate the informational value of additionally analyzing uninstructed behavioral components complementary to instructed responses and allow us to specify essential mechanisms of the complex interplay between the manual and oculomotor control system in goal-directed action control. (PsycINFO Database Record

Perceiving by proxy: effect-based action control with unperceivable effects.

Anticipations of future sensory events have the potential of priming motor actions that would typically cause these events. Such effect anticipations are generally assumed to rely on previous physical experiences of the contingency of own actions and their ensuing effects. Here we propose that merely imagined action effects may influence behaviour similarly as physically experienced action effects do. Three experiments in the response-effect compatibility paradigm show that the mere knowledge of action-effect contingencies is indeed sufficient to incorporate these effects into action control even if the effects are never experienced as causally linked to own actions. The experiments further highlight constraints for this mechanism which seems to be rather effortful and to depend on explicit intentions.