Large-scale analysis of test-retest reliabilities of self-regulation measures.

The ability to regulate behavior in service of long-term goals is a widely studied psychological construct known as self-regulation. This wide interest is in part due to the putative relations between self-regulation and a range of real-world behaviors. Self-regulation is generally viewed as a trait, and individual differences are quantified using a diverse set of measures, including self-report surveys and behavioral tasks. Accurate characterization of individual differences requires measurement reliability, a property frequently characterized in self-report surveys, but rarely assessed in behavioral tasks. We remedy this gap by (i) providing a comprehensive literature review on an extensive set of self-regulation measures and (ii) empirically evaluating test-retest reliability of this battery in a new sample. We find that dependent variables (DVs) from self-report surveys of self-regulation have high test-retest reliability, while DVs derived from behavioral tasks do not. This holds both in the literature and in our sample, although the test-retest reliability estimates in the literature are highly variable. We confirm that this is due to differences in between-subject variability. We also compare different types of task DVs (e.g., model parameters vs. raw response times) in their suitability as individual difference DVs, finding that certain model parameters are as stable as raw DVs. Our results provide greater psychometric footing for the study of self-regulation and provide guidance for future studies of individual differences in this domain.

Correlation Database of 60 Cross-Disciplinary Surveys and Cognitive Tasks Assessing Self-Regulation.

Self-regulation is studied across various disciplines, including personality, social, cognitive, health, developmental, and clinical psychology; psychiatry; neuroscience; medicine; pharmacology; and economics. Widespread interest in self-regulation has led to confusion regarding both the constructs within the nomological network of self-regulation and the measures used to assess these constructs. To facilitate the integration of cross-disciplinary measures of self-regulation, we estimated product-moment and distance correlations among 60 cross-disciplinary measures of self-regulation (23 self-report surveys, 37 cognitive tasks) and measures of health and substance use based on 522 participants. The correlations showed substantial variability, though the surveys demonstrated greater convergent validity than did the cognitive tasks. Variables derived from the surveys only weakly correlated with variables derived from the cognitive tasks (M = .049, range = .000 to .271 for the absolute value of the product-moment correlation; M = .085, range = .028 to .241 for the distance correlation), thus challenging the notion that these surveys and cognitive tasks measure the same construct. We conclude by outlining several potential uses for this publicly available database of correlations.

Sound credit scores and financial decisions despite cognitive aging.

Age-related deterioration in cognitive ability may compromise the ability of older adults to make major financial decisions. We explore whether knowledge and expertise accumulated from past decisions can offset cognitive decline to maintain decision quality over the life span. Using a unique dataset that combines measures of cognitive ability (fluid intelligence) and of general and domain-specific knowledge (crystallized intelligence), credit report data, and other measures of decision quality, we show that domain-specific knowledge and expertise provide an alternative route for sound financial decisions. That is, cognitive aging does not spell doom for financial decision-making in domains where the decision maker has developed expertise. These results have important implications for public policy and for the design of effective interventions and decision aids.

Cognitive tasks, anatomical MRI, and functional MRI data evaluating the construct of self-regulation.

We describe the following shared data from N=103 healthy adults who completed a broad set cognitive tasks, surveys, and neuroimaging measurements to examine the construct of self-regulation. The neuroimaging acquisition involved task-based fMRI, resting fMRI, and structural MRI. Each subject completed the following ten tasks in the scanner across two 90-minute scanning sessions: attention network test (ANT), cued task switching, Columbia card task, dot pattern expectancy (DPX), delay discounting, simple and motor selective stop signal, Stroop, a towers task, and a set of survey questions. Subjects also completed resting state scans. The dataset is shared openly through the OpenNeuro project, and the dataset is formatted according to the Brain Imaging Data Structure (BIDS) standard.

Caltech Conte Center, a multimodal data resource for exploring social cognition and decision-making.

This data release of 117 healthy community-dwelling adults provides multimodal high-quality neuroimaging and behavioral data for the investigation of brain-behavior relationships. We provide structural MRI, resting-state functional MRI, movie functional MRI, together with questionnaire-based and task-based psychological variables; many of the participants have multiple datasets from retesting over the course of several years. Our dataset is distinguished by utilizing open-source data formats and processing tools (BIDS, FreeSurfer, fMRIPrep, MRIQC), providing data that is thoroughly quality checked, preprocessed to various extents and available in multiple anatomical spaces. A customizable denoising pipeline is provided as open-source code that includes tools for the generation of functional connectivity matrices and initialization of individual difference analyses. Behavioral data include a comprehensive set of psychological assessments on gold-standard instruments encompassing cognitive function, mood and personality, together with exploratory factor analyses. The dataset provides an in-depth, multimodal resource for investigating associations between individual differences, brain structure and function, with a focus on the domains of social cognition and decision-making.

Applying novel technologies and methods to inform the ontology of self-regulation.

Self-regulation is a broad construct representing the general ability to recruit cognitive, motivational and emotional resources to achieve long-term goals. This construct has been implicated in a host of health-risk behaviors, and is a promising target for fostering beneficial behavior change. Despite its clear importance, the behavioral, psychological and neural components of self-regulation remain poorly understood, which contributes to theoretical inconsistencies and hinders maximally effective intervention development. We outline a research program that seeks to define a neuropsychological ontology of self-regulation, articulating the cognitive components that compose self-regulation, their relationships, and their associated measurements. The ontology will be informed by two large-scale approaches to assessing individual differences: first purely behaviorally using data collected via Amazon's Mechanical Turk, then coupled with neuroimaging data collected from a separate population. To validate the ontology and demonstrate its utility, we will then use it to contextualize health risk behaviors in two exemplar behavioral groups: overweight/obese adults who binge eat and smokers. After identifying ontological targets that precipitate maladaptive behavior, we will craft interventions that engage these targets. If successful, this work will provide a structured, holistic account of self-regulation in the form of an explicit ontology, which will better clarify the pattern of deficits related to maladaptive health behavior, and provide direction for more effective behavior change interventions.

Evidence for hippocampal dependence of value-based decisions.

Consistent decisions are intuitively desirable and theoretically important for utility maximization. Neuroeconomics has established the neurobiological substrate of value representation, but brain regions that provide input to this network is less explored. The constructed-preference tradition within behavioral decision research gives a critical role to associative cognitive processes, suggesting a hippocampal role in making consistent decisions. We compared the performance of 31 patients with mediotemporal lobe (MTL) epilepsy and hippocampal lesions, 30 patients with extratemporal lobe epilepsy, and 30 healthy controls on two tasks: binary choices between candy bars based on their preferences and a number-comparison control task where the larger number is chosen. MTL patients made more inconsistent choices than the other two groups for the value-based choice but not the number-comparison task. These inconsistencies correlated with the volume of compromised hippocampal tissue. These results add to increasing evidence on a critical involvement of the MTL in preference construction and value-based choices.

The Experiment Factory: Standardizing Behavioral Experiments.

The administration of behavioral and experimental paradigms for psychology research is hindered by lack of a coordinated effort to develop and deploy standardized paradigms. While several frameworks (Mason and Suri, 2011; McDonnell et al., 2012; de Leeuw, 2015; Lange et al., 2015) have provided infrastructure and methods for individual research groups to develop paradigms, missing is a coordinated effort to develop paradigms linked with a system to easily deploy them. This disorganization leads to redundancy in development, divergent implementations of conceptually identical tasks, disorganized and error-prone code lacking documentation, and difficulty in replication. The ongoing reproducibility crisis in psychology and neuroscience research (Baker, 2015; Open Science Collaboration, 2015) highlights the urgency of this challenge: reproducible research in behavioral psychology is conditional on deployment of equivalent experiments. A large, accessible repository of experiments for researchers to develop collaboratively is most efficiently accomplished through an open source framework. Here we present the Experiment Factory, an open source framework for the development and deployment of web-based experiments. The modular infrastructure includes experiments, virtual machines for local or cloud deployment, and an application to drive these components and provide developers with functions and tools for further extension. We release this infrastructure with a deployment (http://www.expfactory.org) that researchers are currently using to run a set of over 80 standardized web-based experiments on Amazon Mechanical Turk. By providing open source tools for both deployment and development, this novel infrastructure holds promise to bring reproducibility to the administration of experiments, and accelerate scientific progress by providing a shared community resource of psychological paradigms.