Replicability, Robustness, and Reproducibility in Psychological Science.

Replication-an important, uncommon, and misunderstood practice-is gaining appreciation in psychology. Achieving replicability is important for making research progress. If findings are not replicable, then prediction and theory development are stifled. If findings are replicable, then interrogation of their meaning and validity can advance knowledge. Assessing replicability can be productive for generating and testing hypotheses by actively confronting current understandings to identify weaknesses and spur innovation. For psychology, the 2010s might be characterized as a decade of active confrontation. Systematic and multi-site replication projects assessed current understandings and observed surprising failures to replicate many published findings. Replication efforts highlighted sociocultural challenges such as disincentives to conduct replications and a tendency to frame replication as a personal attack rather than a healthy scientific practice, and they raised awareness that replication contributes to self-correction. Nevertheless, innovation in doing and understanding replication and its cousins, reproducibility and robustness, has positioned psychology to improve research practices and accelerate progress.

Acceptability of Personal Sensing Among People With Alcohol Use Disorder: Observational Study.

BACKGROUND: Personal sensing may improve digital therapeutics for mental health care by facilitating early screening, symptom monitoring, risk prediction, and personalized adaptive interventions. However, further development and the use of personal sensing requires a better understanding of its acceptability to people targeted for these applications. OBJECTIVE: We aimed to assess the acceptability of active and passive personal sensing methods in a sample of people with moderate to severe alcohol use disorder using both behavioral and self-report measures. This sample was recruited as part of a larger grant-funded project to develop a machine learning algorithm to predict lapses. METHODS: Participants (N=154; n=77, 50% female; mean age 41, SD 11.9 years; n=134, 87% White and n=150, 97% non-Hispanic) in early recovery (1-8 weeks of abstinence) were recruited to participate in a 3-month longitudinal study. Participants were modestly compensated for engaging with active (eg, ecological momentary assessment [EMA], audio check-in, and sleep quality) and passive (eg, geolocation, cellular communication logs, and SMS text message content) sensing methods that were selected to tap into constructs from the Relapse Prevention model by Marlatt. We assessed 3 behavioral indicators of acceptability: participants' choices about their participation in the study at various stages in the procedure, their choice to opt in to provide data for each sensing method, and their adherence to a subset of the active methods (EMA and audio check-in). We also assessed 3 self-report measures of acceptability (interference, dislike, and willingness to use for 1 year) for each method. RESULTS: Of the 192 eligible individuals screened, 191 consented to personal sensing. Most of these individuals (169/191, 88.5%) also returned 1 week later to formally enroll, and 154 participated through the first month follow-up visit. All participants in our analysis sample opted in to provide data for EMA, sleep quality, geolocation, and cellular communication logs. Out of 154 participants, 1 (0.6%) did not provide SMS text message content and 3 (1.9%) did not provide any audio check-ins. The average adherence rate for the 4 times daily EMA was .80. The adherence rate for the daily audio check-in was .54. Aggregate participant ratings indicated that all personal sensing methods were significantly more acceptable (all P<.001) compared with neutral across subjective measures of interference, dislike, and willingness to use for 1 year. Participants did not significantly differ in their dislike of active methods compared with passive methods (P=.23). However, participants reported a higher willingness to use passive (vs active) methods for 1 year (P=.04). CONCLUSIONS: These results suggest that active and passive sensing methods are acceptable for people with alcohol use disorder over a longer period than has previously been assessed. Important individual differences were observed across people and methods, indicating opportunities for future improvement.

The Psychological Science Accelerator's COVID-19 rapid-response dataset.

In response to the COVID-19 pandemic, the Psychological Science Accelerator coordinated three large-scale psychological studies to examine the effects of loss-gain framing, cognitive reappraisals, and autonomy framing manipulations on behavioral intentions and affective measures. The data collected (April to October 2020) included specific measures for each experimental study, a general questionnaire examining health prevention behaviors and COVID-19 experience, geographical and cultural context characterization, and demographic information for each participant. Each participant started the study with the same general questions and then was randomized to complete either one longer experiment or two shorter experiments. Data were provided by 73,223 participants with varying completion rates. Participants completed the survey from 111 geopolitical regions in 44 unique languages/dialects. The anonymized dataset described here is provided in both raw and processed formats to facilitate re-use and further analyses. The dataset offers secondary analytic opportunities to explore coping, framing, and self-determination across a diverse, global sample obtained at the onset of the COVID-19 pandemic, which can be merged with other time-sampled or geographic data.

Resisting, Recognizing, and Returning: A Three-Component Model and Review of Persistence in Episodic Goals.

According to prior work, persistent goal pursuit is a continuous process where persisting is a matter of resisting the urge to give up. In everyday goals, however, persistence is often episodic, and its causes are more complex. People pause and resume pursuit many times. Whether people persist reflects more than will power and motivation, it also reflects the other goals they pursue, their resources, and the attentional demands of daily life. People can fail to persist not just because they gave up, but also because they failed to act. We propose a general model of persistence that accommodates the complexity of episodic goals. We argue that persistent goal pursuit is a function of three processes: resisting the urge to give up, recognizing opportunities for pursuit, and returning to pursuit. The broad factors that help and hurt persistence can be organized within these components. These components can also explain the mechanisms of four effective strategies for persistence: removing distractions, using reminders, using implementation intentions, and forming habits. The recognizing-resisting-returning model integrates and improves on extant theories of persistence and goal pursuit and is consistent with empirical work from laboratory and naturalistic settings.

Prospective Prediction of Lapses in Opioid Use Disorder: Protocol for a Personal Sensing Study.

BACKGROUND: Successful long-term recovery from opioid use disorder (OUD) requires continuous lapse risk monitoring and appropriate use and adaptation of recovery-supportive behaviors as lapse risk changes. Available treatments often fail to support long-term recovery by failing to account for the dynamic nature of long-term recovery. OBJECTIVE: The aim of this protocol paper is to describe research that aims to develop a highly contextualized lapse risk prediction model that forecasts the ongoing probability of lapse. METHODS: The participants will include 480 US adults in their first year of recovery from OUD. Participants will report lapses and provide data relevant to lapse risk for a year with a digital therapeutic smartphone app through both self-report and passive personal sensing methods (eg, cellular communications and geolocation). The lapse risk prediction model will be developed using contemporary rigorous machine learning methods that optimize prediction in new data. RESULTS: The National Institute of Drug Abuse funded this project (R01DA047315) on July 18, 2019 with a funding period from August 1, 2019 to June 30, 2024. The University of Wisconsin-Madison Health Sciences Institutional Review Board approved this project on July 9, 2019. Pilot enrollment began on April 16, 2021. Full enrollment began in September 2021. CONCLUSIONS: The model that will be developed in this project could support long-term recovery from OUD-for example, by enabling just-in-time interventions within digital therapeutics. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/29563.

The Psychological Science Accelerator: Advancing Psychology through a Distributed Collaborative Network.

Concerns have been growing about the veracity of psychological research. Many findings in psychological science are based on studies with insufficient statistical power and nonrepresentative samples, or may otherwise be limited to specific, ungeneralizable settings or populations. Crowdsourced research, a type of large-scale collaboration in which one or more research projects are conducted across multiple lab sites, offers a pragmatic solution to these and other current methodological challenges. The Psychological Science Accelerator (PSA) is a distributed network of laboratories designed to enable and support crowdsourced research projects. These projects can focus on novel research questions, or attempt to replicate prior research, in large, diverse samples. The PSA's mission is to accelerate the accumulation of reliable and generalizable evidence in psychological science. Here, we describe the background, structure, principles, procedures, benefits, and challenges of the PSA. In contrast to other crowdsourced research networks, the PSA is ongoing (as opposed to time-limited), efficient (in terms of re-using structures and principles for different projects), decentralized, diverse (in terms of participants and researchers), and inclusive (of proposals, contributions, and other relevant input from anyone inside or outside of the network). The PSA and other approaches to crowdsourced psychological science will advance our understanding of mental processes and behaviors by enabling rigorous research and systematically examining its generalizability.

Reporting standards for literature searches and report inclusion criteria: making research syntheses more transparent and easy to replicate.

A complete description of the literature search, including the criteria used for the inclusion of reports after they have been located, used in a research synthesis or meta-analysis is critical if subsequent researchers are to accurately evaluate and reproduce a synthesis' methods and results. Based on previous guidelines and new suggestions, we present a set of focused and detailed standards for reporting the methods used in a literature search. The guidelines cover five search strategies: reference database searches, journal and bibliography searches, searches of the reference lists of reports, citation searches, and direct contact searches. First, we bring together all the unique recommendations made in existing guidelines for research synthesis. Second, we identify gaps in reporting standards for search strategies. Third, we address these gaps by providing new reporting recommendations. Our hope is to facilitate successful evaluation and replication of research synthesis results.