lab.js: A free, open, online study builder.

Web-based data collection is increasingly popular in both experimental and survey-based research because it is flexible, efficient, and location-independent. While dedicated software for laboratory-based experimentation and online surveys is commonplace, researchers looking to implement experiments in the browser have, heretofore, often had to manually construct their studies' content and logic using code. We introduce lab.js, a free, open-source experiment builder that makes it easy to build studies for both online and in-laboratory data collection. Through its visual interface, stimuli can be designed and combined into a study without programming, though studies' appearance and behavior can be fully customized using HTML, CSS, and JavaScript code if required. Presentation and response times are kept and measured with high accuracy and precision heretofore unmatched in browser-based studies. Experiments constructed with lab.js can be run directly on a local computer and published online with ease, with direct deployment to cloud hosting, export to web servers, and integration with popular data collection platforms. Studies can also be shared in an editable format, archived, re-used and adapted, enabling effortless, transparent replications, and thus facilitating open, cumulative science. The software is provided free of charge under an open-source license; further information, code, and extensive documentation are available from https://lab.js.org/ .

Mousetrap: An integrated, open-source mouse-tracking package.

Mouse-tracking - the analysis of mouse movements in computerized experiments - is becoming increasingly popular in the cognitive sciences. Mouse movements are taken as an indicator of commitment to or conflict between choice options during the decision process. Using mouse-tracking, researchers have gained insight into the temporal development of cognitive processes across a growing number of psychological domains. In the current article, we present software that offers easy and convenient means of recording and analyzing mouse movements in computerized laboratory experiments. In particular, we introduce and demonstrate the mousetrap plugin that adds mouse-tracking to OpenSesame, a popular general-purpose graphical experiment builder. By integrating with this existing experimental software, mousetrap allows for the creation of mouse-tracking studies through a graphical interface, without requiring programming skills. Thus, researchers can benefit from the core features of a validated software package and the many extensions available for it (e.g., the integration with auxiliary hardware such as eye-tracking, or the support of interactive experiments). In addition, the recorded data can be imported directly into the statistical programming language R using the mousetrap package, which greatly facilitates analysis. Mousetrap is cross-platform, open-source and available free of charge from https://github.com/pascalkieslich/mousetrap-os .

Psynteract: A flexible, cross-platform, open framework for interactive experiments.

We introduce a novel platform for interactive studies, that is, any form of study in which participants' experiences depend not only on their own responses, but also on those of other participants who complete the same study in parallel, for example a prisoner's dilemma or an ultimatum game. The software thus especially serves the rapidly growing field of strategic interaction research within psychology and behavioral economics. In contrast to all available software packages, our platform does not handle stimulus display and response collection itself. Instead, we provide a mechanism to extend existing experimental software to incorporate interactive functionality. This approach allows us to draw upon the capabilities already available, such as accuracy of temporal measurement, integration with auxiliary hardware such as eye-trackers or (neuro-)physiological apparatus, and recent advances in experimental software, for example capturing response dynamics through mouse-tracking. Through integration with OpenSesame, an open-source graphical experiment builder, studies can be assembled via a drag-and-drop interface requiring little or no further programming skills. In addition, by using the same communication mechanism across software packages, we also enable interoperability between systems. Our source code, which provides support for all major operating systems and several popular experimental packages, can be freely used and distributed under an open source license. The communication protocols underlying its functionality are also well documented and easily adapted to further platforms. Code and documentation are available at https://github.com/psynteract/ .

Privacy Attitudes toward Mouse-Tracking Paradata Collection.

Survey participants' mouse movements provide a rich, unobtrusive source of paradata, offering insight into the response process beyond the observed answers. However, the use of mouse tracking may require participants' explicit consent for their movements to be recorded and analyzed. Thus, the question arises of how its presence affects the willingness of participants to take part in a survey at all-if prospective respondents are reluctant to complete a survey if additional measures are recorded, collecting paradata may do more harm than good. Previous research has found that other paradata collection modes reduce the willingness to participate, and that this decrease may be influenced by the specific motivation provided to participants for collecting the data. However, the effects of mouse movement collection on survey consent and participation have not been addressed so far. In a vignette experiment, we show that reported willingness to participate in a survey decreased when mouse tracking was part of the overall consent. However, a larger proportion of the sample indicated willingness to both take part and provide mouse-tracking data when these decisions were combined, compared to an independent opt-in to paradata collection, separated from the decision to complete the study. This suggests that survey practitioners may face a trade-off between maximizing their overall participation rate and maximizing the number of participants who also provide mouse-tracking data. Explaining motivations for paradata collection did not have a positive effect and, in some cases, even reduced participants' reported willingness to take part in the survey.

Revisiting the Open Sampling format: Improving risky choices through a novel graphical representation.

When making risky choices, people often fall short of the norm of expected value (EV) maximization. Previous research has shown that presenting options in the Open Sampling (OSa) format, a 10-by-10 matrix of randomly arranged outcomes, can improve choices and reduce decision times. First, the current research aims to replicate and extend the findings on the OSa format. To this end, we compare OSa to the common description-based format as well as further graphical representations, and investigate the resulting accordance with EV maximization and decision time. Second, we study whether people lower (vs. higher) in numeracy, the ability to use probabilistic and mathematical concepts, particularly benefit from a graphical representation of options. We conducted five high-powered studies (total N = 1,575) in which participants chose repeatedly between two risky gambles, using different populations and gamble-problem sets. Overall, we could not find a benefit of the OSa format in terms of EV accordance in any of the five studies. However, three studies also tested a novel variant of the OSa format with grouped outcomes and found that it consistently improved EV accordance compared with all other formats. All graphical formats led to faster decisions without harming decision quality. The effects of presentation format were not moderated by numeracy in three of the four studies that assessed numeracy. In conclusion, our research introduces a new presentation format which consistently improves risky choices and can also be used to communicate risks in applied contexts such as medical decision making.

The reversed description-experience gap: Disentangling sources of presentation format effects in risky choice.

Previous literature has suggested that risky choice patterns in general--and probability weighting in particular--are strikingly different in experience-based as compared with description-based formats. In 2 reanalyses and 3 new experiments, we investigate differences between experience-based and description-based decisions using a parametric approach based on cumulative prospect theory (CPT). Once controlling for sampling biases, we consistently find a reversal of the typical description-experience gap, that is, a reduced sensitivity to probabilities and increased overweighting of small probabilities in decisions from experience as compared with decisions from descriptions. This finding supports the hypothesis that regression to the mean effects in probability estimation are a crucial source of differences between both presentation formats. Further analyses identified task specific information asymmetry prevalent in gambles involving certainty as a third source of differences. We present a novel conceptualization of multiple independent sources of bias that contribute to the description-experience gap, namely sampling biases and task specific information asymmetry on the one hand, and regression to the mean effects in probability estimation on the other hand.

Response to Comment on "Estimating the reproducibility of psychological science".

Gilbert et al. conclude that evidence from the Open Science Collaboration's Reproducibility Project: Psychology indicates high reproducibility, given the study methodology. Their very optimistic assessment is limited by statistical misconceptions and by causal inferences from selectively interpreted, correlational data. Using the Reproducibility Project: Psychology data, both optimistic and pessimistic conclusions about reproducibility are possible, and neither are yet warranted.