A tutorial on open-source large language models for behavioral science.

Large language models (LLMs) have the potential to revolutionize behavioral science by accelerating and improving the research cycle, from conceptualization to data analysis. Unlike closed-source solutions, open-source frameworks for LLMs can enable transparency, reproducibility, and adherence to data protection standards, which gives them a crucial advantage for use in behavioral science. To help researchers harness the promise of LLMs, this tutorial offers a primer on the open-source Hugging Face ecosystem and demonstrates several applications that advance conceptual and empirical work in behavioral science, including feature extraction, fine-tuning of models for prediction, and generation of behavioral responses. Executable code is made available at github.com/Zak-Hussain/LLM4BeSci.git . Finally, the tutorial discusses challenges faced by research with (open-source) LLMs related to interpretability and safety and offers a perspective on future research at the intersection of language modeling and behavioral science.

Estimating the number of clusters via a corrected clustering instability.

We improve instability-based methods for the selection of the number of clusters k in cluster analysis by developing a corrected clustering distance that corrects for the unwanted influence of the distribution of cluster sizes on cluster instability. We show that our corrected instability measure outperforms current instability-based measures across the whole sequence of possible k, overcoming limitations of current insability-based methods for large k. We also compare, for the first time, model-based and model-free approaches to determining cluster-instability and find their performance to be comparable. We make our method available in the R-package cstab.

Long-term treatment of chronic migraine with OnabotulinumtoxinA: efficacy, quality of life and tolerability in a real-life setting.

Botulinum toxin was shown to be effective in treatment of chronic migraine. We wanted to explore its efficacy and tolerability in chronic application under real-life conditions. For this, 27 consecutive patients (age 45.6 ± 10.8 years, 25 females, 2 males) received altogether 176 injection series (IS) with 189.7 ± 45.8MU onabotulinumtoxinA (Botox(®)) according to the PREEMPT scheme. During the study period altogether 6.5 ± 2.9 (min 4, max 13) IS were applied per patient (total treatment time of 73.1 ± 36.9 weeks). 96 % of the patients reported benefit. Monthly headache days were reduced from 18.9 ± 3.9 to 8.7 ± 4.5 (p < 0.001, -53.7 %), migraine days from 16.8 ± 4.9 to 7.4 ± 4.6 (p < 0.001, -55.1 %), autonomic days from 8.6 ± 7.5 to 2.7 ± 4.2 (p < 0.001, -71.9 %) and medication days from 14.2 ± 4.6 to 8.3 ± 4.2 (p < 0.001, -71.1 %). Health-related quality of life improved by 0.6-1.5 standard deviations (SD) (Short Form Health Survey), migraine-related quality of life by 1.4-2.0 SD (Migraine-Specific Quality of Life Questionnaire) and by 1.9 SD (Headache Impact Test), depression by 1.1 SD (Beck Depression Inventory). Subjective global clinical improvement was 2.6 ± 0.6 (Global Clinical Improvement Scale). All improvements were stable throughout the entire study period. Adverse effects were infrequent, mild and transient. Botulinum toxin provides highly effective and safe long-term treatment of chronic migraine.

Novel embeddings improve the prediction of risk perception.

We assess whether the classic psychometric paradigm of risk perception can be improved or supplanted by novel approaches relying on language embeddings. To this end, we introduce the Basel Risk Norms, a large data set covering 1004 distinct sources of risk (e.g., vaccination, nuclear energy, artificial intelligence) and compare the psychometric paradigm against novel text and free-association embeddings in predicting risk perception. We find that an ensemble model combining text and free association rivals the predictive accuracy of the psychometric paradigm, captures additional affect and frequency-related dimensions of risk perception not accounted for by the classic approach, and has greater range of applicability to real-world text data, such as news headlines. Overall, our results establish the ensemble of text and free-association embeddings as a promising new tool for researchers and policymakers to track real-world risk perception. Supplementary Information: The online version contains supplementary material available at 10.1140/epjds/s13688-024-00478-x.

Using novel data and ensemble models to improve automated labeling of Sustainable Development Goals.

A number of labeling systems based on text have been proposed to help monitor work on the United Nations (UN) Sustainable Development Goals (SDGs). Here, we present a systematic comparison of prominent SDG labeling systems using a variety of text sources and show that these differ considerably in their sensitivity (i.e., true-positive rate) and specificity (i.e., true-negative rate), have systematic biases (e.g., are more sensitive to specific SDGs relative to others), and are susceptible to the type and amount of text analyzed. We then show that an ensemble model that pools SDG labeling systems alleviates some of these limitations, exceeding the performance of the individual SDG labeling systems considered. We conclude that researchers and policymakers should care about the choice of the SDG labeling system and that ensemble methods should be favored when drawing conclusions about the absolute and relative prevalence of work on the SDGs based on automated methods.

Convergence of Age Differences in Risk Preference, Impulsivity, and Self-Control: A Multiverse Analysis.

OBJECTIVES: Numerous theories exist regarding age differences in risk preference and related constructs, yet many of them offer conflicting predictions and fail to consider convergence between measurement modalities or constructs. To pave the way for conceptual clarification and theoretical refinement, in this preregistered study we aimed to comprehensively examine age effects on risk preference, impulsivity, and self-control using different measurement modalities, and to assess their convergence. METHODS: We collected a large battery of self-report, informant report, behavioral, hormone, and neuroimaging measures from a cross-sectional sample of 148 (55% female) healthy human participants between 16 and 81 years (mean age = 46 years, standard deviation [SD] = 19). We used an extended sample of 182 participants (54% female, mean age = 46 years, SD = 19) for robustness checks concerning the results from self-reports, informant reports, and behavioral measures. For our main analysis, we performed specification curve analyses to visualize and estimate the convergence between the different modalities and constructs. RESULTS: Our multiverse analysis approach revealed convergent results for risk preference, impulsivity, and self-control from self- and informant reports, suggesting a negative effect of age. For behavioral, hormonal, and neuroimaging outcomes, age effects were mostly absent. DISCUSSION: Our findings call for conceptual clarification and improved operationalization to capture the putative mechanisms underlying age-related differences in risk preference and related constructs.

How large language models can reshape collective intelligence.

Collective intelligence underpins the success of groups, organizations, markets and societies. Through distributed cognition and coordination, collectives can achieve outcomes that exceed the capabilities of individuals-even experts-resulting in improved accuracy and novel capabilities. Often, collective intelligence is supported by information technology, such as online prediction markets that elicit the 'wisdom of crowds', online forums that structure collective deliberation or digital platforms that crowdsource knowledge from the public. Large language models, however, are transforming how information is aggregated, accessed and transmitted online. Here we focus on the unique opportunities and challenges this transformation poses for collective intelligence. We bring together interdisciplinary perspectives from industry and academia to identify potential benefits, risks, policy-relevant considerations and open research questions, culminating in a call for a closer examination of how large language models affect humans' ability to collectively tackle complex problems.

On the semantic representation of risk.

What are the defining features of lay people's semantic representation of risk? We contribute to mapping the semantics of risk based on word associations to provide insight into both universal and individual differences in the representation of risk. Specifically, we introduce a mini-snowball word association paradigm and use the tools of network and sentiment analysis to characterize the semantics of risk. We find that association-based representations not only corroborate but also extend those extracted from past survey- and text-based approaches. Crucially, we find that the semantics of risk show universal properties and individual and group differences. Most notably, while semantic clusters generalize across languages, their frequency varies systematically across demographic groups, with older and female respondents showing more negative connotations and mentioning more often certain types of activities (e.g., recreational activities) relative to younger adults and males, respectively. Our work has general implications for the measurement of risk-related constructs by suggesting that "risk" can mean different things to different individuals.

A Description-Experience Framework of the Psychology of Risk.

The modern world holds countless risks for humanity, both large-scale and intimately personal-from cyberwarfare, pandemics, and climate change to sexually transmitted diseases and drug use and abuse. Many risks have prompted institutional, regulatory, and technological countermeasures, the success of which depends to some extent on how individuals learn about the risks in question. We distinguish between two powerful but imperfect teachers of risk. First, people may learn by consulting symbolic and descriptive material, such as warnings, statistics, and images. More often than not, however, a risk's fluidity defies precise description. Second, people may learn about risks through personal experience. Responses to risk can differ systematically depending on whether people learn through one mode, both, or neither. One reason for these differences-and by no means the only reason-is the discrepancy in the cognitive impact that rare events (typically the risk event) and common events (typically the nonoccurrence of the risk event) have on the decision maker. We propose a description-experience framework that highlights not only the impact of each mode of learning but also the effects of their interplay on individuals' and collectives' responses to risk. We outline numerous research questions and themes suggested by this framework.

Structural differences in the semantic networks of younger and older adults.

Cognitive science invokes semantic networks to explain diverse phenomena, from memory retrieval to creativity. Research in these areas often assumes a single underlying semantic network that is shared across individuals. Yet, recent evidence suggests that content, size, and connectivity of semantic networks are experience-dependent, implying sizable individual and age-related differences. Here, we investigate individual and age differences in the semantic networks of younger and older adults by deriving semantic networks from both fluency and similarity rating tasks. Crucially, we use a megastudy approach to obtain thousands of similarity ratings per individual to allow us to capture the characteristics of individual semantic networks. We find that older adults possess lexical networks with smaller average degree and longer path lengths relative to those of younger adults, with older adults showing less interindividual agreement and thus more unique lexical representations relative to younger adults. Furthermore, this approach shows that individual and age differences are not evenly distributed but, rather, are related to weakly connected, peripheral parts of the networks. All in all, these results reveal the interindividual differences in both the content and the structure of semantic networks that may accumulate across the life span as a function of idiosyncratic experiences.

Using Network Science to Understand the Aging Lexicon: Linking Individuals' Experience, Semantic Networks, and Cognitive Performance.

People undergo many idiosyncratic experiences throughout their lives that may contribute to individual differences in the size and structure of their knowledge representations. Ultimately, these can have important implications for individuals' cognitive performance. We review evidence that suggests a relationship between individual experiences, the size and structure of semantic representations, as well as individual and age differences in cognitive performance. We conclude that the extent to which experience-dependent changes in semantic representations contribute to individual differences in cognitive aging remains unclear. To help fill this gap, we outline an empirical agenda that utilizes network analysis and involves the concurrent assessment of large-scale semantic networks and cognitive performance in younger and older adults. We present preliminary data to establish the feasibility and limitations of such empirical, network-analytical approaches.

Targeting Higher Intraoperative Blood Pressures Does Not Reduce Adverse Cardiovascular Events Following Noncardiac Surgery.

BACKGROUND: Intraoperative arterial hypotension is strongly associated with postoperative major adverse cardiovascular events (MACE); however, whether targeting higher intraoperative mean arterial blood pressures (MAPs) may prevent adverse events remains unclear. OBJECTIVES: This study sought to determine whether targeting higher intraoperative MAP lowers the incidence of postoperative MACE. METHODS: This single-center randomized controlled trial assigned adult patients at cardiovascular risk undergoing major noncardiac surgery to an intraoperative MAP target of ≥60 mm Hg (control) or ≥75 mm Hg (MAP ≥75). The primary outcome was acute myocardial injury on postoperative days 0-3 and/or 30-day MACE/acute kidney injury (AKI) (acute coronary syndrome, congestive heart failure, coronary revascularization, stroke, AKI, and all-cause mortality). The secondary outcome was 1-year MACE. RESULTS: In total, 458 patients were randomized (intention-to-treat population: 451). The cumulative intraoperative duration with MAP <65 mm Hg was significantly shorter in the MAP ≥75 group (median 9 minutes [interquartile range: 3 to 24 minutes] vs 23 minutes [interquartile range: 8-49 minutes]; P < 0.001). The primary outcome incidence was 48% for MAP ≥75 and 52% for control (risk difference -4.2%; 95% CI: -13% to +5%), the primary contributor being AKI (incidence 44%). Acute myocardial injury occurred in 15% (MAP ≥75) and 19% (control) of patients. The secondary outcome incidence was 17% for MAP ≥75 and 15% for control (risk difference +2.7; 95% CI: -4% to +9.5%). CONCLUSIONS: These findings do not support universally targeting higher intraoperative blood pressures to reduce postoperative complications. Despite a 60% reduction in hypotensive time with MAP <65 mm Hg, no significant reductions in acute myocardial injury or 30-day MACE/AKI could be found. (Biomarkers, Blood Pressure, BIS: Risk Stratification/Management of Patients at Cardiac Risk in Major Noncardiac Surgery [BBB]; NCT02533128).

Online sensory feedback during active search improves tactile localization.

Natural motor behavior is usually refined by ongoing sensory input in closed feedback loops. Research has suggested that humans make systematic errors when localizing touch on the skin, and that perceptual body representations underlying these behaviors are distorted. However, experimental procedures usually prevent participants from touching the target limb, interrupting the natural action-perception loop. It is currently unknown how such experimental strategies affect localization and systematic perceptual distortions. Here, participants received a brief touch on their left forearm and, with closed eyes, searched for the target location by moving the right index finger across the left arm. Tactile search significantly reduced the localization error present at touchdown of the searching finger on the target arm. Localization improvement was largely absent when a barrier above the target arm prevented online tactile feedback of the target region. Vision of the arms while reaching to, and searching on, the skin, greatly reduced the localization error at touchdown, but tactile search further improved localization slightly. Thus, both tactile and visual feedback help matching the positions of reaching and target limbs during localization. Yet, even if small, the unique improvement through tactile information confirms the importance of target-related, closed-loop tactile feedback for tactile localization. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Three gaps and what they may mean for risk preference.

Risk preference is one of the most important building blocks of choice theories in the behavioural sciences. In economics, it is often conceptualized as preferences concerning the variance of monetary payoffs, whereas in psychology, risk preference is often thought to capture the propensity to engage in behaviour with the potential for loss or harm. Both concepts are associated with distinct measurement traditions: economics has traditionally relied on behavioural measures, while psychology has often relied on self-reports. We review three important gaps that have emerged from work stemming from these two measurement traditions: first, a description-experience gap which suggests that behavioural measures do not speak with one voice and can give very different views on an individual's appetite for risk; second, a behaviour-self-report gap which suggests that different self-report measures, but not behavioural measures, show a high degree of convergent validity; and, third, a temporal stability gap which suggests that self-reports, but not behavioural measures, show considerable temporal stability across periods of years. Risk preference, when measured through self-reports-but not behavioural tests-appears as a moderately stable psychological trait with both general and domain-specific components. We argue that future work needs to address the gaps that have emerged from the two measurement traditions and test their differential predictive validity for important economic, health and well-being outcomes. This article is part of the theme issue 'Risk taking and impulsive behaviour: fundamental discoveries, theoretical perspectives and clinical implications'.

New Perspectives on the Aging Lexicon.

The field of cognitive aging has seen considerable advances in describing the linguistic and semantic changes that happen during the adult life span to uncover the structure of the mental lexicon (i.e., the mental repository of lexical and conceptual representations). Nevertheless, there is still debate concerning the sources of these changes, including the role of environmental exposure and several cognitive mechanisms associated with learning, representation, and retrieval of information. We review the current status of research in this field and outline a framework that promises to assess the contribution of both ecological and psychological aspects to the aging lexicon.

A meta-analytic review of two modes of learning and the description-experience gap.

People can learn about the probabilistic consequences of their actions in two ways: One is by consulting descriptions of an action's consequences and probabilities (e.g., reading up on a medication's side effects). The other is by personally experiencing the probabilistic consequences of an action (e.g., beta testing software). In principle, people taking each route can reach analogous states of knowledge and consequently make analogous decisions. In the last dozen years, however, research has demonstrated systematic discrepancies between description- and experienced-based choices. This description-experience gap has been attributed to factors including reliance on a small set of experience, the impact of recency, and different weighting of probability information in the two decision types. In this meta-analysis focusing on studies using the sampling paradigm of decisions from experience, we evaluated these and other determinants of the decision-experience gap by reference to more than 70,000 choices made by more than 6,000 participants. We found, first, a robust description-experience gap but also a key moderator, namely, problem structure. Second, the largest determinant of the gap was reliance on small samples and the associated sampling error: free to terminate search, individuals explored too little to experience all possible outcomes. Third, the gap persisted when sampling error was basically eliminated, suggesting other determinants. Fourth, the occurrence of recency was contingent on decision makers' autonomy to terminate search, consistent with the notion of optional stopping. Finally, we found indications of different probability weighting in decisions from experience versus decisions from description when the problem structure involved a risky and a safe option. (PsycINFO Database Record

Modeling valuations from experience: A comment on Ashby and Rakow (2014).

What are the cognitive mechanisms underlying subjective valuations formed on the basis of sequential experiences of an option's possible outcomes? Ashby and Rakow (2014) have proposed a sliding window model (SWIM), according to which people's valuations represent the average of a limited sample of recent experiences (the size of which is estimated by the model) formed after sampling has been terminated (i.e., an end-of-sequence process). Ashby and Rakow presented results from which they concluded, on the basis of model-selection criteria, that the SWIM performs well compared with alternative models (e.g., value-updating model, summary model). Further, they reported that the individual window sizes estimated by the SWIM correlated with a measure of working-memory capacity. In a reanalysis of the Ashby and Rakow data, we find no clear evidence in support of any of the models tested, and a slight advantage for the summary model. Further, we demonstrate that individual differences in the window-size estimated by the SWIM can reflect differences in noise. In computer simulations, we examine the more general question of how well the models tested by Ashby and Rakow can actually be discriminated. The results reveal that the models' ability to fit data depends on a complex interplay of noise and the sample size of outcomes on which a valuation response is based. This can critically influence model performance and conclusions regarding the underlying cognitive mechanisms. We discuss the implications of these findings and suggest ways of improving model comparisons in valuations from experience. (PsycINFO Database Record

How short- and long-run aspirations impact search and choice in decisions from experience.

To what extent do people adapt their information search policies and subsequent decisions to the long- and short-run consequences of choice environments? To address this question, we investigated exploration and exploitation policies in choice environments that involved single or multiple plays. We further compared behavior in these environments with behavior in the standard sampling paradigm. Frequently used in research on decision from experience, this paradigm does not explicitly implement the choice in terms of the short or long run. Results showed that people searched more in the multi-play environment than in the single-play environment. Moreover, the substantial search effort in the multi-play environment was conducive to choices consistent with expected value maximization, whereas the lesser search effort in the single-play environment was compatible with the goal of maximizing the chance of winning something. Furthermore, choice and search behaviors in the sampling paradigm predominantly echoed those observed in the single-play environment. This suggests that, when not instructed otherwise, participants in the sampling paradigm appear to favor search and choice strategies that embody short-run aspirations. Finally, the present findings challenge the revealed preference approach in decisions from experience, while also suggesting that information search may be an important and potentially even better signal of preference or aspirations than choice.