Diminished diversity-of-thought in a standard large language model.

We test whether large language models (LLMs) can be used to simulate human participants in social-science studies. To do this, we ran replications of 14 studies from the Many Labs 2 replication project with OpenAI's text-davinci-003 model, colloquially known as GPT-3.5. Based on our pre-registered analyses, we find that among the eight studies we could analyse, our GPT sample replicated 37.5% of the original results and 37.5% of the Many Labs 2 results. However, we were unable to analyse the remaining six studies due to an unexpected phenomenon we call the "correct answer" effect. Different runs of GPT-3.5 answered nuanced questions probing political orientation, economic preference, judgement, and moral philosophy with zero or near-zero variation in responses: with the supposedly "correct answer." In one exploratory follow-up study, we found that a "correct answer" was robust to changing the demographic details that precede the prompt. In another, we found that most but not all "correct answers" were robust to changing the order of answer choices. One of our most striking findings occurred in our replication of the Moral Foundations Theory survey results, where we found GPT-3.5 identifying as a political conservative in 99.6% of the cases, and as a liberal in 99.3% of the cases in the reverse-order condition. However, both self-reported 'GPT conservatives' and 'GPT liberals' showed right-leaning moral foundations. Our results cast doubts on the validity of using LLMs as a general replacement for human participants in the social sciences. Our results also raise concerns that a hypothetical AI-led future may be subject to a diminished diversity of thought.

Order effects in PTSD network analysis: important implications for diagnostic conceptualization, treatment refinement, and research.

BACKGROUND: For decades confirmatory factor analysis (CFA) has been the preeminent method to study the underlying structure of posttraumatic stress disorder (PTSD); however, methodological limitations of CFA have led to the emergence of other analytic approaches. In particular, network analysis has become a gold standard to investigate the structure and relationships between PTSD symptoms. A key methodological limitation, however, which has significant clinical implications, is the lack of data on the potential impact of item order effects on the conclusions reached through network analyses. METHODS: The current study, involving a large sample (N = 5055) of active duty army soldiers following deployment to Iraq, assessed the vulnerability of network analyses and prevalence rate to item order effects. This was done by comparing symptom networks of the DSM-IV PTSD checklist items to these same items distributed in random order. Half of the participants rated their symptoms on traditionally ordered items and half the participants rated the same items, but in random order and interspersed between items from other validated scales. Differences in prevalence rate and network composition were examined. RESULTS: The prevalence rate differed between the ordered and random item samples. Network analyses using the ordered survey closely replicated the conclusions reached in the existing network analyses literature. However, in the random item survey, network composition differed considerably. CONCLUSION: Order effects appear to have a significant impact on conclusions reached from PTSD network analysis. Prevalence rates were also impacted by order effects. These findings have important diagnostic and clinical treatment implications.

Russian-German five-year-olds: What omissions in sentence repetition tell us about linguistic knowledge, memory skills and their interrelation.

In error analyses using sentence repetition data, most authors focus on word types of omissions. The current study considers serial order in omission patterns independent of functional categories. Data was collected from Russian and German sentence repetition tasks performed by 53 five-year-old bilingual children. Number and positions of word omissions were analyzed. Serial order effects were found in both languages: medial errors made up the largest percentage of errors. Then, the position of omissions was compared to visuo-verbal n-back working memory and non-verbal visual forward short-term memory scores using stepwise hierarchical linear regression models, taking into account demographic variables and receptive language. The interaction differed between languages: there was a significant negative association between omissions in the medial position in German and the final position in Russian and the visuo-verbal n-back memory score. Our study contributes to the understanding of how working memory and language are intertwined in sentence repetition.

Law and order effects: on cognitive dissonance and belief perseverance.

Order of evidence presentation affects the evaluation and the integration of evidence in mock criminal cases. In this study, we aimed to determine whether the order in which incriminating and exonerating evidence is presented influences cognitive dissonance and subsequent display of confirmation bias. Law students (N = 407) were presented with a murder case vignette, followed by incriminating and exonerating evidence in various orders. Contrary to a predicted primacy effect (i.e. early evidence being most influential), a recency effect (i.e. late evidence being most influential) was observed in ratings of likelihood of the suspect's guilt. The cognitive dissonance ratings and conviction rates were not affected by the order of evidence presentation. The effects of evidence presentation order may be limited to specific aspects of legal decisions. However, there is a need to replicate the results using procedures and samples that are more representative of real-life criminal law trials.

Improving reverse correlation analysis of faces: Diagnostics of order effects, runs, rater agreement, and image pairs.

Examinations of the reliability and validity of classification images of faces using the reverse correlation approach remain rare. In the present paper, we focus on order effects of trials, compliance, and reliability effects, as well as the degree of contextual contrast of image pairs. We present different diagnostic methods to examine these three aspects using data from 12 reverse correlation studies conducted both in-lab and online with diverse samples (i.e., from Burkina Faso, China, the Netherlands, the U.S., and an international sample) using five different base faces (i.e., female black, female Asian, female and gender-neutral white, and black/white/female/male morphed composite). For each of the 12 studies, we compare the individual CIs of subgroups of likely non-complier respondents and trials with non-contrastful image pairs to individual CIs of likely compliers and contrastful image pairs. In an appendix, we also examine the effects of filtering out data from individual participants and trials on the signal-to-noise ratio of group CIs. R scripts are publicly available for easy implementation of our suggestions in related research.

Probing Birth-Order Effects on Narrow Traits Using Specification-Curve Analysis.

The idea that birth-order position has a lasting impact on personality has been discussed for the past 100 years. Recent large-scale studies have indicated that birth-order effects on the Big Five personality traits are negligible. In the current study, we examined a variety of more narrow personality traits in a large representative sample ( n = 6,500-10,500 in between-family analyses; n = 900-1,200 in within-family analyses). We used specification-curve analysis to assess evidence for birth-order effects across a range of models implementing defensible yet arbitrary analytical decisions (e.g., whether to control for age effects or to exclude participants on the basis of sibling spacing). Although specification-curve analysis clearly confirmed the previously reported birth-order effect on intellect, we found no meaningful effects on life satisfaction, locus of control, interpersonal trust, reciprocity, risk taking, patience, impulsivity, or political orientation. The lack of meaningful birth-order effects on self-reports of personality was not limited to broad traits but also held for more narrowly defined characteristics.

Diagnostic causal reasoning with verbal information.

In diagnostic causal reasoning, the goal is to infer the probability of causes from one or multiple observed effects. Typically, studies investigating such tasks provide subjects with precise quantitative information regarding the strength of the relations between causes and effects or sample data from which the relevant quantities can be learned. By contrast, we sought to examine people's inferences when causal information is communicated through qualitative, rather vague verbal expressions (e.g., "X occasionally causes A"). We conducted three experiments using a sequential diagnostic inference task, where multiple pieces of evidence were obtained one after the other. Quantitative predictions of different probabilistic models were derived using the numerical equivalents of the verbal terms, taken from an unrelated study with different subjects. We present a novel Bayesian model that allows for incorporating the temporal weighting of information in sequential diagnostic reasoning, which can be used to model both primacy and recency effects. On the basis of 19,848 judgments from 292 subjects, we found a remarkably close correspondence between the diagnostic inferences made by subjects who received only verbal information and those of a matched control group to whom information was presented numerically. Whether information was conveyed through verbal terms or numerical estimates, diagnostic judgments closely resembled the posterior probabilities entailed by the causes' prior probabilities and the effects' likelihoods. We observed interindividual differences regarding the temporal weighting of evidence in sequential diagnostic reasoning. Our work provides pathways for investigating judgment and decision making with verbal information within a computational modeling framework.

Integrating theory into disturbance interaction experiments to better inform ecosystem management.

Managing multiple, interacting disturbances is a key challenge to biodiversity conservation, and one that will only increase as global change drivers continue to alter disturbance regimes. Theoretical studies have highlighted the importance of a mechanistic understanding of stressor interactions for improving the prediction and management of interactive effects. However, many conservation studies are not designed or interpreted in the context of theory and instead focus on case-specific management questions. This is a problem as it means that few studies test the relationships highlighted in theoretical models as being important for ecological management. We explore the extent of this problem among studies of interacting disturbances by reviewing recent experimental studies of the interaction between fire and grazing in terrestrial ecosystems. Interactions between fire and grazing can occur via a number of pathways; one disturbance can modify the other's likelihood, intensity or spatial distribution, or one disturbance can alter the other's impacts on individual organisms. The strength of such interactions will vary depending on disturbance attributes (e.g. size or intensity), and this variation is likely to be nonlinear. We show that few experiments testing fire-grazing interactions are able to identify the mechanistic pathway driving an observed interaction, and most are unable to detect nonlinear effects. We demonstrate how these limitations compromise the ability of experimental studies to effectively inform ecological management. We propose a series of adjustments to the design of disturbance interaction experiments that would enable tests of key theoretical pathways and provide the deeper ecological understanding necessary for effective management. Such considerations are relevant to studies of a broad range of ecological interactions and are critical to informing the management of disturbance regimes in the context of accelerating global change.

Order of Presentation of Dimensions Does Not Systematically Bias Utility Weights from a Discrete Choice Experiment.

BACKGROUND: Discrete choice experiments (DCEs) are increasingly used to value aspects of health. An issue with their adoption is that results may be sensitive to the order in which dimensions of health are presented in the valuation task. Findings in the literature regarding order effects are discordant at present. OBJECTIVES: To quantify the magnitude of order effect of quality-of-life (QOL) dimensions within the context of a DCE designed to produce country-specific value sets for the EORTC Quality of Life Utility Measure-Core 10 dimensions (QLU-C10D), a new utility instrument derived from the widely used cancer-specific QOL questionnaire, the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire-Core 30. METHODS: The DCE comprised 960 choice sets, divided into 60 versions of 16 choice sets, with each respondent assigned to a version. Within each version, the order of QLU-C10D QOL dimensions was randomized, followed by life duration in the last position. The DCE was completed online by 2053 individuals in France and Germany. We analyzed the data with a series of conditional logit models, adjusted for repeated choices within respondent. We used F tests to assess order effects, correcting for multiple hypothesis testing. RESULTS: Each F test failed to reject the null hypothesis of no position effect: 1) all QOL order positions considered jointly; 2) last QOL position only; 3) first QOL position only. Furthermore, the order coefficients were small relative to those of the QLU-C10D QOL dimension levels. CONCLUSIONS: The order of presentation of QOL dimensions within a DCE designed to provide utility weights for the QLU-C10D had little effect on level coefficients of those QOL dimensions.

Less bang for my buck: Diminished anticipated enjoyment contributes to dysphoria-linked deficit in activity behavioural engagement choice.

This study experimentally investigated the role of anticipated enjoyment and effort in mediating dysphoria-related deficit in activity engagement behavioural choice. Using a novel activity information processing task (about a fictional "new" Nintendo Wii sports game called "Tornado Ball"), N = 249 participants (n = 95 High Dysphoria; n = 154 Low Dysphoria) were presented information about the benefits (enjoyable features) and costs (mental and physical effort barriers) as product reviews from another player. The order of cost vs. benefit information was manipulated such that participants either heard cost information before benefit information, or vice versa. They then rated what their anticipated enjoyment and effort will be if they were to play Tornado Ball, before being given the opportunity to choose to try it themselves or not. The High Dysphoria group reported lower anticipated enjoyment (but not higher effort) relative to the Low Dysphoria group, but only when cost information was presented first. Importantly, a moderated mediation showed that the High Dysphoria group reported lower tendency to choose activity engagement (game play) as a function of having lower anticipated enjoyment, but only when cost information was presented first. The present finding indicate that reduced anticipated enjoyment may causally contribute to dysphoria-linked deficits in activity engagement behavioural choice.

A fragile effect: The influence of episodic memory on delay discounting.

Delay discounting occurs when a reward loses value as a function of delay. Episodic future thinking (EFT) reliably decreases delay discounting. EFT may share cognitive features with recalling episodic memories such as constructive episodic simulation. We therefore explored whether recalling episodic memories also reduces delay discounting. In Experiment 1, participants wrote about episodic memories and recalled those memories before completing a delay discounting task. Episodic memories reduced delay discounting according to one commonly used delay discounting measure (area under the curve) but not another (using the hyperbolic model). Experiment 2 compared the effects of general and episodic memories. Neither general nor episodic memories significantly decreased delay discounting compared with a control "counting" condition, but episodic memories reduced delay discounting compared with general memories under some conditions. In Experiment 3, episodic memories did not decrease delay discounting compared with three other control conditions while EFT did. Experiment 3 therefore found that thinking must be both episodic and future orientated to reduce delay discounting. Together, these results suggest that episodic thinking is not sufficient to reliably decrease delay discounting, rather, features unique to episodic future thinking are required. Episodic memory might reduce delay discounting in some contexts, but this effect is small and fragile.

Broader benefits of the pretesting effect: Placement matters.

Taking a test before learning new information in a lesson improves memory for that information - pretesting effect. Although the specific memory benefit of a pretest on pretested information has been well documented, it remains unclear what the circumstances necessary for the broader memory benefit are - that is, the benefit of a pretest on memory of information in the lesson that was not pretested. Sometimes this broader benefit is present, but other times it disappears or reverses. We investigated if manipulating where the non-pretested information appears in a lesson - either before or after the pretested information - affects broader memory benefits. Participants read a text passage (Experiment 1) or watched a video lecture (Experiment 2) after completing a pretest on half of the lesson content. The pretested information appeared either at the beginning (prior to the non-pretested information) or at the end (after the non-pretested information) of the lesson. The final test assessed memory of both pretested and non-pretested information. We hypothesized that pretests trigger an attentional window that opens during the lesson and closes after pretested information has been identified. Any information, including non-pretested information, will benefit from being in this window because it is more likely to be processed. We found that memory of non-pretested information is better if the non-pretested information is presented at the beginning versus at the end of a lesson, regardless of delivery modality. These results indicate that the presentation order of pretested versus non-pretested information contributes to the broader memory benefits associated with pretesting.

Let's come to order: The influence of question order on willingness to register as an organ donor.

RATIONALE: As over 90% of people who register to be organ donors do so at the Department of Motor Vehicles (DMV), DMVs are perceived as a key context for increasing donor registration rates. Scholars have recently noted that the driver's license application itself, including the placement of the donor registration item relative to other questions, can possibly influence donor registration behavior. The goal of the current study was to experimentally investigate this possibility. METHOD: We conducted an experiment using Amazon's Mechanical Turk (MTurk) between March and May of 2021 to investigate the influence of question order on donor registration willingness. Participants received a question regarding their willingness to register either before or after a series of health and legal questions often asked at DMVs. RESULTS: The placement of the donor registration question had a positive effect on registration willingness for non-registered individuals (OR = 2.01, 95% CI [1.59, 2.54]) and previously registered donors (OR = 2.57, 95% CI [2.22, 2.99]). CONCLUSION: Changing the question order of driver's license applications has the potential to influence registration rates.

Sequential treatment effects on phage-antibiotic synergistic application against multi-drug-resistant Acinetobacter baumannii.

Bacteriophage (phage) therapy, exploiting phages which are the natural enemies of bacteria, has been re-introduced to treat multidrug-resistant (MDR) bacterial infections. However, some intrinsic drawbacks of phages are overshadowing their clinical use, particularly the narrow host spectrum and rapid emergence of resistance upon treatment. The use of phage-antibiotic combinations exhibiting synergistic bacterial killing [termed 'phage-antibiotic synergy' (PAS)] has therefore been proposed. It is well reported that the types and doses of phages and antibiotics are critical in achieving PAS. However, the impact of treatment order has received less research attention. As such, this study used an Acinetobacter baumannii phage vB_AbaM-IME-AB2 and colistin as a model PAS combination to elucidate the order effects in-vitro. While application of the phage 8 h before colistin treatment demonstrated the greatest antibacterial synergy, it failed to prevent the development of phage resistance. On the other hand, simultaneous application and antibiotic followed by phage application were able to suppress/delay the development of resistance effectively, and simultaneous application demonstrated superior antibacterial and antibiofilm activities. Further in-vivo investigation is required to confirm the impact of treatment order on PAS.

Structural testing of concrete walls on-edge with combined axial and out-of-plane loading.

Full-scale testing of structural components can be time consuming and difficult. The design of full-scale slender concrete walls is highly influenced and controlled by second-order and out-of-plane bending loads. Previous experiments on out-plane bending of slender walls and insulated walls have been performed with bending in the direction of gravity (with or against). Additionally, most of the research considering out-of-plane bending does not include an axial load and suffers from inaccurate results due to not simulating the actual loading and constraining conditions or safety issues. This testing method was developed expressly for the determination of slender wall behavior in insulated concrete panels and verified on solid slender walls, which are well understood. The testing setup presented has the following advantages•Reduces the risk of cracking panel prior to testing and provides safe and rapid testing.•Offers ease of implementation in labs with height restrictions, given sufficient floor space.•Integrates axial and lateral uniform loading.

Sequential information processing in persuasion.

We present a theory of sequential information processing in persuasion (SIP). It extends assumptions of the heuristic-systematic model, in particular the idea that information encountered early in a persuasion situation may affect the processing of subsequent information. SIP also builds on the abstraction from content-related dichotomies in accord with the parametric unimodel of social judgment. SIP features one constitutional axiom and three main postulates: (A) Persuasion is the sequential processing of information that is relevant to judgment formation. (1) Inferences drawn from initial information may bias the processing of subsequent information if they are either activated rules or valence expectations that are relevant to the subsequent information. (2) Inferences drawn from initial information are resistant to change. Thus, the interpretation of subsequent information is assimilated to inferences drawn from the initial information. Or, if assimilation is impossible, contrast effects occur. (3) The overall effect of a persuasion attempt corresponds to the recipient's judgment at the moment the processing of information is terminated. We illustrate how our predictions for assimilation and contrast effects may be tested by presenting results from an experiment (N = 216) in which we presented exactly the same arguments but varied the processing sequence. We discuss theoretical and applied implications of sequence effects for persuasion phenomena, as well as challenges for further research developing and testing the theory.

Behavioral Theories That Have Influenced the Way Health State Preferences Are Elicited and Interpreted: A Bibliometric Mapping Analysis of the Time Trade-Off Method With VOSviewer Visualization.

Aim: The aim of this paper is to develop an understanding of how behavioral theories have influenced the way preferences for health-related quality of life are elicited and interpreted. We focus on the Time Trade-off (TTO) method given it represents the quality-adjusted life-year (QALY) concept-that survival in less-than-full health can be deemed equivalent to a shorter survival in full health. To our knowledge this is the first review using a combination of systematic scoping review, bibliometrics and VOSviewer visualization to map the development of ideas in health economics. Methods: A priori, we selected three behavioral theories to explore within our review, referred to here as Expected Utility Theory, Non-Expected Utility Theory and Probabilistic Choice Theory. A fourth topic, Order Effects, is defined broadly to encompass behavioral theories around timing/sequence of events. For the main search, Scopus was used to identify literature that had (a) elicited TTO values and/or (b) contributed to the way TTO values were elicited and interpreted, from inception to July 2021. Papers that focused on the latter category were given the label "behavioral" and underwent additional analyses. A two stage-screening was applied to assess eligibility. Co-citation, co-authorship and co-occurrence of keywords was used to chart the development of TTO over time. Results: A total of 1,727 records were retrieved from Scopus and were supplemented by an additional 188 papers. There were 856 applied and 280 behavioral papers included in the final corpus, with the behavioral set split equally into four sets of 70 papers to chart the development of keywords over time: (1) 1972-1999; (2) 2000-2010, (3) 2010-2015 and (4) 2015-2021. Discussion: The keyword analysis suggested that whilst some ideas transition quickly from economic theory to the TTO literature, such as the impact of Order Effects, others take longer to be assimilated, for example Non-Expected Utility models or failure of constant discounting. It is therefore important that researchers within health economics work more closely with those in mainstream economics and keep abreast of the wider economics and behavioral sciences to expedite the uptake of new and relevant ideas.

The Effect of Question Order on Outcomes in the Core Outcome Set for Brief Alcohol Interventions Among Online Help-Seekers: Protocol for a Factorial Randomized Trial.

BACKGROUND: A core outcome set (COS) for trials and evaluations of the effectiveness and efficacy of alcohol brief interventions (ABIs) has recently been established through international consensus to address the variability of outcomes evaluated. OBJECTIVE: This is a protocol for studies to assess if there are order effects among the questions included in the COS. METHODS: The 10 items of the COS are organized into 4 clusters. A factorial design will be used with 24 arms, where each arm represents 1 order of the 4 clusters. Individuals searching online for help will be asked to complete a questionnaire, and consenting participants will be randomized to 1 of the 24 arms (double-blind with equal allocation). Participants will be included if they are 18 years or older. The primary analyses will (1) estimate how the order of the clusters of outcomes affects how participants respond and (2) investigate patterns of abandonment of the questionnaire. RESULTS: Data collection is expected to commence in November 2020. A Bayesian group sequential design will be used with interim analyses planned for every 50 participants completing the questionnaire. Data collection will end no more than 24 months after commencement, and the results are expected to be published no later than December 2023. CONCLUSIONS: Homogenizing the outcomes evaluated in studies of ABIs is important to support synthesis, and the COS is an important step toward this goal. Determining whether there may be issues with the COS question order may improve confidence in using it and speed up its dissemination in the research community. We encourage others to adopt the protocol as a study within their trial as they adopt the ORBITAL (Outcome Reporting in Brief Intervention Trials: Alcohol) COS to build a worldwide repository and provide materials to support such analysis. TRIAL REGISTRATION: ISRCTN Registry ISRCTN17954645; http://www.isrctn.com/ISRCTN17954645. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/24175.

Primer on quantum cognition.

Quantum cognition is a new field in psychology, which is characterized by the application of quantum probability theory to human judgment and decision making behavior. This article provides an introduction that presents several examples to illustrate in a simple and concrete manner how to apply these principles to interesting psychological phenomena. Following each simple example, we present the general mathematical derivations and new predictions related to these applications.

Positive Effects of Nature on Cognitive Performance Across Multiple Experiments: Test Order but Not Affect Modulates the Cognitive Effects.

Interactions with natural environments and nature-related stimuli have been found to be beneficial to cognitive performance, in particular on executive cognitive tasks with high demands on directed attention processes. However, results vary across different studies. The aim of the present paper was to evaluate the effects of nature vs. urban environments on cognitive performance across all of our published and new/unpublished studies testing the effects of different interactions with nature vs. urban/built control environments, on an executive-functioning test with high demands on directed attention-the backwards digit span (BDS) task. Specific aims in this study were to: (1) evaluate the effect of nature vs. urban environment interactions on BDS across different exposure types (e.g., real-world vs. artificial environments/stimuli); (2) disentangle the effects of testing order (i.e., effects caused by the order in which experimental conditions are administered) from the effects of the environment interactions, and (3) test the (mediating) role of affective changes on BDS performance. To this end, data from 13 experiments are presented, and pooled data-analyses are performed. Results from the pooled data-analyses (N = 528 participants) showed significant time-by-environment interactions with beneficial effects of nature compared to urban environments on BDS performance. There were also clear interactions with the order in which environment conditions were tested. Specifically, there were practice effects across environment conditions in first sessions. Importantly, after parceling out initial practice effects, the positive effects of nature compared to urban interactions on BDS performance were magnified. Changes in positive or negative affect did not mediate the beneficial effects of nature on BDS performance. These results are discussed in relation to the findings of other studies identified in the literature. Uncontrolled and confounding order effects (i.e., effects due to the order of experimental conditions, rather than the treatment conditions) may explain some of the inconsistent findings across studies in the literature on nature effects on cognitive performance. In all, these results highlight the robustness of the effects of natural environments on cognition, particularly when confounding order effects have been considered, and provide a more nuanced account of when a nature intervention will be most effective.