Systematic testing of three Language Models reveals low language accuracy, absence of response stability, and a yes-response bias.

Humans are universally good in providing stable and accurate judgments about what forms part of their language and what not. Large Language Models (LMs) are claimed to possess human-like language abilities; hence, they are expected to emulate this behavior by providing both stable and accurate answers, when asked whether a string of words complies with or deviates from their next-word predictions. This work tests whether stability and accuracy are showcased by GPT-3/text-davinci-002, GPT-3/text-davinci-003, and ChatGPT, using a series of judgment tasks that tap on 8 linguistic phenomena: plural attraction, anaphora, center embedding, comparatives, intrusive resumption, negative polarity items, order of adjectives, and order of adverbs. For every phenomenon, 10 sentences (5 grammatical and 5 ungrammatical) are tested, each randomly repeated 10 times, totaling 800 elicited judgments per LM (total n = 2,400). Our results reveal variable above-chance accuracy in the grammatical condition, below-chance accuracy in the ungrammatical condition, a significant instability of answers across phenomena, and a yes-response bias for all the tested LMs. Furthermore, we found no evidence that repetition aids the Models to converge on a processing strategy that culminates in stable answers, either accurate or inaccurate. We demonstrate that the LMs' performance in identifying (un)grammatical word patterns is in stark contrast to what is observed in humans (n = 80, tested on the same tasks) and argue that adopting LMs as theories of human language is not motivated at their current stage of development.

Examining the relationships between cognition and auditory hallucinations: A systematic review.

OBJECTIVE: Auditory hallucinations (hearing voices) have been associated with a range of altered cognitive functions, pertaining to signal detection, source-monitoring, memory, inhibition and language processes. Yet, empirical results are inconsistent. Despite this, several theoretical models of auditory hallucinations persist, alongside increasing emphasis on the utility of a multidimensional framework. Thus, clarification of current evidence across the broad scope of proposed mechanisms is warranted. METHOD: A systematic search of the Web of Science, PubMed and Scopus databases was conducted. Records were screened to confirm the use of an objective behavioural cognitive task, and valid measurement of hallucinations specific to the auditory modality. RESULTS: Auditory hallucinations were primarily associated with difficulties in perceptual decision-making (i.e. reduced sensitivity/accuracy for signal-noise discrimination; liberal responding to ambiguity), source-monitoring (i.e. self-other and temporal context confusion), working memory and language function (i.e. reduced verbal fluency). Mixed or limited support was observed for perceptual feature discrimination, imagery vividness/illusion susceptibility, source-monitoring for stimulus form and spatial context, recognition and recall memory, executive functions (e.g. attention, inhibition), emotion processing and language comprehension/hemispheric organisation. CONCLUSIONS: Findings were considered within predictive coding and self-monitoring frameworks. Of concern was the portion of studies which - despite offering auditory-hallucination-specific aims and inferences - employed modality-general measures, and/or diagnostic-based contrasts with psychologically healthy individuals. This review highlights disparities within the literature between theoretical conceptualisations of auditory hallucinations and the body of rigorous empirical evidence supporting such inferences. Future cognitive investigations, beyond the schizophrenia-spectrum, which explicitly define and measure the timeframe and sensory modality of hallucinations, are recommended.

SpeedyIBL: A comprehensive, precise, and fast implementation of instance-based learning theory.

Instance-based learning theory (IBLT) is a comprehensive account of how humans make decisions from experience during dynamic tasks. Since it was first proposed almost two decades ago, multiple computational models have been constructed based on IBLT (i.e., IBL models). These models have been demonstrated to be very successful in explaining and predicting human decisions in multiple decision-making contexts. However, as IBLT has evolved, the initial description of the theory has become less precise, and it is unclear how its demonstration can be expanded to more complex, dynamic, and multi-agent environments. This paper presents an updated version of the current theoretical components of IBLT in a comprehensive and precise form. It also provides an advanced implementation of the full set of theoretical mechanisms, SpeedyIBL, to unlock the capabilities of IBLT to handle a diverse taxonomy of individual and multi-agent decision-making problems. SpeedyIBL addresses a practical computational issue in past implementations of IBL models, the curse of exponential growth, that emerges from memory-based tabular computations. When more observations accumulate over time, there is an exponential growth of the memory of instances that leads directly to an exponential slowdown of the computational time. Thus, SpeedyIBL leverages parallel computation with vectorization to speed up the execution time of IBL models. We evaluate the robustness of SpeedyIBL over an existing implementation of IBLT in decision games of increased complexity. The results not only demonstrate the applicability of IBLT through a wide range of decision-making tasks, but also highlight the improvement of SpeedyIBL over its prior implementation as the complexity of decision features the of agents increase. The library is open sourced for the use of the broad research community.

On what motivates us: a detailed review of intrinsic v. extrinsic motivation.

Motivational processes underlie behaviors that enrich the human experience, and impairments in motivation are commonly observed in psychiatric illness. While motivated behavior is often examined with respect to extrinsic reinforcers, not all actions are driven by reactions to external stimuli; some are driven by 'intrinsic' motivation. Intrinsically motivated behaviors are computationally similar to extrinsically motivated behaviors, in that they strive to maximize reward value and minimize punishment. However, our understanding of the neurocognitive mechanisms that underlie intrinsically motivated behavior remains limited. Dysfunction in intrinsic motivation represents an important trans-diagnostic facet of psychiatric symptomology, but due to a lack of clear consensus, the contribution of intrinsic motivation to psychopathology remains poorly understood. This review aims to provide an overview of the conceptualization, measurement, and neurobiology of intrinsic motivation, providing a framework for understanding its potential contributions to psychopathology and its treatment. Distinctions between intrinsic and extrinsic motivation are discussed, including divergence in the types of associated rewards or outcomes that drive behavioral action and choice. A useful framework for understanding intrinsic motivation, and thus separating it from extrinsic motivation, is developed and suggestions for optimization of paradigms to measure intrinsic motivation are proposed.

Evidence for Shifting Cognitive Strategies when Icons Appear in Unexpected Locations.

OBJECTIVE: The present study examines the cognitive effects of placing icons in unexpected spatial locations within websites. BACKGROUND: Prior research has revealed evidence for cognitive conflict when web icons occur in unexpected locations (e.g., cart, top left), generally consistent with a dynamical systems models. Here, we compare the relative strength of evidence for both dual and dynamical systems models. METHODS: Participants clicked on icons located in either expected (e.g., cart, top right) or unexpected (e.g., cart, top left) locations while mouse trajectories were continuously recorded. Trajectories were classified according to prototypes associated with each cognitive model. The dynamical systems model predicts curved trajectories, while the dual-systems model predicts straight and change of mind trajectories. RESULTS: Trajectory classification revealed that curved trajectories increased (+11%), while straight and change of mind trajectories decreased (-12%) when target icons occurred in unexpected locations (p < .001). CONCLUSION: Rather than employing a single cognitive strategy, users shift from a primarily dual-systems to dynamical systems strategy when icons occur in unexpected locations. APPLICATION: Potential applications of this work include the assessment of cognitive impacts such as mental workload and cognitive conflict during real-time interaction with websites and other screen-based interfaces, personalization and adaptive interfaces based on an individual's cognitive strategy, and data-driven A/B testing of alternative interface designs.

Moment-to-moment dynamics between auditory verbal hallucinations and negative affect and the role of beliefs about voices.

BACKGROUND: Negative affect (NA) has been suggested to be both an antecedent and a consequence of auditory verbal hallucinations (AVH). Furthermore, negative appraisals of voices have been theorized to contribute to the maintenance of AVH. Using the experience sampling method (ESM), this study examined the bi-directional relationship between NA and AVH, and the moderating effect of negative beliefs about voices. METHODS: Forty-seven patients diagnosed with schizophrenia spectrum disorders with frequent AVH completed a clinical interview, followed by ESM for 10 times a day over 6 days on an electronic device. Time-lagged analyses were conducted using multilevel regression modeling. Beliefs about voices were assessed at baseline. RESULTS: A total of 1654 data points were obtained. NA predicted an increase in AVH in the subsequent moment, and AVH predicted an increase in NA in the subsequent moment. Baseline beliefs about voices as malevolent and omnipotent significantly strengthened the association between NA and AVH within the same moment. In addition, the belief of omnipotence was associated with more hallucinatory experiences in the moment following NA. However, beliefs about voices were not associated directly with momentary levels of NA or AVH. CONCLUSIONS: Experiences of NA and AVH drove each other, forming a feedback loop that maintained the voices. The associations between NA and AVH, either within the same moment or across moments, were exacerbated by negative beliefs about voices. Our results suggest that affect-improving interventions may stop the feedback loop and reduce AVH frequency.

The jumping to conclusions reasoning bias as a cognitive factor contributing to psychosis progression and persistence: findings from NEMESIS-2.

BACKGROUND: Contemporary models of psychosis implicate the importance of affective dysregulation and cognitive factors (e.g. biases and schemas) in the development and maintenance of psychotic symptoms, but studies testing proposed mechanisms remain limited. This study, uniquely using a prospective design, investigated whether the jumping to conclusions (JTC) reasoning bias contributes to psychosis progression and persistence. METHODS: Data were derived from the second Netherlands Mental Health Survey and Incidence Study (NEMESIS-2). The Composite International Diagnostic Interview and an add-on instrument were used to assess affective dysregulation (i.e. depression, anxiety and mania) and psychotic experiences (PEs), respectively. The beads task was used to assess JTC bias. Time series analyses were conducted using data from T1 and T2 (N = 8666), excluding individuals who reported high psychosis levels at T0. RESULTS: Although the prospective design resulted in low statistical power, the findings suggest that, compared to those without symptoms, individuals with lifetime affective dysregulation were more likely to progress from low/moderate psychosis levels (state of 'aberrant salience', one or two PEs) at T1 to high psychosis levels ('frank psychosis', three or more PEs or psychosis-related help-seeking behaviour) at T2 if the JTC bias was present [adj. relative risk ratio (RRR): 3.8, 95% confidence interval (CI) 0.8-18.6, p = 0.101]. Similarly, the JTC bias contributed to the persistence of high psychosis levels (adj. RRR: 12.7, 95% CI 0.7-239.6, p = 0.091). CONCLUSIONS: We found some evidence that the JTC bias may contribute to psychosis progression and persistence in individuals with affective dysregulation. However, well-powered prospective studies are needed to replicate these findings.

Development and Validation of a Descriptive Cognitive Model for Predicting Usability Issues in a Low-Code Development Platform.

OBJECTIVE: The aim of the study was the development and evaluation of a Descriptive Cognitive Model (DCM) for the identification of three types of usability issues in a low-code development platform (LCDP). BACKGROUND: LCDPs raise the level of abstraction of software development by freeing end-users from implementation details. An effective LCDP requires an understanding of how its users conceptualize programming. It is necessary to identify the gap between the LCDP end-users' conceptualization of programming and the actions required by the platform. It is also relevant to evaluate how the conceptualization of the programming tasks varies according to the end-users' skills. METHOD: DCMs are widely used in the description and analysis of the interaction between users and systems. We propose a DCM which we called PRECOG that combines task decomposition methods with knowledge-based descriptions and criticality analysis. This DCM was validated using empirical techniques to provide the best insight regarding the users' interaction performance. Twenty programmers (10 experts, 10 novices) were observed using an LCDP and their interactions were analyzed according to our DCM. RESULTS: The DCM correctly identified several problems felt by first-time platform users. The patterns of issues observed were qualitatively different between groups. Experts mainly faced interaction-related problems, while novices faced problems attributable to a lack of programming skills. CONCLUSION: By applying the proposed DCM we were able to predict three types of interaction problems felt by first-time users of the LCDP. APPLICATION: The method is applicable when it is relevant to identify possible interaction problems, resulting from the users' background knowledge being insufficient to guarantee a successful completion of the task at hand.

Clinical and theoretical relevance of responses to analogues of psychotic experiences in people with psychotic experiences with and without a need-for-care: an experimental study.

BACKGROUND: Certain ways of responding to psychotic experiences (PEs) appear more commonly associated with clinical distress (e.g. avoidance) and other ways with benign or positive outcomes (e.g. reappraisal and acceptance). Past research has largely been limited to retrospective self-report. We aimed to compare clinical and non-clinical individuals on experimental analogues of anomalous experiences. METHOD: Response styles of two groups with persistent PEs (clinical n = 84; non-clinical n = 92) and a control group without PEs (n = 83) were compared following experimental analogues of thought interference (Cards Task, Telepath) and hearing voices (Virtual Acoustic Space Paradigm). RESULTS: The non-clinical group with PEs were less likely to endorse unhelpful response styles, such as passive responding or attempts to avoid, suppress, worry about or control mental experiences, compared with the clinical group on all three tasks. The clinical group were more likely to endorse unhelpful response styles compared with controls on two out of three tasks (Cards Task and Telepath). The non-clinical group performed similarly to controls on unhelpful responding across all tasks. There were no group differences for helpful response styles, such as cognitive reappraisal or mindful acceptance of experiences. CONCLUSIONS: In line with cognitive models of psychosis, the findings suggest that the way in which individuals respond to unusual experiences may be an important factor in understanding clinical distress, supporting the therapeutic rationale of targeting potentially unhelpful patterns of response.

Attentional biases in depression: Relation to disorder severity, rumination, and anhedonia.

INTRODUCTION: According to cognitive models of depression, selective attentional biases (ABs) for mood-congruent information are core vulnerability factors of depression maintenance. However, findings concerning the presence of these biases in depression are mixed. This study aims to clarify the presence of these ABs among individuals with clinical and subclinical depression. METHOD: We compared three groups based on a semi-structured diagnostic interview and a depressive symptoms scale (BDI-II): 34 individuals with major depressive disorder (clinically depressed); 35 with a dysphoric mood but without the criteria of major depressive disorder (i.e., subclinically depressed), and 26 never been depressed individuals. We examined AB for sad and happy materials in three modified versions of the exogenous cueing task using scenes, facial expressions, and words. Brooding, anhedonia, and anxiety were also evaluated. RESULTS: In contrast to our hypotheses, there were no ABs for negative or positive information, regardless of the task and the groups. Neither the association between AB toward negative information and brooding nor the one between AB away from positive stimuli and anhedonia was significant. Bayes factors analyses revealed that the present pattern of findings does not result from a lack of statistical power. DISCUSSION: Our results raise questions about how common AB is in depression. From a theoretical point of view, because individuals with depression did not exhibit AB, our results also seemingly challenge the claim that AB figures prominently in the maintenance of depression. We believe the present null results to be particularly useful for future meta-research in the field.