Embers of autoregression show how large language models are shaped by the problem they are trained to solve.

The widespread adoption of large language models (LLMs) makes it important to recognize their strengths and limitations. We argue that to develop a holistic understanding of these systems, we must consider the problem that they were trained to solve: next-word prediction over Internet text. By recognizing the pressures that this task exerts, we can make predictions about the strategies that LLMs will adopt, allowing us to reason about when they will succeed or fail. Using this approach-which we call the teleological approach-we identify three factors that we hypothesize will influence LLM accuracy: the probability of the task to be performed, the probability of the target output, and the probability of the provided input. To test our predictions, we evaluate five LLMs (GPT-3.5, GPT-4, Claude 3, Llama 3, and Gemini 1.0) on 11 tasks, and we find robust evidence that LLMs are influenced by probability in the hypothesized ways. Many of the experiments reveal surprising failure modes. For instance, GPT-4's accuracy at decoding a simple cipher is 51% when the output is a high-probability sentence but only 13% when it is low-probability, even though this task is a deterministic one for which probability should not matter. These results show that AI practitioners should be careful about using LLMs in low-probability situations. More broadly, we conclude that we should not evaluate LLMs as if they are humans but should instead treat them as a distinct type of system-one that has been shaped by its own particular set of pressures.

Revisiting Jung's Theory of Archetypes.

Erik Goodwyn in this issue ("Demystifying Jung's 'Archetypes' with Embodied Cognition") argues for a reexamination of the clinical and neuropsychological relevance of Jungian archetypes. This assertion is examined with respect to the evidence Goodwyn provides as well as in the larger context of cognitive science as it is applied to theories rooted in past models of cognition.

The strategic allocation theory of vigilance.

Despite its importance in different occupational and everyday contexts, vigilance, typically defined as the capacity to sustain attention over time, is remarkably limited. What explains these limits? Two theories have been proposed. The Overload Theory states that being vigilant consumes limited information-processing resources; when depleted, task performance degrades. The Underload Theory states that motivation to perform vigilance tasks declines over time, thereby prompting attentional shifts and hindering performance. We highlight some conceptual and empirical problems for both theories and propose an alternative: the Strategic Allocation Theory. For the Strategic Allocation Theory, performance on vigilance tasks optimizes as a function of intrinsic and extrinsic motivations, including metacognitive factors such as the expected value of effort and the expected value of planning. Limited capacities must be deployed across task sets to maximize expected reward. The observed limits of vigilance reflect changes in the perceived value of, among other things, sustaining attention to a task rather than attending to something else. Drawing from recent computational theories of cognitive control and meta-reasoning, we argue that the Strategic Allocation Theory explains more phenomena related to vigilance behavior than other theories, including self-report data. Finally, we outline some of the testable predictions the theory makes across several experimental paradigms. This article is categorized under: Philosophy > Foundations of Cognitive Science Psychology > Attention.

Patricia Goldman-Rakic: A Pioneer in Cognitive Neuroscience.

Patricia Goldman-Rakic (1937-2003) was a trailblazing neuroscientist whose groundbreaking work greatly advanced our understanding of the prefrontal cortex and its crucial role in higher cognitive functions like working memory and executive function. Her innovative research, which integrated anatomical, electrophysiological, and behavioral methods, provided foundational insights into the neural basis of cognition. This review underscores her significant contributions, personal challenges, and the enduring influence of her work in the field of neuroscience.

ChatGPT as Research Scientist: Probing GPT's capabilities as a Research Librarian, Research Ethicist, Data Generator, and Data Predictor.

How good a research scientist is ChatGPT? We systematically probed the capabilities of GPT-3.5 and GPT-4 across four central components of the scientific process: as a Research Librarian, Research Ethicist, Data Generator, and Novel Data Predictor, using psychological science as a testing field. In Study 1 (Research Librarian), unlike human researchers, GPT-3.5 and GPT-4 hallucinated, authoritatively generating fictional references 36.0% and 5.4% of the time, respectively, although GPT-4 exhibited an evolving capacity to acknowledge its fictions. In Study 2 (Research Ethicist), GPT-4 (though not GPT-3.5) proved capable of detecting violations like p-hacking in fictional research protocols, correcting 88.6% of blatantly presented issues, and 72.6% of subtly presented issues. In Study 3 (Data Generator), both models consistently replicated patterns of cultural bias previously discovered in large language corpora, indicating that ChatGPT can simulate known results, an antecedent to usefulness for both data generation and skills like hypothesis generation. Contrastingly, in Study 4 (Novel Data Predictor), neither model was successful at predicting new results absent in their training data, and neither appeared to leverage substantially new information when predicting more vs. less novel outcomes. Together, these results suggest that GPT is a flawed but rapidly improving librarian, a decent research ethicist already, capable of data generation in simple domains with known characteristics but poor at predicting novel patterns of empirical data to aid future experimentation.

Fundamental processes in sensorimotor learning: Reasoning, refinement, and retrieval.

Motor learning is often viewed as a unitary process that operates outside of conscious awareness. This perspective has led to the development of sophisticated models designed to elucidate the mechanisms of implicit sensorimotor learning. In this review, we argue for a broader perspective, emphasizing the contribution of explicit strategies to sensorimotor learning tasks. Furthermore, we propose a theoretical framework for motor learning that consists of three fundamental processes: reasoning, the process of understanding action-outcome relationships; refinement, the process of optimizing sensorimotor and cognitive parameters to achieve motor goals; and retrieval, the process of inferring the context and recalling a control policy. We anticipate that this '3R' framework for understanding how complex movements are learned will open exciting avenues for future research at the intersection between cognition and action.

Machine learning-based energy management and power forecasting in grid-connected microgrids with multiple distributed energy sources.

The growing integration of renewable energy sources into grid-connected microgrids has created new challenges in power generation forecasting and energy management. This paper explores the use of advanced machine learning algorithms, specifically Support Vector Regression (SVR), to enhance the efficiency and reliability of these systems. The proposed SVR algorithm leverages comprehensive historical energy production data, detailed weather patterns, and dynamic grid conditions to accurately forecast power generation. Our model demonstrated significantly lower error metrics compared to traditional linear regression models, achieving a Mean Squared Error of 2.002 for solar PV and 3.059 for wind power forecasting. The Mean Absolute Error was reduced to 0.547 for solar PV and 0.825 for wind scenarios, and the Root Mean Squared Error (RMSE) was 1.415 for solar PV and 1.749 for wind power, showcasing the model's superior accuracy. Enhanced predictive accuracy directly contributes to optimized resource allocation, enabling more precise control of energy generation schedules and reducing the reliance on external power sources. The application of our SVR model resulted in an 8.4% reduction in overall operating costs, highlighting its effectiveness in improving energy management efficiency. Furthermore, the system's ability to predict fluctuations in energy output allowed for adaptive real-time energy management, reducing grid stress and enhancing system stability. This approach led to a 10% improvement in the balance between supply and demand, a 15% reduction in peak load demand, and a 12% increase in the utilization of renewable energy sources. Our approach enhances grid stability by better balancing supply and demand, mitigating the variability and intermittency of renewable energy sources. These advancements promote a more sustainable integration of renewable energy into the microgrid, contributing to a cleaner, more resilient, and efficient energy infrastructure. The findings of this research provide valuable insights into the development of intelligent energy systems capable of adapting to changing conditions, paving the way for future innovations in energy management. Additionally, this work underscores the potential of machine learning to revolutionize energy management practices by providing more accurate, reliable, and cost-effective solutions for integrating renewable energy into existing grid infrastructures.

Can Large Language Models Counter the Recent Decline in Literacy Levels? An Important Role for Cognitive Science.

Literacy is in decline in many parts of the world, accompanied by drops in associated cognitive skills (including IQ) and an increasing susceptibility to fake news. It is possible that the recent explosive growth and widespread deployment of Large Language Models (LLMs) might exacerbate this trend, but there is also a chance that LLMs can help turn things around. We argue that cognitive science is ideally suited to help steer future literacy development in the right direction by challenging and informing current educational practices and policy. Cognitive scientists have the right interdisciplinary skills to study, analyze, evaluate, and change LLMs to facilitate their critical use, to encourage turn-taking that promotes rather than hinders literacy, to support literacy acquisition in diverse and equitable ways, and to scaffold potential future changes in what it means to be literate. We urge cognitive scientists to take up this mantle-the future impact of LLMs on human literacy skills is too important to be left to the large, predominately U.S.-based tech companies.

Minds in movement: embodied cognition in the age of artificial intelligence.

This theme issue brings together researchers from diverse fields to assess the current status and future prospects of embodied cognition in the age of generative artificial intelligence. In this introduction, we first clarify our view of embodiment as a potentially unifying concept in the study of cognition, characterizing this as a perspective that questions mind-body dualism and recognizes a profound continuity between sensorimotor action in the world and more abstract forms of cognition. We then consider how this unifying concept is developed and elaborated by the other contributions to this issue, identifying the following two key themes: (i) the role of language in cognition and its entanglement with the body and (ii) bodily mechanisms of interpersonal perception and alignment across the domains of social affiliation, teaching and learning. On balance, we consider that embodied approaches to the study of cognition, culture and evolution remain promising, but will require greater integration across disciplines to fully realize their potential. We conclude by suggesting that researchers will need to be ready and able to meet the various methodological, theoretical and practical challenges this will entail and remain open to encountering markedly different viewpoints about how and why embodiment matters. This article is the part of this theme issue 'Minds in movement: embodied cognition in the age of artificial intelligence'.

Language models, like humans, show content effects on reasoning tasks.

reasoning is a key ability for an intelligent system. Large language models (LMs) achieve above-chance performance on abstract reasoning tasks but exhibit many imperfections. However, human abstract reasoning is also imperfect. Human reasoning is affected by our real-world knowledge and beliefs, and shows notable "content effects"; humans reason more reliably when the semantic content of a problem supports the correct logical inferences. These content-entangled reasoning patterns are central to debates about the fundamental nature of human intelligence. Here, we investigate whether language models-whose prior expectations capture some aspects of human knowledge-similarly mix content into their answers to logic problems. We explored this question across three logical reasoning tasks: natural language inference, judging the logical validity of syllogisms, and the Wason selection task. We evaluate state of the art LMs, as well as humans, and find that the LMs reflect many of the same qualitative human patterns on these tasks-like humans, models answer more accurately when the semantic content of a task supports the logical inferences. These parallels are reflected in accuracy patterns, and in some lower-level features like the relationship between LM confidence over possible answers and human response times. However, in some cases the humans and models behave differently-particularly on the Wason task, where humans perform much worse than large models, and exhibit a distinct error pattern. Our findings have implications for understanding possible contributors to these human cognitive effects, as well as the factors that influence language model performance.

Introduction to the Emerging Cognitive Science of Distributed Human-Autonomy Teams.

Teams are a fundamental aspect of life-from sports to business, to defense, to science, to education. While the cognitive sciences tend to focus on information processing within individuals, others have argued that teams are also capable of demonstrating cognitive capacities similar to humans, such as skill acquisition and forgetting (cf., Cooke, Gorman, Myers, & Duran, 2013; Fiore et al., 2010). As artificially intelligent and autonomous systems improve in their ability to learn, reason, interact, and coordinate with human teammates combined with the observation that teams can express cognitive capacities typically seen in individuals, a cognitive science of teams is emerging. Consequently, new questions are being asked about teams regarding teamness, trust, the introduction and effects of autonomous systems on teams, and how best to measure team behavior and phenomena. In this topic, four facets of human-autonomy team cognition are introduced with leaders in the field providing in-depth articles associated with one or more of the facets: (1) defining teams; (2) how trust is established, maintained, and repaired when broken; (3) autonomous systems operating as teammates; and (4) metrics for evaluating team cognition across communication, coordination, and performance.

Exploring the impact of a fraction sense intervention in authentic school environments: An initial investigation.

A solid understanding of fractions is the cornerstone for acquiring proficiency with rational numbers and paves the way for learning advanced mathematical concepts such as algebra. Fraction difficulties limit not only students' educational and vocational opportunities but also their ability to solve everyday problems. Students who exit sixth grade with inadequate understanding of fractions may experience far-reaching repercussions that lead to lifelong avoidance of mathematics. This article presents the results of a randomized controlled trial focusing on the first two cohorts of a larger efficacy investigation aimed at building fraction sense in students with mathematics difficulties. Teachers implemented an evidence-informed fraction sense intervention (FSI) within their sixth-grade intervention classrooms. The lessons draw from research in cognitive science as well as mathematics education research. Employing random assignment at the classroom level, multilevel modeling revealed a significant effect of the intervention on posttest fractions scores after controlling for pretest fractions scores, working memory, vocabulary, proportional reasoning, and classroom attentive behavior. Students in the FSI group outperformed their counterparts in the control group, with noteworthy effect sizes on most fraction measures. Challenges associated with carrying out school-based intervention research are addressed.

Unshared Minds, Decaying Worlds: Towards a Pathology of Chronic Loneliness.

The moment when a person's actual relationships fall short of desired relationships is commonly identified as the etiological moment of chronic loneliness, which can lead to physical and psychological effects like depression, worse recovery from illness and increased mortality. But, this etiology fails to explain the nature and severe impact of loneliness. Here, we use philosophical analysis and neuroscience to show that human beings develop and maintain our world-picture (our sense of what is true, important, and good) through joint attention and action, motivated by friendship, in the Aristotelian sense of "other selves" who share a sense of the true and the good, and desire the good for each other as much as for themselves. The true etiological event of loneliness is the moment one's world-picture becomes unshared. The pathogenesis is a resultant decay of our world-picture, with brain and behavior changes following as sequelae.

Overcoming boundaries: Interdisciplinary challenges and opportunities in cognitive neuroscience.

Cognitive neuroscience has considerable untapped potential to translate our understanding of brain function into applications that maintain, restore, or enhance human cognition. Complex, real-world phenomena encountered in daily life, professional contexts, and in the arts, can also be a rich source of information for better understanding cognition, which in turn can lead to advances in knowledge and health outcomes. Interdisciplinary work is needed for these bi-directional benefits to be realized. Our cognitive neuroscience team has been collaborating on several interdisciplinary projects: hardware and software development for brain stimulation, measuring human operator state in safety-critical robotics environments, and exploring emotional regulation in actors who perform traumatic narratives. Our approach is to study research questions of mutual interest in the contexts of domain-specific applications, using (and sometimes improving) the experimental tools and techniques of cognitive neuroscience. These interdisciplinary attempts are described as case studies in the present work to illustrate non-trivial challenges that come from working across traditional disciplinary boundaries. We reflect on how obstacles to interdisciplinary work can be overcome, with the goals of enriching our understanding of human cognition and amplifying the positive effects cognitive neuroscientists have on society and innovation.

Understanding behavioral and cognitive dispositions as subsystem topologies within cognitive models: A proposal.

In our 2023 paper, entitled "Modeling interactions between the embodied and the narrative self: Dynamics of the self-pattern within LIDA," Kugele, Newen, Franklin, and I propose a functional description and implementation of a central element of Gallagher & Newen's pattern theory of self, which identifies an agent's self with a dynamic pattern of so-called cognitive aspects which govern their thought and behavior (Gallagher, 2013; Newen, 2018; Gallagher & Daly, 2018). The pattern theory explicitly rejects the traditional conceptualization of the self as a unitary entity with certain properties that resides within agents, with the idea of a pattern of aspects being central to its ability to account for the dynamic, yet relatively stable development of most natural agents' selves. Implementing the pattern theory within Learning Intelligent Distribution Agent revealed that, in order for a cognitive architecture to account for both the dynamic and stable nature of an agent's self-pattern, aspects of that pattern had to be realized by dispositions of the agent to either think or act in a certain way. In this commentary, I argue that this fundamental role of dispositions extends to cognitive processes in general and that cognitive systems should be understood in terms of the dynamical interactions of dispositions over time. In order to facilitate such an understanding, dispositions will have to be identified with topologies of cognitive (sub)systems. I provide an example of such a topology by reference to informational topologies in neuronal systems.

The challenge of transfer to clinical practice: The illusion of learning and illusion of teaching.

There is increasing pressure to accelerate health professions education programs and educators have the challenge of ensuring that students can effectively transfer their learning into clinical practice. In this personal view, we discuss how insights from cognitive science can inform the redesign of current curricula and highlight the challenge of implementing these new approaches for instructional design and assessment. We also recommend that educators disseminate the important lessons learned from their endeavors.

Large Language Models: A Historical and Sociocultural Perspective.

This letter explores the intricate historical and contemporary links between large language models (LLMs) and cognitive science through the lens of information theory, statistical language models, and socioanthropological linguistic theories. The emergence of LLMs highlights the enduring significance of information-based and statistical learning theories in understanding human communication. These theories, initially proposed in the mid-20th century, offered a visionary framework for integrating computational science, social sciences, and humanities, which nonetheless was not fully fulfilled at that time. The subsequent development of sociolinguistics and linguistic anthropology, especially since the 1970s, provided critical perspectives and empirical methods that both challenged and enriched this framework. This letter proposes that two pivotal concepts derived from this development, metapragmatic function and indexicality, offer a fruitful theoretical perspective for integrating the semantic, textual, and pragmatic, contextual dimensions of communication, an amalgamation that contemporary LLMs have yet to fully achieve. The author believes that contemporary cognitive science is at a crucial crossroads, where fostering interdisciplinary dialogues among computational linguistics, social linguistics and linguistic anthropology, and cognitive and social psychology is in particular imperative. Such collaboration is vital to bridge the computational, cognitive, and sociocultural aspects of human communication and human-AI interaction, especially in the era of large language and multimodal models and human-centric Artificial Intelligence (AI).

Predicting COVID-19 Vaccination Uptake Using a Small and Interpretable Set of Judgment and Demographic Variables: Cross-Sectional Cognitive Science Study.

BACKGROUND: Despite COVID-19 vaccine mandates, many chose to forgo vaccination, raising questions about the psychology underlying how judgment affects these choices. Research shows that reward and aversion judgments are important for vaccination choice; however, no studies have integrated such cognitive science with machine learning to predict COVID-19 vaccine uptake. OBJECTIVE: This study aims to determine the predictive power of a small but interpretable set of judgment variables using 3 machine learning algorithms to predict COVID-19 vaccine uptake and interpret what profile of judgment variables was important for prediction. METHODS: We surveyed 3476 adults across the United States in December 2021. Participants answered demographic, COVID-19 vaccine uptake (ie, whether participants were fully vaccinated), and COVID-19 precaution questions. Participants also completed a picture-rating task using images from the International Affective Picture System. Images were rated on a Likert-type scale to calibrate the degree of liking and disliking. Ratings were computationally modeled using relative preference theory to produce a set of graphs for each participant (minimum R2>0.8). In total, 15 judgment features were extracted from these graphs, 2 being analogous to risk and loss aversion from behavioral economics. These judgment variables, along with demographics, were compared between those who were fully vaccinated and those who were not. In total, 3 machine learning approaches (random forest, balanced random forest [BRF], and logistic regression) were used to test how well judgment, demographic, and COVID-19 precaution variables predicted vaccine uptake. Mediation and moderation were implemented to assess statistical mechanisms underlying successful prediction. RESULTS: Age, income, marital status, employment status, ethnicity, educational level, and sex differed by vaccine uptake (Wilcoxon rank sum and chi-square P<.001). Most judgment variables also differed by vaccine uptake (Wilcoxon rank sum P<.05). A similar area under the receiver operating characteristic curve (AUROC) was achieved by the 3 machine learning frameworks, although random forest and logistic regression produced specificities between 30% and 38% (vs 74.2% for BRF), indicating a lower performance in predicting unvaccinated participants. BRF achieved high precision (87.8%) and AUROC (79%) with moderate to high accuracy (70.8%) and balanced recall (69.6%) and specificity (74.2%). It should be noted that, for BRF, the negative predictive value was <50% despite good specificity. For BRF and random forest, 63% to 75% of the feature importance came from the 15 judgment variables. Furthermore, age, income, and educational level mediated relationships between judgment variables and vaccine uptake. CONCLUSIONS: The findings demonstrate the underlying importance of judgment variables for vaccine choice and uptake, suggesting that vaccine education and messaging might target varying judgment profiles to improve uptake. These methods could also be used to aid vaccine rollouts and health care preparedness by providing location-specific details (eg, identifying areas that may experience low vaccination and high hospitalization).

Algorithms in medical decision-making and in everyday life: what's the difference?

Algorithms are a ubiquitous part of modern life. Despite being a component of medicine since early efforts to deploy computers in medicine, clinicians' resistance to using decision support and use algorithms to address cognitive biases has been limited. This resistance is not just limited to the use of algorithmic clinical decision support, but also evidence and stochastic reasoning and the implications of the forcing function of the electronic medical record. Physician resistance to algorithmic support in clinical decision making is in stark contrast to their general acceptance of algorithmic support in other aspects of life.