Solving olympiad geometry without human demonstrations.

Proving mathematical theorems at the olympiad level represents a notable milestone in human-level automated reasoning1-4, owing to their reputed difficulty among the world's best talents in pre-university mathematics. Current machine-learning approaches, however, are not applicable to most mathematical domains owing to the high cost of translating human proofs into machine-verifiable format. The problem is even worse for geometry because of its unique translation challenges1,5, resulting in severe scarcity of training data. We propose AlphaGeometry, a theorem prover for Euclidean plane geometry that sidesteps the need for human demonstrations by synthesizing millions of theorems and proofs across different levels of complexity. AlphaGeometry is a neuro-symbolic system that uses a neural language model, trained from scratch on our large-scale synthetic data, to guide a symbolic deduction engine through infinite branching points in challenging problems. On a test set of 30 latest olympiad-level problems, AlphaGeometry solves 25, outperforming the previous best method that only solves ten problems and approaching the performance of an average International Mathematical Olympiad (IMO) gold medallist. Notably, AlphaGeometry produces human-readable proofs, solves all geometry problems in the IMO 2000 and 2015 under human expert evaluation and discovers a generalized version of a translated IMO theorem in 2004.

People systematically overlook subtractive changes.

Improving objects, ideas or situations-whether a designer seeks to advance technology, a writer seeks to strengthen an argument or a manager seeks to encourage desired behaviour-requires a mental search for possible changes1-3. We investigated whether people are as likely to consider changes that subtract components from an object, idea or situation as they are to consider changes that add new components. People typically consider a limited number of promising ideas in order to manage the cognitive burden of searching through all possible ideas, but this can lead them to accept adequate solutions without considering potentially superior alternatives4-10. Here we show that people systematically default to searching for additive transformations, and consequently overlook subtractive transformations. Across eight experiments, participants were less likely to identify advantageous subtractive changes when the task did not (versus did) cue them to consider subtraction, when they had only one opportunity (versus several) to recognize the shortcomings of an additive search strategy or when they were under a higher (versus lower) cognitive load. Defaulting to searches for additive changes may be one reason that people struggle to mitigate overburdened schedules11, institutional red tape12 and damaging effects on the planet13,14.

Evaluating language models for mathematics through interactions.

There is much excitement about the opportunity to harness the power of large language models (LLMs) when building problem-solving assistants. However, the standard methodology of evaluating LLMs relies on static pairs of inputs and outputs; this is insufficient for making an informed decision about which LLMs are best to use in an interactive setting, and how that varies by setting. Static assessment therefore limits how we understand language model capabilities. We introduce CheckMate, an adaptable prototype platform for humans to interact with and evaluate LLMs. We conduct a study with CheckMate to evaluate three language models (InstructGPT, ChatGPT, and GPT-4) as assistants in proving undergraduate-level mathematics, with a mixed cohort of participants from undergraduate students to professors of mathematics. We release the resulting interaction and rating dataset, MathConverse. By analyzing MathConverse, we derive a taxonomy of human query behaviors and uncover that despite a generally positive correlation, there are notable instances of divergence between correctness and perceived helpfulness in LLM generations, among other findings. Further, we garner a more granular understanding of GPT-4 mathematical problem-solving through a series of case studies, contributed by experienced mathematicians. We conclude with actionable takeaways for ML practitioners and mathematicians: models that communicate uncertainty, respond well to user corrections, and can provide a concise rationale for their recommendations, may constitute better assistants. Humans should inspect LLM output carefully given their current shortcomings and potential for surprising fallibility.

People see more of their biases in algorithms.

Algorithmic bias occurs when algorithms incorporate biases in the human decisions on which they are trained. We find that people see more of their biases (e.g., age, gender, race) in the decisions of algorithms than in their own decisions. Research participants saw more bias in the decisions of algorithms trained on their decisions than in their own decisions, even when those decisions were the same and participants were incentivized to reveal their true beliefs. By contrast, participants saw as much bias in the decisions of algorithms trained on their decisions as in the decisions of other participants and algorithms trained on the decisions of other participants. Cognitive psychological processes and motivated reasoning help explain why people see more of their biases in algorithms. Research participants most susceptible to bias blind spot were most likely to see more bias in algorithms than self. Participants were also more likely to perceive algorithms than themselves to have been influenced by irrelevant biasing attributes (e.g., race) but not by relevant attributes (e.g., user reviews). Because participants saw more of their biases in algorithms than themselves, they were more likely to make debiasing corrections to decisions attributed to an algorithm than to themselves. Our findings show that bias is more readily perceived in algorithms than in self and suggest how to use algorithms to reveal and correct biased human decisions.

No association between numerical ability and politically motivated reasoning in a large US probability sample.

The highly influential theory of "Motivated System 2 Reasoning" argues that analytical, deliberative ("System 2") reasoning is hijacked by identity when considering ideologically charged issues-leading people who are more likely to engage in such reasoning to be more polarized, rather than more accurate. Here, we fail to replicate the key empirical support for this theory across five contentious issues, using a large gold-standard nationally representative probability sample of Americans. While participants were more accurate in evaluating a contingency table when the outcome aligned with their politics (even when controlling for prior beliefs), we find that participants with higher numeracy were more accurate in evaluating the contingency table, regardless of whether or not the table's outcome aligned with their politics. These findings call for a reconsideration of the effect of identity on analytical reasoning.

Using cognitive psychology to understand GPT-3.

We study GPT-3, a recent large language model, using tools from cognitive psychology. More specifically, we assess GPT-3's decision-making, information search, deliberation, and causal reasoning abilities on a battery of canonical experiments from the literature. We find that much of GPT-3's behavior is impressive: It solves vignette-based tasks similarly or better than human subjects, is able to make decent decisions from descriptions, outperforms humans in a multiarmed bandit task, and shows signatures of model-based reinforcement learning. Yet, we also find that small perturbations to vignette-based tasks can lead GPT-3 vastly astray, that it shows no signatures of directed exploration, and that it fails miserably in a causal reasoning task. Taken together, these results enrich our understanding of current large language models and pave the way for future investigations using tools from cognitive psychology to study increasingly capable and opaque artificial agents.

Sharing of misinformation is habitual, not just lazy or biased.

Why do people share misinformation on social media? In this research (N = 2,476), we show that the structure of online sharing built into social platforms is more important than individual deficits in critical reasoning and partisan bias-commonly cited drivers of misinformation. Due to the reward-based learning systems on social media, users form habits of sharing information that attracts others' attention. Once habits form, information sharing is automatically activated by cues on the platform without users considering response outcomes such as spreading misinformation. As a result of user habits, 30 to 40% of the false news shared in our research was due to the 15% most habitual news sharers. Suggesting that sharing of false news is part of a broader response pattern established by social media platforms, habitual users also shared information that challenged their own political beliefs. Finally, we show that sharing of false news is not an inevitable consequence of user habits: Social media sites could be restructured to build habits to share accurate information.

Recurrent Neural Circuits Overcome Partial Inactivation by Compensation and Re-learning.

Technical advances in artificial manipulation of neural activity have precipitated a surge in studying the causal contribution of brain circuits to cognition and behavior. However, complexities of neural circuits challenge interpretation of experimental results, necessitating new theoretical frameworks for reasoning about causal effects. Here, we take a step in this direction, through the lens of recurrent neural networks trained to perform perceptual decisions. We show that understanding the dynamical system structure that underlies network solutions provides a precise account for the magnitude of behavioral effects due to perturbations. Our framework explains past empirical observations by clarifying the most sensitive features of behavior, and how complex circuits compensate and adapt to perturbations. In the process, we also identify strategies that can improve the interpretability of inactivation experiments.

TemplateFlow: FAIR-sharing of multi-scale, multi-species brain models.

Reference anatomies of the brain ('templates') and corresponding atlases are the foundation for reporting standardized neuroimaging results. Currently, there is no registry of templates and atlases; therefore, the redistribution of these resources occurs either bundled within existing software or in ad hoc ways such as downloads from institutional sites and general-purpose data repositories. We introduce TemplateFlow as a publicly available framework for human and non-human brain models. The framework combines an open database with software for access, management, and vetting, allowing scientists to share their resources under FAIR-findable, accessible, interoperable, and reusable-principles. TemplateFlow enables multifaceted insights into brains across species, and supports multiverse analyses testing whether results generalize across standard references, scales, and in the long term, species.

KRAGEN: a knowledge graph-enhanced RAG framework for biomedical problem solving using large language models.

MOTIVATION: Answering and solving complex problems using a large language model (LLM) given a certain domain such as biomedicine is a challenging task that requires both factual consistency and logic, and LLMs often suffer from some major limitations, such as hallucinating false or irrelevant information, or being influenced by noisy data. These issues can compromise the trustworthiness, accuracy, and compliance of LLM-generated text and insights. RESULTS: Knowledge Retrieval Augmented Generation ENgine (KRAGEN) is a new tool that combines knowledge graphs, Retrieval Augmented Generation (RAG), and advanced prompting techniques to solve complex problems with natural language. KRAGEN converts knowledge graphs into a vector database and uses RAG to retrieve relevant facts from it. KRAGEN uses advanced prompting techniques: namely graph-of-thoughts (GoT), to dynamically break down a complex problem into smaller subproblems, and proceeds to solve each subproblem by using the relevant knowledge through the RAG framework, which limits the hallucinations, and finally, consolidates the subproblems and provides a solution. KRAGEN's graph visualization allows the user to interact with and evaluate the quality of the solution's GoT structure and logic. AVAILABILITY AND IMPLEMENTATION: KRAGEN is deployed by running its custom Docker containers. KRAGEN is available as open-source from GitHub at: https://github.com/EpistasisLab/KRAGEN.

Cross-frequency and inter-regional phase synchronization in explicit transitive inference.

Explicit logical reasoning, like transitive inference, is a hallmark of human intelligence. This study investigated cortical oscillations and their interactions in transitive inference with EEG. Participants viewed premises describing abstract relations among items. They accurately recalled the relationship between old pairs of items, effectively inferred the relationship between new pairs of items, and discriminated between true and false relationships for new pairs. First, theta (4-7 Hz) and alpha oscillations (8-15 Hz) had distinct functional roles. Frontal theta oscillations distinguished between new and old pairs, reflecting the inference of new information. Parietal alpha oscillations changed with serial position and symbolic distance of the pairs, representing the underlying relational structure. Frontal alpha oscillations distinguished between true and false pairs, linking the new information with the underlying relational structure. Second, theta and alpha oscillations interacted through cross-frequency and inter-regional phase synchronization. Frontal theta-alpha 1:2 phase locking appeared to coordinate spectrally diverse neural activity, enhanced for new versus old pairs and true versus false pairs. Alpha-band frontal-parietal phase coherence appeared to coordinate anatomically distributed neural activity, enhanced for new versus old pairs and false versus true pairs. It suggests that cross-frequency and inter-regional phase synchronization among theta and alpha oscillations supports human transitive inference.

Math on cortex-enhanced delta phase synchrony in math experts during long and complex math demonstrations.

Neural oscillations are important for working memory and reasoning and they are modulated during cognitively challenging tasks, like mathematics. Previous work has examined local cortical synchrony on theta (4-8 Hz) and alpha (8-13 Hz) bands over frontal and parietal electrodes during short mathematical tasks when sitting. However, it is unknown whether processing of long and complex math stimuli evokes inter-regional functional connectivity. We recorded cortical activity with EEG while math experts and novices watched long (13-68 seconds) and complex (bachelor-level) math demonstrations when sitting and standing. Fronto-parietal connectivity over the left hemisphere was stronger in math experts than novices reflected by enhanced delta (0.5-4 Hz) phase synchrony in experts. Processing of complex math tasks when standing extended the difference to right hemisphere, suggesting that other cognitive processes, such as maintenance of body balance when standing, may interfere with novice's internal concentration required during complex math tasks more than in experts. There were no groups differences in phase synchrony over theta or alpha frequencies. These results suggest that low-frequency oscillations modulate inter-regional connectivity during long and complex mathematical cognition and demonstrate one way in which the brain functions of math experts differ from those of novices: through enhanced fronto-parietal functional connectivity.

Inferior parietal cortex represents relational structures for explicit transitive inference.

The human brain is distinguished by its ability to perform explicit logical reasoning like transitive inference. This study investigated the functional role of the inferior parietal cortex in transitive inference with functional MRI. Participants viewed premises describing abstract relations among items. They accurately recalled the relationship between old pairs of items, effectively inferred the relationship between new pairs of items, and discriminated between true and false relationships for new pairs. First, the inferior parietal cortex, but not the hippocampus or lateral prefrontal cortex, was associated with transitive inference. The inferior parietal activity and functional connectivity were modulated by inference (new versus old pairs) and discrimination (true versus false pairs). Moreover, the new/old and true/false pairs were decodable from the inferior parietal representation. Second, the inferior parietal cortex represented an integrated relational structure (ordered and directed series). The inferior parietal activity was modulated by serial position (larger end versus center pairs). The inferior parietal representation was modulated by symbolic distance (adjacent versus distant pairs) and direction (preceding versus following pairs). It suggests that the inferior parietal cortex may flexibly integrate observed relations into a relational structure and use the relational structure to infer unobserved relations and discriminate between true and false relations.

A theory of consciousness from a theoretical computer science perspective: Insights from the Conscious Turing Machine.

This paper examines consciousness from the perspective of theoretical computer science (TCS), a branch of mathematics concerned with understanding the underlying principles of computation and complexity, including the implications and surprising consequences of resource limitations. We propose a formal TCS model, the Conscious Turing Machine (CTM). The CTM is influenced by Alan Turing's simple yet powerful model of computation, the Turing machine (TM), and by the global workspace theory (GWT) of consciousness originated by cognitive neuroscientist Bernard Baars and further developed by him, Stanislas Dehaene, Jean-Pierre Changeux, George Mashour, and others. Phenomena generally associated with consciousness, such as blindsight, inattentional blindness, change blindness, dream creation, and free will, are considered. Explanations derived from the model draw confirmation from consistencies at a high level, well above the level of neurons, with the cognitive neuroscience literature.

A neural network solves, explains, and generates university math problems by program synthesis and few-shot learning at human level.

We demonstrate that a neural network pretrained on text and fine-tuned on code solves mathematics course problems, explains solutions, and generates questions at a human level. We automatically synthesize programs using few-shot learning and OpenAI's Codex transformer and execute them to solve course problems at 81% automatic accuracy. We curate a dataset of questions from Massachusetts Institute of Technology (MIT)'s largest mathematics courses (Single Variable and Multivariable Calculus, Differential Equations, Introduction to Probability and Statistics, Linear Algebra, and Mathematics for Computer Science) and Columbia University's Computational Linear Algebra. We solve questions from a MATH dataset (on Prealgebra, Algebra, Counting and Probability, Intermediate Algebra, Number Theory, and Precalculus), the latest benchmark of advanced mathematics problems designed to assess mathematical reasoning. We randomly sample questions and generate solutions with multiple modalities, including numbers, equations, and plots. The latest GPT-3 language model pretrained on text automatically solves only 18.8% of these university questions using zero-shot learning and 30.8% using few-shot learning and the most recent chain of thought prompting. In contrast, program synthesis with few-shot learning using Codex fine-tuned on code generates programs that automatically solve 81% of these questions. Our approach improves the previous state-of-the-art automatic solution accuracy on the benchmark topics from 8.8 to 81.1%. We perform a survey to evaluate the quality and difficulty of generated questions. This work automatically solves university-level mathematics course questions at a human level and explains and generates university-level mathematics course questions at scale, a milestone for higher education.

Autism attenuates the perception of the mind-body divide.

People are intuitive Dualists-they tacitly consider the mind as ethereal, distinct from the body. Here we ask whether Dualism emerges naturally from the conflicting core principles that guide reasoning about objects, on the one hand, and about the minds of agents (theory of mind, ToM), on the other. To address this question, we explore Dualist reasoning in autism spectrum disorder (ASD)-a congenital disorder known to compromise ToM. If Dualism arises from ToM, then ASD ought to attenuate Dualism and promote Physicalism. In line with this prediction, Experiment 1 shows that, compared to controls, people with ASD are more likely to view psychological traits as embodied-as likely to manifest in a replica of one's body. Experiment 2 demonstrates that, unlike controls, people with ASD do not consider thoughts as disembodied-as persistent in the afterlife (upon the body's demise). If ASD promotes the perception of the psyche as embodied, and if (per Essentialism) embodiment suggests innateness, then ASD should further promote Nativism-this bias is shown in Experiment 3. Finally, Experiment 4 demonstrates that, in neurotypical (NT) participants, difficulties with ToM correlate with Physicalism. These results are the first to show that ASD attenuates Dualist reasoning and to link Dualism to ToM. These conclusions suggest that the mind-body distinction might be natural for people to entertain.

A reinforcement-based mechanism for discontinuous learning.

Problem-solving and reasoning involve mental exploration and navigation in sparse relational spaces. A physical analogue is spatial navigation in structured environments such as a network of burrows. Recent experiments with mice navigating a labyrinth show a sharp discontinuity during learning, corresponding to a distinct moment of "sudden insight" when mice figure out long, direct paths to the goal. This discontinuity is seemingly at odds with reinforcement learning (RL), which involves a gradual build-up of a value signal during learning. Here, we show that biologically plausible RL rules combined with persistent exploration generically exhibit discontinuous learning. In tree-like structured environments, positive feedback from learning on behavior generates a "reinforcement wave" with a steep profile. The discontinuity occurs when the wave reaches the starting point. By examining the nonlinear dynamics of reinforcement propagation, we establish a quantitative relationship between the learning rule, the agent's exploration biases, and learning speed. Predictions explain existing data and motivate specific experiments to isolate the phenomenon. Additionally, we characterize the exact learning dynamics of various RL rules for a complex sequential task.

Individuals prefer to harm their own group rather than help an opposing group.

Group-based conflict enacts a severe toll on society, yet the psychological factors governing behavior in group conflicts remain unclear. Past work finds that group members seek to maximize relative differences between their in-group and out-group ("in-group favoritism") and are driven by a desire to benefit in-groups rather than harm out-groups (the "in-group love" hypothesis). This prior research studies how decision-makers approach trade-offs between two net-positive outcomes for their in-group. However, in the real world, group members often face trade-offs between net-negative options, entailing either losses to their group or gains for the opposition. Anecdotally, under such conditions, individuals may avoid supporting their opponents even if this harms their own group, seemingly inconsistent with "in-group love" or a harm minimizing strategy. Yet, to the best of our knowledge, these circumstances have not been investigated. In six pre-registered studies, we find consistent evidence that individuals prefer to harm their own group rather than provide even minimal support to an opposing group across polarized issues (abortion access, political party, gun rights). Strikingly, in an incentive-compatible experiment, individuals preferred to subtract more than three times as much from their own group rather than support an opposing group, despite believing that their in-group is more effective with funds. We find that identity concerns drive preferences in group decision-making, and individuals believe that supporting an opposing group is less value-compatible than harming their own group. Our results hold valuable insights for the psychology of decision-making in intergroup conflict as well as potential interventions for conflict resolution.

Development of Human Lateral Prefrontal Sulcal Morphology and Its Relation to Reasoning Performance.

Previous findings show that the morphology of folds (sulci) of the human cerebral cortex flatten during postnatal development. However, previous studies did not consider the relationship between sulcal morphology and cognitive development in individual participants. Here, we fill this gap in knowledge by leveraging cross-sectional morphologic neuroimaging data in the lateral PFC (LPFC) from individual human participants (6-36 years old, males and females; N = 108; 3672 sulci), as well as longitudinal morphologic and behavioral data from a subset of child and adolescent participants scanned at two time points (6-18 years old; N = 44; 2992 sulci). Manually defining thousands of sulci revealed that LPFC sulcal morphology (depth, surface area, and gray matter thickness) differed between children (6-11 years old)/adolescents (11-18 years old) and young adults (22-36 years old) cross-sectionally, but only cortical thickness showed differences across childhood and adolescence and presented longitudinal changes during childhood and adolescence. Furthermore, a data-driven approach relating morphology and cognition identified that longitudinal changes in cortical thickness of four left-hemisphere LPFC sulci predicted longitudinal changes in reasoning performance, a higher-level cognitive ability that relies on LPFC. Contrary to previous findings, these results suggest that sulci may flatten either after this time frame or over a longer longitudinal period of time than previously presented. Crucially, these results also suggest that longitudinal changes in the cortex within specific LPFC sulci are behaviorally meaningful, providing targeted structures, and areas of the cortex, for future neuroimaging studies examining the development of cognitive abilities.SIGNIFICANCE STATEMENT Recent work has shown that individual differences in neuroanatomical structures (indentations, or sulci) within the lateral PFC are behaviorally meaningful during childhood and adolescence. Here, we describe how specific lateral PFC sulci develop at the level of individual participants for the first time: from both cross-sectional and longitudinal perspectives. Further, we show, also for the first time, that the longitudinal morphologic changes in these structures are behaviorally relevant. These findings lay the foundation for a future avenue to precisely study the development of the cortex and highlight the importance of studying the development of sulci in other cortical expanses and charting how these changes relate to the cognitive abilities those areas support at the level of individual participants.

Neural Contributions to Reduced Fluid Intelligence across the Adult Lifespan.

Fluid intelligence, the ability to solve novel, complex problems, declines steeply during healthy human aging. Using fMRI, fluid intelligence has been repeatedly associated with activation of a frontoparietal brain network, and impairment following focal damage to these regions suggests that fluid intelligence depends on their integrity. It is therefore possible that age-related functional differences in frontoparietal activity contribute to the reduction in fluid intelligence. This paper reports on analysis of the Cambridge Center for Ageing and Neuroscience data, a large, population-based cohort of healthy males and females across the adult lifespan. The data support a model in which age-related differences in fluid intelligence are partially mediated by the responsiveness of frontoparietal regions to novel problem-solving. We first replicate a prior finding of such mediation using an independent sample. We then precisely localize the mediating brain regions, and show that mediation is specifically associated with voxels most activated by cognitive demand, but not with voxels suppressed by cognitive demand. We quantify the robustness of this result to potential unmodeled confounders, and estimate the causal direction of the effects. Finally, exploratory analyses suggest that neural mediation of age-related differences in fluid intelligence is moderated by the variety of regular physical activities, more reliably than by their frequency or duration. An additional moderating role of the variety of nonphysical activities emerged when controlling for head motion. A better understanding of the mechanisms that link healthy aging with lower fluid intelligence may suggest strategies for mitigating such decline.SIGNIFICANCE STATEMENT Global populations are living longer, driving urgency to understand age-related cognitive declines. Fluid intelligence is of prime importance because it reflects performance across many domains, and declines especially steeply during healthy aging. Despite consensus that fluid intelligence is associated with particular frontoparietal brain regions, little research has investigated suggestions that under-responsiveness of these regions mediates age-related decline. We replicate a recent demonstration of such mediation, showing specific association with brain regions most activated by cognitive demand, and robustness to moderate confounding by unmodeled variables. By showing that this mediation model is moderated by the variety of regular physical activities, more reliably than by their frequency or duration, we identify a potential modifiable lifestyle factor that may help promote successful aging.