Solving olympiad geometry without human demonstrations.

Proving mathematical theorems at the olympiad level represents a notable milestone in human-level automated reasoning1-4, owing to their reputed difficulty among the world's best talents in pre-university mathematics. Current machine-learning approaches, however, are not applicable to most mathematical domains owing to the high cost of translating human proofs into machine-verifiable format. The problem is even worse for geometry because of its unique translation challenges1,5, resulting in severe scarcity of training data. We propose AlphaGeometry, a theorem prover for Euclidean plane geometry that sidesteps the need for human demonstrations by synthesizing millions of theorems and proofs across different levels of complexity. AlphaGeometry is a neuro-symbolic system that uses a neural language model, trained from scratch on our large-scale synthetic data, to guide a symbolic deduction engine through infinite branching points in challenging problems. On a test set of 30 latest olympiad-level problems, AlphaGeometry solves 25, outperforming the previous best method that only solves ten problems and approaching the performance of an average International Mathematical Olympiad (IMO) gold medallist. Notably, AlphaGeometry produces human-readable proofs, solves all geometry problems in the IMO 2000 and 2015 under human expert evaluation and discovers a generalized version of a translated IMO theorem in 2004.

People systematically overlook subtractive changes.

Improving objects, ideas or situations-whether a designer seeks to advance technology, a writer seeks to strengthen an argument or a manager seeks to encourage desired behaviour-requires a mental search for possible changes1-3. We investigated whether people are as likely to consider changes that subtract components from an object, idea or situation as they are to consider changes that add new components. People typically consider a limited number of promising ideas in order to manage the cognitive burden of searching through all possible ideas, but this can lead them to accept adequate solutions without considering potentially superior alternatives4-10. Here we show that people systematically default to searching for additive transformations, and consequently overlook subtractive transformations. Across eight experiments, participants were less likely to identify advantageous subtractive changes when the task did not (versus did) cue them to consider subtraction, when they had only one opportunity (versus several) to recognize the shortcomings of an additive search strategy or when they were under a higher (versus lower) cognitive load. Defaulting to searches for additive changes may be one reason that people struggle to mitigate overburdened schedules11, institutional red tape12 and damaging effects on the planet13,14.

Evaluating language models for mathematics through interactions.

There is much excitement about the opportunity to harness the power of large language models (LLMs) when building problem-solving assistants. However, the standard methodology of evaluating LLMs relies on static pairs of inputs and outputs; this is insufficient for making an informed decision about which LLMs are best to use in an interactive setting, and how that varies by setting. Static assessment therefore limits how we understand language model capabilities. We introduce CheckMate, an adaptable prototype platform for humans to interact with and evaluate LLMs. We conduct a study with CheckMate to evaluate three language models (InstructGPT, ChatGPT, and GPT-4) as assistants in proving undergraduate-level mathematics, with a mixed cohort of participants from undergraduate students to professors of mathematics. We release the resulting interaction and rating dataset, MathConverse. By analyzing MathConverse, we derive a taxonomy of human query behaviors and uncover that despite a generally positive correlation, there are notable instances of divergence between correctness and perceived helpfulness in LLM generations, among other findings. Further, we garner a more granular understanding of GPT-4 mathematical problem-solving through a series of case studies, contributed by experienced mathematicians. We conclude with actionable takeaways for ML practitioners and mathematicians: models that communicate uncertainty, respond well to user corrections, and can provide a concise rationale for their recommendations, may constitute better assistants. Humans should inspect LLM output carefully given their current shortcomings and potential for surprising fallibility.

People see more of their biases in algorithms.

Algorithmic bias occurs when algorithms incorporate biases in the human decisions on which they are trained. We find that people see more of their biases (e.g., age, gender, race) in the decisions of algorithms than in their own decisions. Research participants saw more bias in the decisions of algorithms trained on their decisions than in their own decisions, even when those decisions were the same and participants were incentivized to reveal their true beliefs. By contrast, participants saw as much bias in the decisions of algorithms trained on their decisions as in the decisions of other participants and algorithms trained on the decisions of other participants. Cognitive psychological processes and motivated reasoning help explain why people see more of their biases in algorithms. Research participants most susceptible to bias blind spot were most likely to see more bias in algorithms than self. Participants were also more likely to perceive algorithms than themselves to have been influenced by irrelevant biasing attributes (e.g., race) but not by relevant attributes (e.g., user reviews). Because participants saw more of their biases in algorithms than themselves, they were more likely to make debiasing corrections to decisions attributed to an algorithm than to themselves. Our findings show that bias is more readily perceived in algorithms than in self and suggest how to use algorithms to reveal and correct biased human decisions.

Multicellular artificial neural network-type architectures demonstrate computational problem solving.

Here, we report a modular multicellular system created by mixing and matching discrete engineered bacterial cells. This system can be designed to solve multiple computational decision problems. The modular system is based on a set of engineered bacteria that are modeled as an 'artificial neurosynapse' that, in a coculture, formed a single-layer artificial neural network-type architecture that can perform computational tasks. As a demonstration, we constructed devices that function as a full subtractor and a full adder. The system is also capable of solving problems such as determining if a number between 0 and 9 is a prime number and if a letter between A and L is a vowel. Finally, we built a system that determines the maximum number of pieces of a pie that can be made for a given number of straight cuts. This work may have importance in biocomputer technology development and multicellular synthetic biology.

No association between numerical ability and politically motivated reasoning in a large US probability sample.

The highly influential theory of "Motivated System 2 Reasoning" argues that analytical, deliberative ("System 2") reasoning is hijacked by identity when considering ideologically charged issues-leading people who are more likely to engage in such reasoning to be more polarized, rather than more accurate. Here, we fail to replicate the key empirical support for this theory across five contentious issues, using a large gold-standard nationally representative probability sample of Americans. While participants were more accurate in evaluating a contingency table when the outcome aligned with their politics (even when controlling for prior beliefs), we find that participants with higher numeracy were more accurate in evaluating the contingency table, regardless of whether or not the table's outcome aligned with their politics. These findings call for a reconsideration of the effect of identity on analytical reasoning.

Using cognitive psychology to understand GPT-3.

We study GPT-3, a recent large language model, using tools from cognitive psychology. More specifically, we assess GPT-3's decision-making, information search, deliberation, and causal reasoning abilities on a battery of canonical experiments from the literature. We find that much of GPT-3's behavior is impressive: It solves vignette-based tasks similarly or better than human subjects, is able to make decent decisions from descriptions, outperforms humans in a multiarmed bandit task, and shows signatures of model-based reinforcement learning. Yet, we also find that small perturbations to vignette-based tasks can lead GPT-3 vastly astray, that it shows no signatures of directed exploration, and that it fails miserably in a causal reasoning task. Taken together, these results enrich our understanding of current large language models and pave the way for future investigations using tools from cognitive psychology to study increasingly capable and opaque artificial agents.

Sharing of misinformation is habitual, not just lazy or biased.

Why do people share misinformation on social media? In this research (N = 2,476), we show that the structure of online sharing built into social platforms is more important than individual deficits in critical reasoning and partisan bias-commonly cited drivers of misinformation. Due to the reward-based learning systems on social media, users form habits of sharing information that attracts others' attention. Once habits form, information sharing is automatically activated by cues on the platform without users considering response outcomes such as spreading misinformation. As a result of user habits, 30 to 40% of the false news shared in our research was due to the 15% most habitual news sharers. Suggesting that sharing of false news is part of a broader response pattern established by social media platforms, habitual users also shared information that challenged their own political beliefs. Finally, we show that sharing of false news is not an inevitable consequence of user habits: Social media sites could be restructured to build habits to share accurate information.

Recurrent Neural Circuits Overcome Partial Inactivation by Compensation and Re-learning.

Technical advances in artificial manipulation of neural activity have precipitated a surge in studying the causal contribution of brain circuits to cognition and behavior. However, complexities of neural circuits challenge interpretation of experimental results, necessitating new theoretical frameworks for reasoning about causal effects. Here, we take a step in this direction, through the lens of recurrent neural networks trained to perform perceptual decisions. We show that understanding the dynamical system structure that underlies network solutions provides a precise account for the magnitude of behavioral effects due to perturbations. Our framework explains past empirical observations by clarifying the most sensitive features of behavior, and how complex circuits compensate and adapt to perturbations. In the process, we also identify strategies that can improve the interpretability of inactivation experiments.

TemplateFlow: FAIR-sharing of multi-scale, multi-species brain models.

Reference anatomies of the brain ('templates') and corresponding atlases are the foundation for reporting standardized neuroimaging results. Currently, there is no registry of templates and atlases; therefore, the redistribution of these resources occurs either bundled within existing software or in ad hoc ways such as downloads from institutional sites and general-purpose data repositories. We introduce TemplateFlow as a publicly available framework for human and non-human brain models. The framework combines an open database with software for access, management, and vetting, allowing scientists to share their resources under FAIR-findable, accessible, interoperable, and reusable-principles. TemplateFlow enables multifaceted insights into brains across species, and supports multiverse analyses testing whether results generalize across standard references, scales, and in the long term, species.

KRAGEN: a knowledge graph-enhanced RAG framework for biomedical problem solving using large language models.

MOTIVATION: Answering and solving complex problems using a large language model (LLM) given a certain domain such as biomedicine is a challenging task that requires both factual consistency and logic, and LLMs often suffer from some major limitations, such as hallucinating false or irrelevant information, or being influenced by noisy data. These issues can compromise the trustworthiness, accuracy, and compliance of LLM-generated text and insights. RESULTS: Knowledge Retrieval Augmented Generation ENgine (KRAGEN) is a new tool that combines knowledge graphs, Retrieval Augmented Generation (RAG), and advanced prompting techniques to solve complex problems with natural language. KRAGEN converts knowledge graphs into a vector database and uses RAG to retrieve relevant facts from it. KRAGEN uses advanced prompting techniques: namely graph-of-thoughts (GoT), to dynamically break down a complex problem into smaller subproblems, and proceeds to solve each subproblem by using the relevant knowledge through the RAG framework, which limits the hallucinations, and finally, consolidates the subproblems and provides a solution. KRAGEN's graph visualization allows the user to interact with and evaluate the quality of the solution's GoT structure and logic. AVAILABILITY AND IMPLEMENTATION: KRAGEN is deployed by running its custom Docker containers. KRAGEN is available as open-source from GitHub at: https://github.com/EpistasisLab/KRAGEN.

The neural dynamics associated with computational complexity.

Many everyday tasks require people to solve computationally complex problems. However, little is known about the effects of computational hardness on the neural processes associated with solving such problems. Here, we draw on computational complexity theory to address this issue. We performed an experiment in which participants solved several instances of the 0-1 knapsack problem, a combinatorial optimization problem, while undergoing ultra-high field (7T) functional magnetic resonance imaging (fMRI). Instances varied in computational hardness. We characterize a network of brain regions whose activation was correlated with computational complexity, including the anterior insula, dorsal anterior cingulate cortex and the intra-parietal sulcus/angular gyrus. Activation and connectivity changed dynamically as a function of complexity, in line with theoretical computational requirements. Overall, our results suggest that computational complexity theory provides a suitable framework to study the effects of computational hardness on the neural processes associated with solving complex cognitive tasks.

Cross-frequency and inter-regional phase synchronization in explicit transitive inference.

Explicit logical reasoning, like transitive inference, is a hallmark of human intelligence. This study investigated cortical oscillations and their interactions in transitive inference with EEG. Participants viewed premises describing abstract relations among items. They accurately recalled the relationship between old pairs of items, effectively inferred the relationship between new pairs of items, and discriminated between true and false relationships for new pairs. First, theta (4-7 Hz) and alpha oscillations (8-15 Hz) had distinct functional roles. Frontal theta oscillations distinguished between new and old pairs, reflecting the inference of new information. Parietal alpha oscillations changed with serial position and symbolic distance of the pairs, representing the underlying relational structure. Frontal alpha oscillations distinguished between true and false pairs, linking the new information with the underlying relational structure. Second, theta and alpha oscillations interacted through cross-frequency and inter-regional phase synchronization. Frontal theta-alpha 1:2 phase locking appeared to coordinate spectrally diverse neural activity, enhanced for new versus old pairs and true versus false pairs. Alpha-band frontal-parietal phase coherence appeared to coordinate anatomically distributed neural activity, enhanced for new versus old pairs and false versus true pairs. It suggests that cross-frequency and inter-regional phase synchronization among theta and alpha oscillations supports human transitive inference.

Math on cortex-enhanced delta phase synchrony in math experts during long and complex math demonstrations.

Neural oscillations are important for working memory and reasoning and they are modulated during cognitively challenging tasks, like mathematics. Previous work has examined local cortical synchrony on theta (4-8 Hz) and alpha (8-13 Hz) bands over frontal and parietal electrodes during short mathematical tasks when sitting. However, it is unknown whether processing of long and complex math stimuli evokes inter-regional functional connectivity. We recorded cortical activity with EEG while math experts and novices watched long (13-68 seconds) and complex (bachelor-level) math demonstrations when sitting and standing. Fronto-parietal connectivity over the left hemisphere was stronger in math experts than novices reflected by enhanced delta (0.5-4 Hz) phase synchrony in experts. Processing of complex math tasks when standing extended the difference to right hemisphere, suggesting that other cognitive processes, such as maintenance of body balance when standing, may interfere with novice's internal concentration required during complex math tasks more than in experts. There were no groups differences in phase synchrony over theta or alpha frequencies. These results suggest that low-frequency oscillations modulate inter-regional connectivity during long and complex mathematical cognition and demonstrate one way in which the brain functions of math experts differ from those of novices: through enhanced fronto-parietal functional connectivity.

Inferior parietal cortex represents relational structures for explicit transitive inference.

The human brain is distinguished by its ability to perform explicit logical reasoning like transitive inference. This study investigated the functional role of the inferior parietal cortex in transitive inference with functional MRI. Participants viewed premises describing abstract relations among items. They accurately recalled the relationship between old pairs of items, effectively inferred the relationship between new pairs of items, and discriminated between true and false relationships for new pairs. First, the inferior parietal cortex, but not the hippocampus or lateral prefrontal cortex, was associated with transitive inference. The inferior parietal activity and functional connectivity were modulated by inference (new versus old pairs) and discrimination (true versus false pairs). Moreover, the new/old and true/false pairs were decodable from the inferior parietal representation. Second, the inferior parietal cortex represented an integrated relational structure (ordered and directed series). The inferior parietal activity was modulated by serial position (larger end versus center pairs). The inferior parietal representation was modulated by symbolic distance (adjacent versus distant pairs) and direction (preceding versus following pairs). It suggests that the inferior parietal cortex may flexibly integrate observed relations into a relational structure and use the relational structure to infer unobserved relations and discriminate between true and false relations.

A theory of consciousness from a theoretical computer science perspective: Insights from the Conscious Turing Machine.

This paper examines consciousness from the perspective of theoretical computer science (TCS), a branch of mathematics concerned with understanding the underlying principles of computation and complexity, including the implications and surprising consequences of resource limitations. We propose a formal TCS model, the Conscious Turing Machine (CTM). The CTM is influenced by Alan Turing's simple yet powerful model of computation, the Turing machine (TM), and by the global workspace theory (GWT) of consciousness originated by cognitive neuroscientist Bernard Baars and further developed by him, Stanislas Dehaene, Jean-Pierre Changeux, George Mashour, and others. Phenomena generally associated with consciousness, such as blindsight, inattentional blindness, change blindness, dream creation, and free will, are considered. Explanations derived from the model draw confirmation from consistencies at a high level, well above the level of neurons, with the cognitive neuroscience literature.

A neural network solves, explains, and generates university math problems by program synthesis and few-shot learning at human level.

We demonstrate that a neural network pretrained on text and fine-tuned on code solves mathematics course problems, explains solutions, and generates questions at a human level. We automatically synthesize programs using few-shot learning and OpenAI's Codex transformer and execute them to solve course problems at 81% automatic accuracy. We curate a dataset of questions from Massachusetts Institute of Technology (MIT)'s largest mathematics courses (Single Variable and Multivariable Calculus, Differential Equations, Introduction to Probability and Statistics, Linear Algebra, and Mathematics for Computer Science) and Columbia University's Computational Linear Algebra. We solve questions from a MATH dataset (on Prealgebra, Algebra, Counting and Probability, Intermediate Algebra, Number Theory, and Precalculus), the latest benchmark of advanced mathematics problems designed to assess mathematical reasoning. We randomly sample questions and generate solutions with multiple modalities, including numbers, equations, and plots. The latest GPT-3 language model pretrained on text automatically solves only 18.8% of these university questions using zero-shot learning and 30.8% using few-shot learning and the most recent chain of thought prompting. In contrast, program synthesis with few-shot learning using Codex fine-tuned on code generates programs that automatically solve 81% of these questions. Our approach improves the previous state-of-the-art automatic solution accuracy on the benchmark topics from 8.8 to 81.1%. We perform a survey to evaluate the quality and difficulty of generated questions. This work automatically solves university-level mathematics course questions at a human level and explains and generates university-level mathematics course questions at scale, a milestone for higher education.

Autism attenuates the perception of the mind-body divide.

People are intuitive Dualists-they tacitly consider the mind as ethereal, distinct from the body. Here we ask whether Dualism emerges naturally from the conflicting core principles that guide reasoning about objects, on the one hand, and about the minds of agents (theory of mind, ToM), on the other. To address this question, we explore Dualist reasoning in autism spectrum disorder (ASD)-a congenital disorder known to compromise ToM. If Dualism arises from ToM, then ASD ought to attenuate Dualism and promote Physicalism. In line with this prediction, Experiment 1 shows that, compared to controls, people with ASD are more likely to view psychological traits as embodied-as likely to manifest in a replica of one's body. Experiment 2 demonstrates that, unlike controls, people with ASD do not consider thoughts as disembodied-as persistent in the afterlife (upon the body's demise). If ASD promotes the perception of the psyche as embodied, and if (per Essentialism) embodiment suggests innateness, then ASD should further promote Nativism-this bias is shown in Experiment 3. Finally, Experiment 4 demonstrates that, in neurotypical (NT) participants, difficulties with ToM correlate with Physicalism. These results are the first to show that ASD attenuates Dualist reasoning and to link Dualism to ToM. These conclusions suggest that the mind-body distinction might be natural for people to entertain.

A reinforcement-based mechanism for discontinuous learning.

Problem-solving and reasoning involve mental exploration and navigation in sparse relational spaces. A physical analogue is spatial navigation in structured environments such as a network of burrows. Recent experiments with mice navigating a labyrinth show a sharp discontinuity during learning, corresponding to a distinct moment of "sudden insight" when mice figure out long, direct paths to the goal. This discontinuity is seemingly at odds with reinforcement learning (RL), which involves a gradual build-up of a value signal during learning. Here, we show that biologically plausible RL rules combined with persistent exploration generically exhibit discontinuous learning. In tree-like structured environments, positive feedback from learning on behavior generates a "reinforcement wave" with a steep profile. The discontinuity occurs when the wave reaches the starting point. By examining the nonlinear dynamics of reinforcement propagation, we establish a quantitative relationship between the learning rule, the agent's exploration biases, and learning speed. Predictions explain existing data and motivate specific experiments to isolate the phenomenon. Additionally, we characterize the exact learning dynamics of various RL rules for a complex sequential task.