Advancing mathematics by guiding human intuition with AI.

The practice of mathematics involves discovering patterns and using these to formulate and prove conjectures, resulting in theorems. Since the 1960s, mathematicians have used computers to assist in the discovery of patterns and formulation of conjectures1, most famously in the Birch and Swinnerton-Dyer conjecture2, a Millennium Prize Problem3. Here we provide examples of new fundamental results in pure mathematics that have been discovered with the assistance of machine learning-demonstrating a method by which machine learning can aid mathematicians in discovering new conjectures and theorems. We propose a process of using machine learning to discover potential patterns and relations between mathematical objects, understanding them with attribution techniques and using these observations to guide intuition and propose conjectures. We outline this machine-learning-guided framework and demonstrate its successful application to current research questions in distinct areas of pure mathematics, in each case showing how it led to meaningful mathematical contributions on important open problems: a new connection between the algebraic and geometric structure of knots, and a candidate algorithm predicted by the combinatorial invariance conjecture for symmetric groups4. Our work may serve as a model for collaboration between the fields of mathematics and artificial intelligence (AI) that can achieve surprising results by leveraging the respective strengths of mathematicians and machine learning.

Polymorph Identification for Flexible Molecules: Linear Regression Analysis of Experimental and Calculated Solution- and Solid-State NMR Data.

The Δδ regression approach of Blade et al. [ J. Phys. Chem. A 2020, 124(43), 8959-8977] for accurately discriminating between solid forms using a combination of experimental solution- and solid-state NMR data with density functional theory (DFT) calculation is here extended to molecules with multiple conformational degrees of freedom, using furosemide polymorphs as an exemplar. As before, the differences in measured 1H and 13C chemical shifts between solution-state NMR and solid-state magic-angle spinning (MAS) NMR (Δδexperimental) are compared to those determined by gauge-including projector augmented wave (GIPAW) calculations (Δδcalculated) by regression analysis and a t-test, allowing the correct furosemide polymorph to be precisely identified. Monte Carlo random sampling is used to calculate solution-state NMR chemical shifts, reducing computation times by avoiding the need to systematically sample the multidimensional conformational landscape that furosemide occupies in solution. The solvent conditions should be chosen to match the molecule's charge state between the solution and solid states. The Δδ regression approach indicates whether or not correlations between Δδexperimental and Δδcalculated are statistically significant; the approach is differently sensitive to the popular root mean squared error (RMSE) method, being shown to exhibit a much greater dynamic range. An alternative method for estimating solution-state NMR chemical shifts by approximating the measured solution-state dynamic 3D behavior with an ensemble of 54 furosemide crystal structures (polymorphs and cocrystals) from the Cambridge Structural Database (CSD) was also successful in this case, suggesting new avenues for this method that may overcome its current dependency on the prior determination of solution dynamic 3D structures.

Vector-based navigation using grid-like representations in artificial agents.

Deep neural networks have achieved impressive successes in fields ranging from object recognition to complex games such as Go1,2. Navigation, however, remains a substantial challenge for artificial agents, with deep neural networks trained by reinforcement learning3-5 failing to rival the proficiency of mammalian spatial behaviour, which is underpinned by grid cells in the entorhinal cortex 6 . Grid cells are thought to provide a multi-scale periodic representation that functions as a metric for coding space7,8 and is critical for integrating self-motion (path integration)6,7,9 and planning direct trajectories to goals (vector-based navigation)7,10,11. Here we set out to leverage the computational functions of grid cells to develop a deep reinforcement learning agent with mammal-like navigational abilities. We first trained a recurrent network to perform path integration, leading to the emergence of representations resembling grid cells, as well as other entorhinal cell types 12 . We then showed that this representation provided an effective basis for an agent to locate goals in challenging, unfamiliar, and changeable environments-optimizing the primary objective of navigation through deep reinforcement learning. The performance of agents endowed with grid-like representations surpassed that of an expert human and comparison agents, with the metric quantities necessary for vector-based navigation derived from grid-like units within the network. Furthermore, grid-like representations enabled agents to conduct shortcut behaviours reminiscent of those performed by mammals. Our findings show that emergent grid-like representations furnish agents with a Euclidean spatial metric and associated vector operations, providing a foundation for proficient navigation. As such, our results support neuroscientific theories that see grid cells as critical for vector-based navigation7,10,11, demonstrating that the latter can be combined with path-based strategies to support navigation in challenging environments.

Conformations in Solution and in Solid-State Polymorphs: Correlating Experimental and Calculated Nuclear Magnetic Resonance Chemical Shifts for Tolfenamic Acid.

A new approach for quantitively assessing putative crystal structures with applications in crystal structure prediction (CSP) is introduced that is based upon experimental solution- and magic-angle spinning (MAS) solid-state NMR data and density functional theory (DFT) calculation. For the specific case of tolfenamic acid (TFA), we consider experimental solution-state NMR for a range of solvents, experimental MAS NMR of polymorphs I and II, and DFT calculations for four polymorphs. The change in NMR chemical shift observed in passing from the solution state to the solid state (ΔδExperimental) is calculated as the difference between 1H and 13C experimental solid-state chemical shifts for each polymorphic form (δSolid expt) and the corresponding solution-state NMR chemical shifts (δSolution expt). Separately, we use the gauge-included projector augmented wave (GIPAW) method to calculate the NMR chemical shifts for each form (δSolid calc) and for TFA in solution (δSolution calc) using the dynamic 3D solution conformational ensemble determined from NMR spectroscopy. The calculated change in passing from the solution state to the solid state (ΔδCalculated) is then calculated as the difference of δSolid calc and δSolution calc. Regression analysis for ΔδCalculated against ΔδExperimental followed by a t-test for statistical significance provides a robust quantitative assessment. We show that this assessment clearly identifies the correct polymorph, i.e., when comparing ΔδExperimental based on the experimental MAS NMR chemical shifts of form I or II with ΔδCalculated based on calculated chemical shifts for polymorphs I, II, III, and IV. Complementarity to the established approach of comparing δSolid expt to δSolid calc is explored. We further show that our approach is applicable if there are no solid-state crystal structure data. Specifically, δSolid calc in ΔδCalculated is replaced by the chemical shift for an isolated molecule with a specific conformation. Sampling conformations at specific 15° angle values and comparing them against experimental 13C chemical shift data for forms I and II identifies matching narrow ranges of conformations, successfully predicting the conformation of tolfenamic acid in each form. This methodology can therefore be used in crystal structure prediction to both reduce the initial conformational search space and also quantitatively assess subsequent putative structures to reliably and unambiguously identify the correct structure.

Targeted free energy estimation via learned mappings.

Free energy perturbation (FEP) was proposed by Zwanzig [J. Chem. Phys. 22, 1420 (1954)] more than six decades ago as a method to estimate free energy differences and has since inspired a huge body of related methods that use it as an integral building block. Being an importance sampling based estimator, however, FEP suffers from a severe limitation: the requirement of sufficient overlap between distributions. One strategy to mitigate this problem, called Targeted FEP, uses a high-dimensional mapping in configuration space to increase the overlap of the underlying distributions. Despite its potential, this method has attracted only limited attention due to the formidable challenge of formulating a tractable mapping. Here, we cast Targeted FEP as a machine learning problem in which the mapping is parameterized as a neural network that is optimized so as to increase the overlap. We develop a new model architecture that respects permutational and periodic symmetries often encountered in atomistic simulations and test our method on a fully periodic solvation system. We demonstrate that our method leads to a substantial variance reduction in free energy estimates when compared against baselines, without requiring any additional data.

Accurate measurement of long range proton-carbon scalar coupling constants.

The accuracy and practicality of measuring heteronuclear scalar coupling constants, nJCH, from modern NMR experimental methods is examined, based on F1 or F2 evolution of nJCH in HSQMBC (including EXSIDE) and HMBC experiments. The results from these methods are compared to both robust experimental data (derived from coupled 13C spectra), computed (Density Functional Theory) and literature values where available. We report on the accuracy, ease of use and time efficiency of these multi-dimensional methods and highlight their extent and limitations.

A refined model for the TSG-6 link module in complex with hyaluronan: use of defined oligosaccharides to probe structure and function.

Tumor necrosis factor-stimulated gene-6 (TSG-6) is an inflammation-associated hyaluronan (HA)-binding protein that contributes to remodeling of HA-rich extracellular matrices during inflammatory processes and ovulation. The HA-binding domain of TSG-6 consists solely of a Link module, making it a prototypical member of the superfamily of proteins that interacts with this high molecular weight polysaccharide composed of repeating disaccharides of D-glucuronic acid and N-acetyl-D-glucosamine (GlcNAc). Previously we modeled a complex of the TSG-6 Link module in association with an HA octasaccharide based on the structure of the domain in its HA-bound conformation. Here we have generated a refined model for a HA/Link module complex using novel restraints identified from NMR spectroscopy of the protein in the presence of 10 distinct HA oligosaccharides (from 4- to 8-mers); the model was then tested using unique sugar reagents, i.e. chondroitin/HA hybrid oligomers and an octasaccharide in which a single sugar ring was (13)C-labeled. The HA chain was found to make more extensive contacts with the TSG-6 surface than thought previously, such that a D-glucuronic acid ring makes stacking and ionic interactions with a histidine and lysine, respectively. Importantly, this causes the HA to bend around two faces of the Link module (resembling the way that HA binds to CD44), potentially providing a mechanism for how TSG-6 can reorganize HA during inflammation. However, the HA-binding site defined here may not play a role in TSG-6-mediated transfer of heavy chains from inter-α-inhibitor onto HA, a process known to be essential for ovulation.

Quantification of free ligand conformational preferences by NMR and their relationship to the bioactive conformation.

Accurate unbound solution 3D-structures of ligands provide unique opportunities for medicinal chemistry and, in particular, a context to understand binding thermodynamics and kinetics. Previous methods of deriving these 3D-structures have had neither the accuracy nor resolution needed for drug design and have not yet realized their potential. Here, we describe and apply a NMR methodology to the aminoglycoside streptomycin that can accurately quantify accessible 3D-space and rank the occupancy of observed conformers to a resolution that enables medicinal chemistry understanding and design. Importantly, it is based upon conventional small molecule NMR techniques and can be performed in physiologically-relevant solvents. The methodology uses multiple datasets, an order of magnitude more experimental data than previous NMR approaches and a dynamic model during refinement, is independent of computational chemistry and avoids the problem of virtual conformations. The refined set of solution 3D-shapes for streptomycin can be grouped into two major families, of which the most populated is almost identical to the 30S ribosomal subunit bioactive shape. We therefore propose that accurate unbound ligand solution conformations may, in some cases, provide a subsidiary route to bioactive shape without crystallography. This experimental technique opens up new opportunities for drug design and more so when complemented with protein co-crystal structures, SAR data and pharmacophore modeling.

Neural algorithmic reasoning.

We present neural algorithmic reasoning-the art of building neural networks that are able to execute algorithmic computation-and provide our opinion on its transformative potential for running classical algorithms on inputs previously considered inaccessible to them.

Crystal structures of tolfenamic acid polymorphic forms I and II with precise hydrogen-atom positions for nuclear magnetic resonance studies.

The structures of tolfenamic acid [TFA; 2-(3-chloro-2-methyl-anilino)benzoic acid, C14H12ClNO2] polymorph forms I and II have been redetermined [compare Andersen et al. (1989 ▸). J. Chem. Soc., Perkin Trans. 2, pp. 1443-1447] with improved precision using high-resolution X-ray diffraction data and Hirshfield atom refinement in order to better define both hydrogen-atom locations and their associated bond lengths. Covalent bond lengths to hydrogen were found to be significantly longer throughout both structures, especially for the anilino H atom, which is involved in an important intra-molecular N-H⋯O hydrogen bond to the carb-oxy-lic acid group. This hydrogen bond is shown to clearly perturb the electron density around both oxygen atoms in the latter group. The extended structures of both polymorphs feature carb-oxy-lic acid inversion dimers. These structures provide an improved foundation for nuclear magnetic resonance studies in both solution and the solid state.

Confidence modulates exploration and exploitation in value-based learning.

Uncertainty is ubiquitous in cognitive processing. In this study, we aim to investigate the ability agents possess to track and report the noise inherent in their mental operations, often in the form of confidence judgments. Here, we argue that humans can use uncertainty inherent in their representations of value beliefs to arbitrate between exploration and exploitation. Such uncertainty is reflected in explicit confidence judgments. Using a novel variant of a multi-armed bandit paradigm, we studied how beliefs were formed and how uncertainty in the encoding of these value beliefs (belief confidence) evolved over time. We found that people used uncertainty to arbitrate between exploration and exploitation, reflected in a higher tendency toward exploration when their confidence in their value representations was low. We furthermore found that value uncertainty can be linked to frameworks of metacognition in decision making in two ways. First, belief confidence drives decision confidence, i.e. people's evaluation of their own choices. Second, individuals with higher metacognitive insight into their choices were also better at tracing the uncertainty in their environment. Together, these findings argue that such uncertainty representations play a key role in the context of cognitive control.

Reinforcement Learning, Fast and Slow.

Deep reinforcement learning (RL) methods have driven impressive advances in artificial intelligence in recent years, exceeding human performance in domains ranging from Atari to Go to no-limit poker. This progress has drawn the attention of cognitive scientists interested in understanding human learning. However, the concern has been raised that deep RL may be too sample-inefficient - that is, it may simply be too slow - to provide a plausible model of how humans learn. In the present review, we counter this critique by describing recently developed techniques that allow deep RL to operate more nimbly, solving problems much more quickly than previous methods. Although these techniques were developed in an AI context, we propose that they may have rich implications for psychology and neuroscience. A key insight, arising from these AI methods, concerns the fundamental connection between fast RL and slower, more incremental forms of learning.

Plasticity of the TSG-6 HA-binding loop and mobility in the TSG-6-HA complex revealed by NMR and X-ray crystallography.

Tumour necrosis factor-stimulated gene-6 (TSG-6) is a glycosaminoglycan-binding protein expressed during inflammatory and inflammation-like processes. Previously NMR structures were calculated for the Link module of TSG-6 (Link_TSG6) in its free state and when bound to an octasaccharide of hyaluronan (HA(8)). Heparin was found to compete for HA binding even though it interacts at a site that is distinct from the HA-binding surface. Here we present crystallography data on the free protein, and (15)N NMR relaxation data for the uncomplexed and HA(8)-bound forms of Link_TSG6. Although the Link module is comparatively rigid overall, the free protein shows a high degree of mobility in the beta4/beta5 loop and at the Cys47-Cys68 disulfide bond, both of which are regions involved in HA binding. When bound to HA(8), this dynamic behaviour is dampened, but not eliminated, suggesting a degree of dynamic matching between the protein and sugar that may decrease the entropic penalty of complex formation. A further highly dynamic residue is Lys54, which is distant from the HA-binding site, but was previously shown to be involved in heparin binding. When HA is bound, Lys54 becomes less mobile, providing evidence for an allosteric effect linking the HA and heparin-binding sites. A mechanism is suggested involving the beta2-strand and alpha2-helix. The crystal structure of free Link_TSG6 contains five molecules in the asymmetric unit that are highly similar to the NMR structure and support the dynamic behaviour seen near the HA-binding site: they show little or no electron density for the beta4/beta5 loop and display multiple conformations for the Cys47-Cys68 disulfide bond. The crystal structures were used in docking calculations with heparin. An extended interface between a Link_TSG6 dimer and heparin 11-mer was identified that is in excellent agreement with previous mutagenesis and calorimetric data, providing the basis for further investigation of this interaction.

Pushing the Limits of Molecular Crystal Structure Determination From Powder Diffraction Data in High-Throughput Chemical Environments.

Crystal structure determination from powder diffraction data (SDPD) using the DASH software package is evaluated for data recorded using transmission capillary, transmission flat plate, and reflection flat plate geometries on a selection of pharmaceutical compounds. We show that transmission capillary geometry remains the best option when crystal structure determination is the primary consideration and, as expected, reflection flat plate geometry is not recommended for SDPD because of preferred orientation effects. However, the quality of crystal structures obtained from transmission plate instruments can be excellent, and the convenience factor for sample preparation, throughput, and retrieval is higher than that of transmission capillary instruments. Indeed, it is possible to solve crystal structures within an hour of a polycrystalline sample arriving in the laboratory, which has clear implications for making small-molecule crystal structures more routinely available to the practicing laboratory medicinal chemist. With appropriate modifications to crystal structure determination software, it can be imagined that SDPD could become a rapid turn-around walk-up analytical service in high-throughput chemical environments.

Hyaluronan: the local solution conformation determined by NMR and computer modeling is close to a contracted left-handed 4-fold helix.

The polysaccharide hyaluronan (HA) is a ubiquitous component of the vertebrate extracellular matrix with diverse physiological roles from space-filling to acting as a scaffold for other macromolecules. The molecular interactions responsible for these solution properties have been the subject of much debate and, primarily due to the lack of residue-specific experimental data, no consensus model for the three-dimensional conformation nor dynamics of HA in solution has emerged. Here, the solution conformation of HA is investigated using molecular dynamics (MD) simulations and high-field nuclear magnetic resonance (NMR). In contrast to previous studies, MD simulations incorporated explicit water molecules and sodium ions, while NMR experiments utilized (15)N-enriched oligosaccharides to allow residue-specific information to be obtained. The resultant average conformation is predicted to be almost a contracted left-handed 4-fold helix; i.e. similar to that observed for sodium hyaluronate fibers by X-ray diffraction, but with the acetamido side-chain trans to H(2). The glycosidic linkages and acetamido side-chains are predicted to have standard deviation rotations of 13 degrees and 18 degrees around their mean conformations in free solution, respectively, and are not observed to be stabilized by strong intramolecular hydrogen bonds as X-ray fiber diffraction refinements describe for the solid-state. Rather, weak and transient hydrogen bonds that are in rapid interchange with solvent molecules are predicted. These predictions are quantitatively consistent with demanding residue-specific NMR data and correspond to an HA molecule that is rod-like as an oligosaccharide and behaves as a stiffened random coil at large molecular mass, in close agreement with previous hydrodynamic observations. This new description of the solution conformation of HA is consistent with all available experimental data and accounts for its viscoelastic space-filling properties. This representation can be used as a basis for modeling the association between HA and proteins, which will elucidate important aspects of extracellular matrix assembly.

Hyaluronan: the absence of amide-carboxylate hydrogen bonds and the chain conformation in aqueous solution are incompatible with stable secondary and tertiary structure models.

Contradictory descriptions for the aqueous solution conformation of the glycosaminoglycan hyaluronan (HA) exist in the literature. According to hydrodynamic and simulation data, HA molecules are stiffened by a rapidly interchanging network of transient hydrogen bonds at the local level and do not significantly associate at the global level. In marked contrast, models derived from NMR data suggest that the secondary structure involves persistent hydrogen bonds and that strong associations between chains can occur to form vast stable tertiary structures. These models require an extended 2-fold helical conformation of the HA chain and specific hydrogen bonds between amide and carboxylate groups. To test these descriptions, we have used 15N-labelled oligosaccharides and high-field NMR to measure pertinent properties of the acetamido group. The amide proton chemical shift perturbation and carboxylate group pK(a) value are inconsistent with a highly populated hydrogen bond between the amide and carboxylate groups. Amide proton temperature coefficients and chemical exchange rates confirm this conclusion. Comparison of oligomer properties with polymeric HA indicates that there is no discernible difference in amide proton environment between the centre of octasaccharides and the polymer, inconsistent with the formation of tertiary structures. A [1H-1H-15N] NOESY-HSQC (heteronuclear single-quantum correlation) spectrum recorded on an HA octasaccharide revealed that amide groups in the centre are in a trans orientation and that the average solution conformation is not an extended 2-fold helix. Therefore the two key aspects of the secondary and tertiary structure models are unlikely to be correct. Rather, these new NMR data agree with descriptions from hydrodynamic and simulations data.

Complete assignment of hyaluronan oligosaccharides up to hexasaccharides.

The glycosaminoglycan hyaluronan is involved in a diverse range of physiological and diseases processes and comprises repeated disaccharide units of N-acetyl-d-glucosamine (GlcNAc) and d-glucuronic acid (GlcA). A molecular description of the solution conformation of HA is required to account for this biology, which is best attained using nuclear magnetic resonance (NMR). NMR studies of the polymer, however, are frustrated by resonance overlap arising from the highly degenerate structure. In contrast, end-effects in oligosaccharides can produce some chemical shift dispersion, giving the possibility that their conformational properties can be measured and extrapolated to models of the polymer. We report the complete resolution and assignment of (1)H, (13)C and (15)N nuclei in hyaluronan oligosaccharides with seven different naturally occurring terminal rings. At 900MHz, all (1)H nuclei in the hexasaccharide GlcA-beta-(1-->3)-GlcNAc-beta-(1-->4)-GlcA-beta-(1-->3)-GlcNAc-beta-(1-->4)-GlcA-beta-(1-->3)-GlcNAc-OH were uniquely resolved and the two central rings were found to be a good model for the polymer environment. These assignments now allow resolved, unambiguous structural restraints to be acquired on this oligosaccharide and extrapolated to models for the solution conformation of the polymer.

NMR spectra of oligosaccharides at ultra-high field (900 MHz) have better resolution than expected due to favourable molecular tumbling.

Nuclear magnetic resonance (NMR) remains the most promising technique for acquiring atomic-resolution information in complex carbohydrates. Significant obstacles to the acquisition of such data are the poor chemical-shift dispersion and artifacts resultant from their degenerate chemical structures. The recent development of ultra-high-field NMR (at 900 MHz and beyond) gives new potential to overcome these problems, as we demonstrate on a hexasaccharide of the highly repetitive glycosaminoglycan hyaluronan. At 900 MHz, the expected increase in spectral dispersion due to higher resonance frequencies and reduction in strong coupling-associated distortions are observed. In addition, the fortuitous molecular tumbling rate of oligosaccharides results in longer T2-values that further significantly enhances resolution, an effect not available to proteins. Combined, the resolution enhancement can be as much as twofold relative to 600 MHz, allowing all 1H-resonances in the hexasaccharide to be unambiguously assigned using standard natural-abundance experiments. The use of ultra-high-field spectrometers is clearly advantageous and promises a new and exciting era in carbohydrate structural biology.

Computations Underlying Social Hierarchy Learning: Distinct Neural Mechanisms for Updating and Representing Self-Relevant Information.

Knowledge about social hierarchies organizes human behavior, yet we understand little about the underlying computations. Here we show that a Bayesian inference scheme, which tracks the power of individuals, better captures behavioral and neural data compared with a reinforcement learning model inspired by rating systems used in games such as chess. We provide evidence that the medial prefrontal cortex (MPFC) selectively mediates the updating of knowledge about one's own hierarchy, as opposed to that of another individual, a process that underpinned successful performance and involved functional interactions with the amygdala and hippocampus. In contrast, we observed domain-general coding of rank in the amygdala and hippocampus, even when the task did not require it. Our findings reveal the computations underlying a core aspect of social cognition and provide new evidence that self-relevant information may indeed be afforded a unique representational status in the brain.