Towards parsimonious generative modeling of RNA families.

Generative probabilistic models emerge as a new paradigm in data-driven, evolution-informed design of biomolecular sequences. This paper introduces a novel approach, called Edge Activation Direct Coupling Analysis (eaDCA), tailored to the characteristics of RNA sequences, with a strong emphasis on simplicity, efficiency, and interpretability. eaDCA explicitly constructs sparse coevolutionary models for RNA families, achieving performance levels comparable to more complex methods while utilizing a significantly lower number of parameters. Our approach demonstrates efficiency in generating artificial RNA sequences that closely resemble their natural counterparts in both statistical analyses and SHAPE-MaP experiments, and in predicting the effect of mutations. Notably, eaDCA provides a unique feature: estimating the number of potential functional sequences within a given RNA family. For example, in the case of cyclic di-AMP riboswitches (RF00379), our analysis suggests the existence of approximately 1039 functional nucleotide sequences. While huge compared to the known <4000 natural sequences, this number represents only a tiny fraction of the vast pool of nearly 1082 possible nucleotide sequences of the same length (136 nucleotides). These results underscore the promise of sparse and interpretable generative models, such as eaDCA, in enhancing our understanding of the expansive RNA sequence space.

The nature of non-phononic excitations in disordered systems.

The frequency scaling exponent of low-frequency excitations in microscopically small glasses, which do not allow for the existence of waves (phonons), has been in the focus of the recent literature. The density of states g(ω) of these modes obeys an ωs scaling, where the exponent s, ranging between 2 and 5, depends on the quenching protocol. The orgin of these findings remains controversal. Here we show, using heterogeneous-elasticity theory, that in a marginally-stable glass sample g(ω) follows a Debye-like scaling (s = 2), and the associated excitations (type-I) are of random-matrix type. Further, using a generalisation of the theory, we demonstrate that in more stable samples, other, (type-II) excitations prevail, which are non-irrotational oscillations, associated with local frozen-in stresses. The corresponding frequency scaling exponent s is governed by the statistics of small values of the stresses and, therefore, depends on the details of the interaction potential.

Scaling regimes and fluctuations of observables in computer glasses approaching the unjamming transition.

Under decompression, disordered solids undergo an unjamming transition where they become under-coordinated and lose their structural rigidity. The mechanical and vibrational properties of these materials have been an object of theoretical, numerical, and experimental research for decades. In the study of low-coordination solids, understanding the behavior and physical interpretation of observables that diverge near the transition is of particular importance. Several such quantities are length scales (ξ or l) that characterize the size of excitations, the decay of spatial correlations, the response to perturbations, or the effect of physical constraints in the boundary or bulk of the material. Additionally, the spatial and sample-to-sample fluctuations of macroscopic observables such as contact statistics or elastic moduli diverge approaching unjamming. Here, we discuss important connections between all of these quantities and present numerical results that characterize the scaling properties of sample-to-sample contact and shear modulus fluctuations in ensembles of low-coordination disordered sphere packings and spring networks. Overall, we highlight three distinct scaling regimes and two crossovers in the disorder quantifiers χz and χµ as functions of system size N and proximity to unjamming δz. As we discuss, χX relates to the standard deviation σX of the sample-to-sample distribution of the quantity X (e.g., excess coordination δz or shear modulus µ) for an ensemble of systems. Importantly, χµ has been linked to experimentally accessible quantities that pertain to sound attenuation and the density of vibrational states in glasses. We investigate similarities and differences in the behaviors of χz and χµ near the transition and discuss the implications of our findings on current literature, unifying findings in previous studies.

Finding defects in glasses through machine learning.

Structural defects control the kinetic, thermodynamic and mechanical properties of glasses. For instance, rare quantum tunneling two-level systems (TLS) govern the physics of glasses at very low temperature. Due to their extremely low density, it is very hard to directly identify them in computer simulations. We introduce a machine learning approach to efficiently explore the potential energy landscape of glass models and identify desired classes of defects. We focus in particular on TLS and we design an algorithm that is able to rapidly predict the quantum splitting between any two amorphous configurations produced by classical simulations. This in turn allows us to shift the computational effort towards the collection and identification of a larger number of TLS, rather than the useless characterization of non-tunneling defects which are much more abundant. Finally, we interpret our machine learning model to understand how TLS are identified and characterized, thus giving direct physical insight into their microscopic nature.

Microscopic observation of two-level systems in a metallic glass model.

The low-temperature quasi-universal behavior of amorphous solids has been attributed to the existence of spatially localized tunneling defects found in the low-energy regions of the potential energy landscape. Computational models of glasses can be studied to elucidate the microscopic nature of these defects. Recent simulation work has demonstrated the means of generating stable glassy configurations for models that mimic metallic glasses using the swap Monte Carlo algorithm. Building on these studies, we present an extensive exploration of the glassy metabasins of the potential energy landscape of a variant of the most widely used model of metallic glasses. We carefully identify tunneling defects and reveal their depletion with increased glass stability. The density of tunneling defects near the experimental glass transition temperature appears to be in good agreement with experimental measurements.

Creating bulk ultrastable glasses by random particle bonding.

A recent breakthrough in glass science has been the synthesis of ultrastable glasses via physical vapor deposition techniques. These samples display enhanced thermodynamic, kinetic and mechanical stability, with important implications for fundamental science and technological applications. However, the vapor deposition technique is limited to atomic, polymer and organic glass-formers and is only able to produce thin film samples. Here, we propose a novel approach to generate ultrastable glassy configurations in the bulk, via random particle bonding, and using computer simulations we show that this method does indeed allow for the production of ultrastable glasses. Our technique is in principle applicable to any molecular or soft matter system, such as colloidal particles with tunable bonding interactions, thus opening the way to the design of a large class of ultrastable glasses.

Finite-Disorder Critical Point in the Yielding Transition of Elastoplastic Models.

Upon loading, amorphous solids can exhibit brittle yielding, with the abrupt formation of macroscopic shear bands leading to fracture, or ductile yielding, with a multitude of plastic events leading to homogeneous flow. It has been recently proposed, and subsequently questioned, that the two regimes are separated by a sharp critical point, as a function of some control parameter characterizing the intrinsic disorder strength and the degree of stability of the solid. In order to resolve this issue, we have performed extensive numerical simulations of athermally driven elastoplastic models with long-range and anisotropic realistic interaction kernels in two and three dimensions. Our results provide clear evidence for a finite-disorder critical point separating brittle and ductile yielding, and we provide an estimate of the critical exponents in 2D and 3D.

Equilibrium fluctuations in mean-field disordered models.

Mean-field models of glasses that present a random first order transition exhibit highly nontrivial fluctuations. Building on previous studies that focused on the critical scaling regime, we here obtain a fully quantitative framework for all equilibrium conditions. By means of the replica method we evaluate Gaussian fluctuations of the overlaps around the thermodynamic limit, decomposing them in thermal fluctuations inside each state and heterogeneous fluctuations between different states. We first test and compare our analytical results with numerical simulation results for the p-spin spherical model and the random orthogonal model, and then analyze the random Lorentz gas. In all cases, a strong quantitative agreement is obtained. Our analysis thus provides a robust scheme for identifying the key finite-size (or finite-dimensional) corrections to the mean-field treatment of these paradigmatic glass models.

Local Dynamical Heterogeneity in Simple Glass Formers.

We study the local dynamical fluctuations in glass-forming models of particles embedded in d-dimensional space, in the mean-field limit of d→∞. Our analytical calculation reveals that single-particle observables, such as squared particle displacements, display divergent fluctuations around the dynamical (or mode-coupling) transition, due to the emergence of nontrivial correlations between displacements along different directions. This effect notably gives rise to a divergent non-Gaussian parameter, α_{2}. The d→∞ local dynamics therefore becomes quite rich upon approaching the glass transition. The finite-d remnant of this phenomenon further provides a long sought-after, first-principle explanation for the growth of α_{2} around the glass transition that is not based on multiparticle correlations.

Supervised perceptron learning vs unsupervised Hebbian unlearning: Approaching optimal memory retrieval in Hopfield-like networks.

The Hebbian unlearning algorithm, i.e., an unsupervised local procedure used to improve the retrieval properties in Hopfield-like neural networks, is numerically compared to a supervised algorithm to train a linear symmetric perceptron. We analyze the stability of the stored memories: basins of attraction obtained by the Hebbian unlearning technique are found to be comparable in size to those obtained in the symmetric perceptron, while the two algorithms are found to converge in the same region of Gardner's space of interactions, having followed similar learning paths. A geometric interpretation of Hebbian unlearning is proposed to explain its optimal performances. Because the Hopfield model is also a prototypical model of the disordered magnetic system, it might be possible to translate our results to other models of interest for memory storage in materials.

Modeling Sequence-Space Exploration and Emergence of Epistatic Signals in Protein Evolution.

During their evolution, proteins explore sequence space via an interplay between random mutations and phenotypic selection. Here, we build upon recent progress in reconstructing data-driven fitness landscapes for families of homologous proteins, to propose stochastic models of experimental protein evolution. These models predict quantitatively important features of experimentally evolved sequence libraries, like fitness distributions and position-specific mutational spectra. They also allow us to efficiently simulate sequence libraries for a vast array of combinations of experimental parameters like sequence divergence, selection strength, and library size. We showcase the potential of the approach in reanalyzing two recent experiments to determine protein structure from signals of epistasis emerging in experimental sequence libraries. To be detectable, these signals require sufficiently large and sufficiently diverged libraries. Our modeling framework offers a quantitative explanation for different outcomes of recently published experiments. Furthermore, we can forecast the outcome of time- and resource-intensive evolution experiments, opening thereby a way to computationally optimize experimental protocols.

Active matter in infinite dimensions: Fokker-Planck equation and dynamical mean-field theory at low density.

We investigate the behavior of self-propelled particles in infinite space dimensions by comparing two powerful approaches in many-body dynamics: the Fokker-Planck equation and dynamical mean-field theory. The dynamics of the particles at low densities and infinite persistence time is solved in the steady state with both methods, thereby proving the consistency of the two approaches in a paradigmatic out-of-equilibrium system. We obtain the analytic expression for the pair distribution function and the effective self-propulsion to first-order in the density, confirming the results obtained in a previous paper [T. Arnoulx de Pirey et al., Phys. Rev. Lett. 123, 260602 (2019)] and extending them to the case of a non-monotonous interaction potential. Furthermore, we obtain the transient behavior of active hard spheres when relaxing from the equilibrium to the nonequilibrium steady state. Our results show how collective dynamics is affected by interactions to first-order in the density and point out future directions for further analytical and numerical solutions of this problem.

adabmDCA: adaptive Boltzmann machine learning for biological sequences.

BACKGROUND: Boltzmann machines are energy-based models that have been shown to provide an accurate statistical description of domains of evolutionary-related protein and RNA families. They are parametrized in terms of local biases accounting for residue conservation, and pairwise terms to model epistatic coevolution between residues. From the model parameters, it is possible to extract an accurate prediction of the three-dimensional contact map of the target domain. More recently, the accuracy of these models has been also assessed in terms of their ability in predicting mutational effects and generating in silico functional sequences. RESULTS: Our adaptive implementation of Boltzmann machine learning, adabmDCA, can be generally applied to both protein and RNA families and accomplishes several learning set-ups, depending on the complexity of the input data and on the user requirements. The code is fully available at https://github.com/anna-pa-m/adabmDCA . As an example, we have performed the learning of three Boltzmann machines modeling the Kunitz and Beta-lactamase2 protein domains and TPP-riboswitch RNA domain. CONCLUSIONS: The models learned by adabmDCA are comparable to those obtained by state-of-the-art techniques for this task, in terms of the quality of the inferred contact map as well as of the synthetically generated sequences. In addition, the code implements both equilibrium and out-of-equilibrium learning, which allows for an accurate and lossless training when the equilibrium one is prohibitive in terms of computational time, and allows for pruning irrelevant parameters using an information-based criterion.

Efficient generative modeling of protein sequences using simple autoregressive models.

Generative models emerge as promising candidates for novel sequence-data driven approaches to protein design, and for the extraction of structural and functional information about proteins deeply hidden in rapidly growing sequence databases. Here we propose simple autoregressive models as highly accurate but computationally efficient generative sequence models. We show that they perform similarly to existing approaches based on Boltzmann machines or deep generative models, but at a substantially lower computational cost (by a factor between 102 and 103). Furthermore, the simple structure of our models has distinctive mathematical advantages, which translate into an improved applicability in sequence generation and evaluation. Within these models, we can easily estimate both the probability of a given sequence, and, using the model's entropy, the size of the functional sequence space related to a specific protein family. In the example of response regulators, we find a huge number of ca. 1068 possible sequences, which nevertheless constitute only the astronomically small fraction 10-80 of all amino-acid sequences of the same length. These findings illustrate the potential and the difficulty in exploring sequence space via generative sequence models.

Sparse generative modeling via parameter reduction of Boltzmann machines: Application to protein-sequence families.

Boltzmann machines (BMs) are widely used as generative models. For example, pairwise Potts models (PMs), which are instances of the BM class, provide accurate statistical models of families of evolutionarily related protein sequences. Their parameters are the local fields, which describe site-specific patterns of amino acid conservation, and the two-site couplings, which mirror the coevolution between pairs of sites. This coevolution reflects structural and functional constraints acting on protein sequences during evolution. The most conservative choice to describe the coevolution signal is to include all possible two-site couplings into the PM. This choice, typical of what is known as Direct Coupling Analysis, has been successful for predicting residue contacts in the three-dimensional structure, mutational effects, and generating new functional sequences. However, the resulting PM suffers from important overfitting effects: many couplings are small, noisy, and hardly interpretable; the PM is close to a critical point, meaning that it is highly sensitive to small parameter perturbations. In this work, we introduce a general parameter-reduction procedure for BMs, via a controlled iterative decimation of the less statistically significant couplings, identified by an information-based criterion that selects either weak or statistically unsupported couplings. For several protein families, our procedure allows one to remove more than 90% of the PM couplings, while preserving the predictive and generative properties of the original dense PM, and the resulting model is far away from criticality, hence more robust to noise.

Mean-Field Caging in a Random Lorentz Gas.

The random Lorentz gas (RLG) is a minimal model of both percolation and glassiness, which leads to a paradox in the infinite-dimensional, d → ∞ limit: the localization transition is then expected to be continuous for the former and discontinuous for the latter. As a putative resolution, we have recently suggested that, as d increases, the behavior of the RLG converges to the glassy description and that percolation physics is recovered thanks to finite-d perturbative and nonperturbative (instantonic) corrections [Biroli et al. Phys. Rev. E 2021, 103, L030104]. Here, we expand on the d → ∞ physics by considering a simpler static solution as well as the dynamical solution of the RLG. Comparing the 1/d correction of this solution with numerical results reveals that even perturbative corrections fall out of reach of existing theoretical descriptions. Comparing the dynamical solution with the mode-coupling theory (MCT) results further reveals that, although key quantitative features of MCT are far off the mark, it does properly capture the discontinuous nature of the d → ∞ RLG. These insights help chart a path toward a complete description of finite-dimensional glasses.

Interplay between percolation and glassiness in the random Lorentz gas.

The random Lorentz gas (RLG) is a minimal model of transport in heterogeneous media that exhibits a continuous localization transition controlled by void space percolation. The RLG also provides a toy model of particle caging, which is known to be relevant for describing the discontinuous dynamical transition of glasses. In order to clarify the interplay between the seemingly incompatible percolation and caging descriptions of the RLG, we consider its exact mean-field solution in the infinite-dimensional d→∞ limit and perform numerics in d=2...20. We find that for sufficiently high d the mean-field caging transition precedes and prevents the percolation transition, which only happens on timescales diverging with d. We further show that activated processes related to rare cage escapes destroy the glass transition in finite dimensions, leading to a rich interplay between glassiness and percolation physics. This advance suggests that the RLG can be used as a toy model to develop a first-principle description of particle hopping in structural glasses.

V-, U-, L- or W-shaped economic recovery after Covid-19: Insights from an Agent Based Model.

We discuss the impact of a Covid-19-like shock on a simple model economy, described by the previously developed Mark-0 Agent-Based Model. We consider a mixed supply and demand shock, and show that depending on the shock parameters (amplitude and duration), our model economy can display V-shaped, U-shaped or W-shaped recoveries, and even an L-shaped output curve with permanent output loss. This is due to the economy getting trapped in a self-sustained "bad" state. We then discuss two policies that attempt to moderate the impact of the shock: giving easy credit to firms, and the so-called helicopter money, i.e. injecting new money into the households savings. We find that both policies are effective if strong enough. We highlight the potential danger of terminating these policies too early, although inflation is substantially increased by lax access to credit. Finally, we consider the impact of a second lockdown. While we only discuss a limited number of scenarios, our model is flexible and versatile enough to accommodate a wide variety of situations, thus serving as a useful exploratory tool for a qualitative, scenario-based understanding of post-Covid recovery. The corresponding code is available on-line.

Three simple scenarios for high-dimensional sphere packings.

Based on results from the physics and mathematics literature which suggest a series of clearly defined conjectures, we formulate three simple scenarios for the fate of hard sphere crystallization in high dimension: in scenario A, crystallization is impeded and the glass phase constitutes the densest packing; in scenario B, crystallization from the liquid is possible, but takes place much beyond the dynamical glass transition and is thus dynamically implausible; and in scenario C, crystallization is possible and takes place before (or just after) dynamical arrest, thus making it plausibly accessible from the liquid state. In order to assess the underlying conjectures and thus obtain insight into which scenario is most likely to be realized, we investigate the densest sphere packings for dimension d=3-10 using cell-cluster expansions as well as numerical simulations. These resulting estimates of the crystal entropy near close packing tend to support scenario C. We additionally confirm that the crystal equation of state is dominated by the free-volume expansion and that a meaningful polynomial correction can be formulated.