On the different regimes of stochastic gradient descent.

Modern deep networks are trained with stochastic gradient descent (SGD) whose key hyperparameters are the number of data considered at each step or batch size [Formula: see text], and the step size or learning rate [Formula: see text]. For small [Formula: see text] and large [Formula: see text], SGD corresponds to a stochastic evolution of the parameters, whose noise amplitude is governed by the "temperature" [Formula: see text]. Yet this description is observed to break down for sufficiently large batches [Formula: see text], or simplifies to gradient descent (GD) when the temperature is sufficiently small. Understanding where these cross-overs take place remains a central challenge. Here, we resolve these questions for a teacher-student perceptron classification model and show empirically that our key predictions still apply to deep networks. Specifically, we obtain a phase diagram in the [Formula: see text]-[Formula: see text] plane that separates three dynamical phases: i) a noise-dominated SGD governed by temperature, ii) a large-first-step-dominated SGD and iii) GD. These different phases also correspond to different regimes of generalization error. Remarkably, our analysis reveals that the batch size [Formula: see text] separating regimes (i) and (ii) scale with the size [Formula: see text] of the training set, with an exponent that characterizes the hardness of the classification problem.

Local vs. cooperative: Unraveling glass transition mechanisms with SEER.

Which phenomenon slows down the dynamics in supercooled liquids and turns them into glasses is a long-standing question of condensed matter. Most popular theories posit that as the temperature decreases, many events must occur in a coordinated fashion on a growing length scale for relaxation to occur. Instead, other approaches consider that local barriers associated with the elementary rearrangement of a few particles or "excitations" govern the dynamics. To resolve this conundrum, our central result is to introduce an algorithm, Systematic Excitation ExtRaction, which can systematically extract hundreds of excitations and their energy from any given configuration. We also provide a measurement of the activation energy, characterizing the liquid dynamics, based on fast quenching and reheating. We use these two methods in a popular liquid model of polydisperse particles. Such polydisperse models are known to capture the hallmarks of the glass transition and can be equilibrated efficiently up to millisecond time scales. The analysis reveals that cooperative effects do not control the fragility of such liquids: the change of energy of local barriers determines the change of activation energy. More generally, these methods can now be used to measure the degree of cooperativity of any liquid model.

Testing Theories of the Glass Transition with the Same Liquid but Many Kinetic Rules.

We study the glass transition by exploring a broad class of kinetic rules that can significantly modify the normal dynamics of supercooled liquids while maintaining thermal equilibrium. Beyond the usual dynamics of liquids, this class includes dynamics in which a fraction (1-f_{R}) of the particles can perform pairwise exchange or "swap moves," while a fraction f_{P} of the particles can move only along restricted directions. We find that (i) the location of the glass transition varies greatly but smoothly as f_{P} and f_{R} change and (ii) it is governed by a linear combination of f_{P} and f_{R}. (iii) Dynamical heterogeneities (DHs) are not governed by the static structure of the material; their magnitude correlates instead with the relaxation time. (iv) We show that a recent theory for temporal growth of DHs based on thermal avalanches holds quantitatively throughout the (f_{R},f_{P}) diagram. These observations are negative items for some existing theories of the glass transition, particularly those reliant on growing thermodynamic order or locally favored structure, and open new avenues to test other approaches, as we illustrate.

Scaling Description of Creep Flow in Amorphous Solids.

Amorphous solids such as coffee foam, toothpaste, or mayonnaise display a transient creep flow when a stress Σ is suddenly imposed. The associated strain rate is commonly found to decay in time as γ[over ˙]∼t^{-ν}, followed either by arrest or by a sudden fluidization. Various empirical laws have been suggested for the creep exponent ν and fluidization time τ_{f} in experimental and numerical studies. Here, we postulate that plastic flow is governed by the difference between Σ and the transient yield stress Σ_{t}(γ) that characterizes the stability of configurations visited by the system at strain γ. Assuming the analyticity of Σ_{t}(γ) allows us to predict ν and asymptotic behaviors of τ_{f} in terms of properties of stationary flows. We test successfully our predictions using elastoplastic models and published experimental results.

How collective asperity detachments nucleate slip at frictional interfaces.

Sliding at a quasi-statically loaded frictional interface can occur via macroscopic slip events, which nucleate locally before propagating as rupture fronts very similar to fracture. We introduce a microscopic model of a frictional interface that includes asperity-level disorder, elastic interaction between local slip events, and inertia. For a perfectly flat and homogeneously loaded interface, we find that slip is nucleated by avalanches of asperity detachments of extension larger than a critical radius [Formula: see text] governed by a Griffith criterion. We find that after slip, the density of asperities at a local distance to yielding [Formula: see text] presents a pseudogap [Formula: see text], where θ is a nonuniversal exponent that depends on the statistics of the disorder. This result makes a link between friction and the plasticity of amorphous materials where a pseudogap is also present. For friction, we find that a consequence is that stick-slip is an extremely slowly decaying finite-size effect, while the slip nucleation radius [Formula: see text] diverges as a θ-dependent power law of the system size. We discuss how these predictions can be tested experimentally.

Nonlocal Effects Reflect the Jamming Criticality in Frictionless Granular Flows Down Inclines.

The jamming transition is accompanied by a rich phenomenology such as hysteresis or nonlocal effects that is still not well understood. Here, we experimentally investigate a model frictionless granular layer flowing down an inclined plane as a way to disentangle generic collective effects from those arising from frictional interactions. We find that thin frictionless granular layers are devoid of hysteresis of the avalanche angle, yet the layer stability increases as it gets thinner. Steady rheological laws obtained for different layer thicknesses can be collapsed into a unique master curve, supporting the idea that nonlocal effects are the consequence of the usual finite-size effects associated with the presence of a critical point. This collapse indicates that the so-called isostatic length l^{*}, the scale on which pinning a boundary freezes all remaining floppy modes, governs the effect of boundaries on flow and rules out other propositions made in the past.

Direct coupling analysis of epistasis in allosteric materials.

In allosteric proteins, the binding of a ligand modifies function at a distant active site. Such allosteric pathways can be used as target for drug design, generating considerable interest in inferring them from sequence alignment data. Currently, different methods lead to conflicting results, in particular on the existence of long-range evolutionary couplings between distant amino-acids mediating allostery. Here we propose a resolution of this conundrum, by studying epistasis and its inference in models where an allosteric material is evolved in silico to perform a mechanical task. We find in our model the four types of epistasis (Synergistic, Sign, Antagonistic, Saturation), which can be both short or long-range and have a simple mechanical interpretation. We perform a Direct Coupling Analysis (DCA) and find that DCA predicts well the cost of point mutations but is a rather poor generative model. Strikingly, it can predict short-range epistasis but fails to capture long-range epistasis, in consistence with empirical findings. We propose that such failure is generic when function requires subparts to work in concert. We illustrate this idea with a simple model, which suggests that other methods may be better suited to capture long-range effects.

Universality of jamming of nonspherical particles.

Amorphous packings of nonspherical particles such as ellipsoids and spherocylinders are known to be hypostatic: The number of mechanical contacts between particles is smaller than the number of degrees of freedom, thus violating Maxwell's mechanical stability criterion. In this work, we propose a general theory of hypostatic amorphous packings and the associated jamming transition. First, we show that many systems fall into a same universality class. As an example, we explicitly map ellipsoids into a system of "breathing" particles. We show by using a marginal stability argument that in both cases jammed packings are hypostatic and that the critical exponents related to the contact number and the vibrational density of states are the same. Furthermore, we introduce a generalized perceptron model which can be solved analytically by the replica method. The analytical solution predicts critical exponents in the same hypostatic jamming universality class. Our analysis further reveals that the force and gap distributions of hypostatic jamming do not show power-law behavior, in marked contrast to the isostatic jamming of spherical particles. Finally, we confirm our theoretical predictions by numerical simulations.

Jamming with Tunable Roughness.

We introduce a new model to study the effect of surface roughness on the jamming transition. By performing numerical simulations, we show that for a smooth surface, the jamming transition density and the contact number at the transition point both increase upon increasing asphericity, as for ellipsoids and spherocylinders. Conversely, for a rough surface, both quantities decrease, in quantitative agreement with the behavior of frictional particles. Furthermore, in the limit corresponding to the Coulomb friction law, the model satisfies a generalized isostaticity criterion proposed in previous studies. We introduce a counting argument that justifies this criterion and interprets it geometrically. Finally, we propose a simple theory to predict the contact number at finite friction from the knowledge of the force distribution in the infinite friction limit.

Architecture and coevolution of allosteric materials.

We introduce a numerical scheme to evolve functional elastic materials that can accomplish a specified mechanical task. In this scheme, the number of solutions, their spatial architectures, and the correlations among them can be computed. As an example, we consider an "allosteric" task, which requires the material to respond specifically to a stimulus at a distant active site. We find that functioning materials evolve a less-constrained trumpet-shaped region connecting the stimulus and active sites, and that the amplitude of the elastic response varies nonmonotonically along the trumpet. As previously shown for some proteins, we find that correlations appearing during evolution alone are sufficient to identify key aspects of this design. Finally, we show that the success of this architecture stems from the emergence of soft edge modes recently found to appear near the surface of marginally connected materials. Overall, our in silico evolution experiment offers a window to study the relationship between structure, function, and correlations emerging during evolution.

Mechanics of Allostery: Contrasting the Induced Fit and Population Shift Scenarios.

In allosteric proteins, binding a ligand can affect function at a distant location, for example, by changing the binding affinity of a substrate at the active site. The induced fit and population shift models, which differ by the assumed number of stable configurations, explain such cooperative binding from a thermodynamic viewpoint. Yet, understanding what mechanical principles constrain these models remains a challenge. Here, we provide an empirical study on 34 proteins supporting the idea that allosteric conformational change generally occurs along a soft elastic mode presenting extended regions of high shear. We argue, based on a detailed analysis of how the energy profile along such a mode depends on binding, that in the induced fit scenario, there is an optimal stiffness ka∗ ∼ 1/N for cooperative binding, where N is the number of residues. We find that the population shift scenario is more robust to mutations affecting stiffness because binding becomes more and more cooperative with stiffness up to the same characteristic value ka∗, beyond which cooperativity saturates instead of decaying. We numerically confirm these findings in a nonlinear mechanical model. Dynamical considerations suggest that a stiffness of order ka∗ is favorable in that scenario as well, supporting that for proper function, proteins must evolve a functional elastic mode that is softer as their size increases. In consistency with this view, we find a fair anticorrelation between the stiffness of the allosteric response and protein size in our data set.

Scale-free channeling patterns near the onset of erosion of sheared granular beds.

Erosion shapes our landscape and occurs when a sufficient shear stress is exerted by a fluid on a sedimented layer. What controls erosion at a microscopic level remains debated, especially near the threshold forcing where it stops. Here we study, experimentally, the collective dynamics of the moving particles, using a setup where the system spontaneously evolves toward the erosion onset. We find that the spatial organization of the erosion flux is heterogeneous in space and occurs along channels of local flux σ whose distribution displays scaling near threshold and follows [Formula: see text], where J is the mean erosion flux. Channels are strongly correlated in the direction of forcing but not in the transverse direction. We show that these results quantitatively agree with a model where the dynamics is governed by the competition of disorder (which channels mobile particles) and particle interactions (which reduces channeling). These observations support that, for laminar flows, erosion is a dynamical phase transition that shares similarity with the plastic depinning transition occurring in dirty superconductors. The methodology we introduce here could be applied to probe these systems as well.

Principles for Optimal Cooperativity in Allosteric Materials.

Allosteric proteins transmit a mechanical signal induced by binding a ligand. However, understanding the nature of the information transmitted and the architectures optimizing such transmission remains a challenge. Here we show, using an in silico evolution scheme and theoretical arguments, that architectures optimized to be cooperative, which efficiently propagate energy, qualitatively differ from previously investigated materials optimized to propagate strain. Although we observe a large diversity of functioning cooperative architectures (including shear, hinge, and twist designs), they all obey the same principle of displaying a mechanism, i.e., an extended soft mode. We show that its optimal frequency decreases with the spatial extension L of the system as L-d/2, where d is the spatial dimension. For these optimal designs, cooperativity decays logarithmically with L for d = 2 and does not decay for d = 3. Overall, our approach leads to a natural explanation for several observations in allosteric proteins and indicates an experimental path to test if allosteric proteins lie close to optimality.

Does a Growing Static Length Scale Control the Glass Transition?

Several theories of the glass transition propose that the structural relaxation time τ_{α} is controlled by a growing static length scale ξ that is determined by the free energy landscape but not by the local dynamic rules governing its exploration. We argue, based on recent simulations using particle-radius-swap dynamics, that only a modest factor in the increase in τ_{α} on approach to the glass transition may stem from the growth of a static length, with a vastly larger contribution attributable, instead, to a slowdown of local dynamics. This reinforces arguments that we base on the observed strong coupling of particle diffusion and density fluctuations in real glasses.

Edge mode amplification in disordered elastic networks.

Understanding how mechanical systems can be designed to efficiently transport elastic information is important in a variety of fields, including in materials science and biology. Recently, it has been discovered that certain crystalline lattices present "topologically-protected" edge modes that can amplify elastic signals. Several observations suggest that edge modes are important in disordered systems as well, an effect not well understood presently. Here we build a theory of edge modes in disordered isostatic materials and compute the distribution g(κ) of Lyapunov exponents κ characterizing how modes penetrate in the bulk, and find good agreement with numerical results. We show that disordered isostatic materials generically act as levers with amplification of an order LL where L is the system size, whereas more connected materials amplify signals only close to free surfaces. Our approach, which is based on recent results in "free" random matrix theory, makes an analogy with electronic transport in a disordered conductor.

Scaling description of non-local rheology.

The plastic flow of amorphous materials displays non-local effects, characterized by a cooperativity length scale ξ. We argue that these effects enter in the more general description of surface phenomena near critical points. Using this approach, we obtain a scaling relation between exponents that describe the strain rate profiles in shear driven and pressure driven flow, which we confirm both in numerical models and experimental data. We find empirically that the cooperative length follows closely the characteristic length previously extracted in homogenous bulk flows. This analysis shows that the often used mean field exponents fail to capture quantitatively the non-local effects. Our analysis also explains the unusually large finite size effects previously observed in pressure driven flows.

Force distribution affects vibrational properties in hard-sphere glasses.

We theoretically and numerically study the elastic properties of hard-sphere glasses and provide a real-space description of their mechanical stability. In contrast to repulsive particles at zero temperature, we argue that the presence of certain pairs of particles interacting with a small force f soften elastic properties. This softening affects the exponents characterizing elasticity at high pressure, leading to experimentally testable predictions. Denoting P(f) ~ f(θ(e)), the force distribution of such pairs and ϕ(c) the packing fraction at which pressure diverges, we predict that (i) the density of states has a low-frequency peak at a scale ω*, rising up to it as D(ω) ~ ω(2+a), and decaying above ω* as D(ω) ~ ω(-a) where a = (1 - θ(e))/(3 + θ(e)) and ω is the frequency, (ii) shear modulus and mean-squared displacement are inversely proportional with ⟨δR²⟩ ~ 1/µ ~ (ϕ(c) - ϕ)(κ), where κ = 2 - 2/(3 + θ(e)), and (iii) continuum elasticity breaks down on a scale ℓ(c) ~ 1/√(δz) ~ (ϕ(c) - ϕ)(-b), where b = (1 + θ(e))/(6 + 2θ(e)) and δz = z - 2d, where z is the coordination and d the spatial dimension. We numerically test (i) and provide data supporting that θ(e) ≈ 0.41 in our bidisperse system, independently of system preparation in two and three dimensions, leading to κ ≈ 1.41, a ≈ 0.17, and b ≈ 0.21. Our results for the mean-square displacement are consistent with a recent exact replica computation for d = ∞, whereas some observations differ, as rationalized by the present approach.

Scaling description of the yielding transition in soft amorphous solids at zero temperature.

Yield stress materials flow if a sufficiently large shear stress is applied. Although such materials are ubiquitous and relevant for industry, there is no accepted microscopic description of how they yield, even in the simplest situations in which temperature is negligible and in which flow inhomogeneities such as shear bands or fractures are absent. Here we propose a scaling description of the yielding transition in amorphous solids made of soft particles at zero temperature. Our description makes a connection between the Herschel-Bulkley exponent characterizing the singularity of the flow curve near the yield stress Σc, the extension and duration of the avalanches of plasticity observed at threshold, and the density P(x) of soft spots, or shear transformation zones, as a function of the stress increment x beyond which they yield. We argue that the critical exponents of the yielding transition may be expressed in terms of three independent exponents, θ, df, and z, characterizing, respectively, the density of soft spots, the fractal dimension of the avalanches, and their duration. Our description shares some similarity with the depinning transition that occurs when an elastic manifold is driven through a random potential, but also presents some striking differences. We test our arguments in an elasto-plastic model, an automaton model similar to those used in depinning, but with a different interaction kernel, and find satisfying agreement with our predictions in both two and three dimensions.

Evidence for Marginal Stability in Emulsions.

We report the first measurements of the effect of pressure on vibrational modes in emulsions, which serve as a model for soft frictionless spheres at zero temperature. As a function of the applied pressure, we find that the density of states D(ω) exhibits a low-frequency cutoff ω^{*}, which scales linearly with the number of extra contacts per particle δz. Moreover, for ω<ω^{*}, our results are consistent with D(ω)∼ω^{2}/ω^{*2}, a quadratic behavior whose prefactor is larger than what is expected from Debye theory. This surprising result agrees with recent theoretical findings [E. DeGiuli, A. Laversanne-Finot, G. A. Düring, E. Lerner, and M. Wyart, Soft Matter 10, 5628 (2014); S. Franz, G. Parisi, P. Urbani, and F. Zamponi, Proc. Natl. Acad. Sci. U.S.A. 112, 14539 (2015)]. Finally, the degree of localization of the softest low frequency modes increases with compression, as shown by the participation ratio as well as their spatial configurations. Overall, our observations show that emulsions are marginally stable and display non-plane-wave modes up to vanishing frequencies.

Why glass elasticity affects the thermodynamics and fragility of supercooled liquids.

Supercooled liquids are characterized by their fragility: The slowing down of the dynamics under cooling is more sudden and the jump of specific heat at the glass transition is generally larger in fragile liquids than in strong ones. Despite the importance of this quantity in classifying liquids, explaining what aspects of the microscopic structure controls fragility remains a challenge. Surprisingly, experiments indicate that the linear elasticity of the glass--a purely local property of the free energy landscape--is a good predictor of fragility. In particular, materials presenting a large excess of soft elastic modes, the so-called boson peak, are strong. This is also the case for network liquids near the rigidity percolation, known to affect elasticity. Here we introduce a model of the glass transition based on the assumption that particles can organize locally into distinct configurations that are coupled spatially via elasticity. The model captures the mentioned observations connecting elasticity and fragility. We find that materials presenting an abundance of soft elastic modes have little elastic frustration: Energy is insensitive to most directions in phase space, leading to a small jump of specific heat. In this framework strong liquids turn out to lie the closest to a critical point associated with a rigidity or jamming transition, and their thermodynamic properties are related to the problem of number partitioning and to Hopfield nets in the limit of small memory.