Estimating Distributions of Parameters in Nonlinear State Space Models with Replica Exchange Particle Marginal Metropolis-Hastings Method.

Extracting latent nonlinear dynamics from observed time-series data is important for understanding a dynamic system against the background of the observed data. A state space model is a probabilistic graphical model for time-series data, which describes the probabilistic dependence between latent variables at subsequent times and between latent variables and observations. Since, in many situations, the values of the parameters in the state space model are unknown, estimating the parameters from observations is an important task. The particle marginal Metropolis-Hastings (PMMH) method is a method for estimating the marginal posterior distribution of parameters obtained by marginalization over the distribution of latent variables in the state space model. Although, in principle, we can estimate the marginal posterior distribution of parameters by iterating this method infinitely, the estimated result depends on the initial values for a finite number of times in practice. In this paper, we propose a replica exchange particle marginal Metropolis-Hastings (REPMMH) method as a method to improve this problem by combining the PMMH method with the replica exchange method. By using the proposed method, we simultaneously realize a global search at a high temperature and a local fine search at a low temperature. We evaluate the proposed method using simulated data obtained from the Izhikevich neuron model and Lévy-driven stochastic volatility model, and we show that the proposed REPMMH method improves the problem of the initial value dependence in the PMMH method, and realizes efficient sampling of parameters in the state space models compared with existing methods.

Lattice Glass Model in Three Spatial Dimensions.

The understanding of thermodynamic glass transition has been hindered by the lack of proper models beyond mean-field theories. Here, we propose a three-dimensional lattice glass model on a simple cubic lattice that exhibits the typical dynamics observed in fragile supercooled liquids such as two-step relaxation, super-Arrhenius growth in the relaxation time, and dynamical heterogeneity. Using advanced Monte Carlo methods, we compute the thermodynamic properties deep inside the glassy temperature regime, well below the onset temperature of the slow dynamics. The specific heat has a finite jump towards the thermodynamic limit with critical exponents close to those expected from the hyperscaling and the random first-order transition theory for the glass transition. We also study an effective free energy of glasses, the Franz-Parisi potential, as a function of the overlap between equilibrium and quenched configurations. The effective free energy indicates the existence of a first-order phase transition, consistent with the random first-order transition theory. These findings strongly suggest that the glassy dynamics of the model has its origin in thermodynamics.

Power-law decay in the nonadiabatic photodissociation dynamics of alkali halides due to quantum wavepacket interference.

The nonadiabatic photodissociation dynamics of alkali halide molecules excited by a femtosecond laser pulse in the gas phase are investigated theoretically, and it is shown that the population of the photoexcited molecules exhibits power-law decay with exponent -1/2, in contrast to exponential decay, which is often assumed in femtosecond spectroscopy and unimolecular reaction theory. To elucidate the mechanism of the power-law decay, a diagrammatic method that visualizes the structure of the nonadiabatic reaction dynamics as a pattern of occurrence of dynamical events, such as wavepacket bifurcation, turning, and dissociation, is developed. Using this diagrammatic method, an analytical formula for the power-law decay is derived, and the theoretical decay curve is compared with the corresponding numerical decay curve computed by a wavepacket dynamics simulation in the case of lithium fluoride. This study reveals that the cause of the power-law decay is the quantum interference arising from the wavepacket bifurcation and merging due to nonadiabatic transitions.

Emergence of compact disordered phase in a polymer Potts model.

One of the central problems in epigenetics is how epigenetic modification patterns and chromatin structure are regulated in the cell nucleus. The polymer Potts model, a recently studied model of chromatins, is introduced with an offset in the interaction energy as a parameter, and the equilibrium properties are investigated using the mean-field analysis of the lattice model and molecular dynamics simulations of the off-lattice model. The results show that in common with both models, a phase emerges, which could be called the compact-disordered phase, in which the polymer conformation is compact and the epigenetic modification pattern is disordered, depending on the offset in the interaction energy and the fraction of the modified nucleosomes.

Effect of constraint relaxation on the minimum vertex cover problem in random graphs.

A statistical-mechanical study of the effect of constraint relaxation on the minimum vertex cover problem in Erdos-Rényi random graphs is presented. Using a penalty-method formulation for constraint relaxation, typical properties of solutions, including infeasible solutions that violate the constraints, are analyzed by means of the replica method and cavity method. The problem involves a competition between reducing the number of vertices to be covered and satisfying the edge constraints. The analysis under the replica-symmetric (RS) ansatz clarifies that the competition leads to degeneracies in the vertex and edge states, which determine the quantitative properties of the system, such as the cover and penalty ratios. A precise analysis of these effects improves the accuracy of RS approximation for the minimum cover ratio in the replica symmetry-breaking (RSB) region. Furthermore, the analysis based on the RS cavity method indicates that the RS/RSB boundary of the ground states with respect to the mean degree of the graphs is expanded, and the critical temperature is lowered by constraint relaxation.

Phase transition in compressed sensing with horseshoe prior.

In Bayesian statistics, horseshoe prior has attracted increasing attention as an approach to compressed sensing. By considering compressed sensing as a randomly correlated many-body problem, statistical mechanics methods can be used to analyze the problem. In this paper, the estimation accuracy of compressed sensing with the horseshoe prior is evaluated by the statistical mechanical methods of random systems. It is found that there exists a phase transition in signal recoverability in the plane of the number of observations and the number of nonzero signals, and that the recoverable phase is more extended than that using the well-known l_{1} norm regularization.

Higher-order tensor renormalization group study of the J_{1}-J_{2} Ising model on a square lattice.

Phase transitions of the J_{1}-J_{2} Ising model on a square lattice are studied using the higher-order tensor renormalization group (HOTRG) method. This system involves a competition between the ferromagnetic interaction J_{1} and antiferromagnetic interaction J_{2}, and in previous studies, weak first-order and second-order transitions were observed near the ratio g=J_{2}/|J_{1}|=1/2. It has also been suggested that the universality class of the second-order phase transition connected to the first-order transition line for g>1/2 belongs to the Ashkin-Teller class, which is characterized by a continuously varying critical exponent with g, as predicted by field-theoretical and other studies. Our results, based on the HOTRG calculations for significantly larger sizes, indicate that the region of the first-order transition is marginally narrower than that in previous studies. Furthermore, it is suggested that the region where the critical exponent changes does not necessarily coincide with the Ashkin-Teller region.

Nonaffine displacements below jamming under athermal quasistatic compression.

Critical properties of frictionless spherical particles below jamming are studied using extensive numerical simulations, paying particular attention to the nonaffine part of the displacements during the athermal quasistatic compression. It is shown that the squared norm of the nonaffine displacement exhibits a power-law divergence toward the jamming transition point. A possible connection between this critical exponent and that of the shear viscosity is discussed. The participation ratio of the displacements vanishes in the thermodynamic limit at the transition point, meaning that the nonaffine displacements are localized marginally with a fractal dimension. Furthermore, the distribution of the displacement is shown to have a power-law tail, the exponent of which is related to the fractal dimension.

Bayesian optimization for computationally extensive probability distributions.

An efficient method for finding a better maximizer of computationally extensive probability distributions is proposed on the basis of a Bayesian optimization technique. A key idea of the proposed method is to use extreme values of acquisition functions by Gaussian processes for the next training phase, which should be located near a local maximum or a global maximum of the probability distribution. Our Bayesian optimization technique is applied to the posterior distribution in the effective physical model estimation, which is a computationally extensive probability distribution. Even when the number of sampling points on the posterior distributions is fixed to be small, the Bayesian optimization provides a better maximizer of the posterior distributions in comparison to those by the random search method, the steepest descent method, or the Monte Carlo method. Furthermore, the Bayesian optimization improves the results efficiently by combining the steepest descent method and thus it is a powerful tool to search for a better maximizer of computationally extensive probability distributions.

Typical approximation performance for maximum coverage problem.

This study investigated the typical performance of approximation algorithms known as belief propagation, the greedy algorithm, and linear-programming relaxation for maximum coverage problems in sparse biregular random graphs. After we used the cavity method for a corresponding hard-core lattice-gas model, results showed that two distinct thresholds of replica-symmetry and its breaking exist in the typical performance threshold of belief propagation. In the low-density region, the superiority of three algorithms in terms of a typical performance threshold is obtained by some theoretical analyses. Although the greedy algorithm and linear-programming relaxation have the same approximation ratio in worst-case performance, their typical performance thresholds are mutually different, indicating the importance of typical performance. Results of numerical simulations validate the theoretical analyses and imply further mutual relations of approximation algorithms.

Statistical mechanical analysis of linear programming relaxation for combinatorial optimization problems.

Typical behavior of the linear programming (LP) problem is studied as a relaxation of the minimum vertex cover (min-VC), a type of integer programming (IP) problem. A lattice-gas model on the Erdös-Rényi random graphs of α-uniform hyperedges is proposed to express both the LP and IP problems of the min-VC in the common statistical mechanical model with a one-parameter family. Statistical mechanical analyses reveal for α=2 that the LP optimal solution is typically equal to that given by the IP below the critical average degree c=e in the thermodynamic limit. The critical threshold for good accuracy of the relaxation extends the mathematical result c=1 and coincides with the replica symmetry-breaking threshold of the IP. The LP relaxation for the minimum hitting sets with α≥3, minimum vertex covers on α-uniform random graphs, is also studied. Analytic and numerical results strongly suggest that the LP relaxation fails to estimate optimal values above the critical average degree c=e/(α-1) where the replica symmetry is broken.

Eigenvalue analysis of an irreversible random walk with skew detailed balance conditions.

An irreversible Markov-chain Monte Carlo (MCMC) algorithm with skew detailed balance conditions originally proposed by Turitsyn et al. is extended to general discrete systems on the basis of the Metropolis-Hastings scheme. To evaluate the efficiency of our proposed method, the relaxation dynamics of the slowest mode and the asymptotic variance are studied analytically in a random walk on one dimension. It is found that the performance in irreversible MCMC methods violating the detailed balance condition is improved by appropriately choosing parameters in the algorithm.

Bayesian inversion analysis of nonlinear dynamics in surface heterogeneous reactions.

It is essential to extract nonlinear dynamics from time-series data as an inverse problem in natural sciences. We propose a Bayesian statistical framework for extracting nonlinear dynamics of surface heterogeneous reactions from sparse and noisy observable data. Surface heterogeneous reactions are chemical reactions with conjugation of multiple phases, and they have the intrinsic nonlinearity of their dynamics caused by the effect of surface-area between different phases. We adapt a belief propagation method and an expectation-maximization (EM) algorithm to partial observation problem, in order to simultaneously estimate the time course of hidden variables and the kinetic parameters underlying dynamics. The proposed belief propagation method is performed by using sequential Monte Carlo algorithm in order to estimate nonlinear dynamical system. Using our proposed method, we show that the rate constants of dissolution and precipitation reactions, which are typical examples of surface heterogeneous reactions, as well as the temporal changes of solid reactants and products, were successfully estimated only from the observable temporal changes in the concentration of the dissolved intermediate product.

Evidence of a one-step replica symmetry breaking in a three-dimensional Potts glass model.

We study a seven-state Potts glass model in three dimensions with first-, second-, and third-nearest-neighbor interactions with a bimodal distribution of couplings by Monte Carlo simulations. Our results show the existence of a spin-glass transition at a finite temperature T(c), a discontinuous jump of an order parameter at T(c) without latent heat, and a nontrivial structure in the order parameter distribution below T(c). They are compatible with one-step replica symmetry breaking.

Free-energy landscape and nucleation pathway of polymorphic minerals from solution in a Potts lattice-gas model.

Metastable minerals commonly form during reactions between water and rock. The nucleation mechanism of polymorphic phases from solution are explored here using a two-dimensional Potts model. The model system is composed of a solvent and three polymorphic solid phases. The local state and position of the solid phase are updated by Metropolis dynamics. Below the critical temperature, a large cluster of the least stable solid phase initially forms in the solution before transitioning into more-stable phases following the Ostwald step rule. The free-energy landscape as a function of the modal abundance of each solid phase clearly reveals that before cluster formation, the least stable phase has an energetic advantage because of its low interfacial energy with the solution, and after cluster formation, phase transformation occurs along the valley of the free-energy landscape, which contains several minima for the regions of three phases. Our results indicate that the solid-solid and solid-liquid interfacial energy contribute to the formation of the complex free-energy landscape and nucleation pathways following the Ostwald step rule.

Event-chain algorithm for the Heisenberg model: Evidence for z≃1 dynamic scaling.

We apply the event-chain Monte Carlo algorithm to the three-dimensional ferromagnetic Heisenberg model. The algorithm is rejection-free and also realizes an irreversible Markov chain that satisfies global balance. The autocorrelation functions of the magnetic susceptibility and the energy indicate a dynamical critical exponent z≈1 at the critical temperature, while that of the magnetization does not measure the performance of the algorithm. We show that the event-chain Monte Carlo algorithm substantially reduces the dynamical critical exponent from the conventional value of z≃2.

Minimum vertex cover problems on random hypergraphs: replica symmetric solution and a leaf removal algorithm.

The minimum vertex-cover problems on random α-uniform hypergraphs are studied using two different approaches, a replica method in statistical mechanics of random systems and a leaf removal algorithm. It is found that there exists a phase transition at the critical average degree e/(α-1), below which a replica symmetric ansatz in the replica method holds and the algorithm estimates exactly the same solution of the problem as that by the replica method. In contrast, above the critical degree, the replica symmetric solution becomes unstable and the leaf-removal algorithm fails to estimate the optimal solution because of the emergence of a large size core. These results strongly suggest a close relation between the replica symmetry and the performance of an approximation algorithm. Critical properties of the core percolation are also examined numerically by a finite-size scaling.

Partial annealing of a coupled mean-field spin-glass model with an embedded pattern.

A partially annealed mean-field spin-glass model with a locally embedded pattern is studied. The model consists of two dynamical variables, spins and interactions, that are in contact with thermal baths at temperatures T(S) and T(J), respectively. Unlike the quenched system, characteristic correlations among the interactions are induced by the partial annealing. The model exhibits three phases: paramagnetic, ferromagnetic and spin-glass. In the ferromagnetic phase, the embedded pattern is stably realized. The phase diagram depends significantly on the ratio of the two temperatures, n=T(S)/T(J). In particular, a reentrant transition from the embedded ferromagnetic to the spin-glass phase with T(S) decreasing is found only below a certain value of n. This indicates that above the critical value n(c) the embedded pattern is supported by a local field from a nonembedded region. Some equilibrium properties of the interactions in the partial annealing are also discussed in terms of frustration.

Multicanonical sampling of rare events in random matrices.

A method based on multicanonical Monte Carlo is applied to the calculation of large deviations in the largest eigenvalue of random matrices. The method is successfully tested with the Gaussian orthogonal ensemble, sparse random matrices, and matrices whose components are subject to uniform density. Specifically, the probability that all eigenvalues of a matrix are negative is estimated in these cases down to the values of ∼10(-200), a region where simple random sampling is ineffective. The method can be applied to any ensemble of matrices and used for sampling rare events characterized by any statistics.

Funnel landscape and mutational robustness as a result of evolution under thermal noise.

Using a statistical-mechanical model of spins, the evolution of phenotype dynamics is studied. Configurations of spins and their interaction J represent the phenotype and genotype, respectively. The fitness for selection of J is given by the equilibrium spin configurations determined by a Hamiltonian with J under thermal noise. The genotype J evolves through mutational changes under selection pressure to raise its fitness value. From Monte Carlo simulations we find that the frustration around the target spins disappears for J evolved under temperature beyond a certain threshold. The evolved Js give the funnel-like dynamics, which is robust to noise and also to mutation.