Information Engine in a Nonequilibrium Bath.

Information engines can convert thermal fluctuations of a bath at temperature T into work at rates of order k_{B}T per relaxation time of the system. We show experimentally that such engines, when in contact with a bath that is out of equilibrium, can extract much more work. We place a heavy, micron-scale bead in a harmonic potential that ratchets up to capture favorable fluctuations. Adding a fluctuating electric field increases work extraction up to ten times, limited only by the strength of the applied field. Our results connect Maxwell's demon with energy harvesting and demonstrate that information engines in nonequilibrium baths can greatly outperform conventional engines.

Optimal work extraction and mutual information in a generalized Szilárd engine.

A 1929 Gedankenexperiment proposed by Szilárd, often referred to as "Szilárd's engine", has served as a foundation for computing fundamental thermodynamic bounds to information processing. While Szilárd's original box could be partitioned into two halves and contains one gas molecule, we calculate here the maximal average work that can be extracted in a system with N particles and q partitions, given an observer which counts the molecules in each partition, and given a work extraction mechanism that is limited to pressure equalization. We find that the average extracted work is proportional to the mutual information between the one-particle position and the vector containing the counts of how many particles are in each partition. We optimize this quantity over the initial locations of the dividing walls, and find that there exists a critical number of particles N^{★}(q) below which the extracted work is maximized by a symmetric configuration of the q partitions, and above which the optimal partitioning is asymmetric. Overall, the average extracted work is maximized for a number of particles N[over ̂](q)

Thermodynamic Cost and Benefit of Memory.

This Letter exposes a tight connection between the thermodynamic efficiency of information processing and predictive inference. A generalized lower bound on dissipation is derived for partially observable information engines which are allowed to use temperature differences. It is shown that the retention of irrelevant information limits efficiency. A data representation method is derived from optimizing a fundamental physical limit to information processing: minimizing the lower bound on dissipation leads to a compression method that maximally retains relevant, predictive, information. In that sense, predictive inference emerges as the strategy that least precludes energy efficiency.

Nonequilibrium abundances for the building blocks of life.

The difficulty of obtaining appreciable quantities of biologically important molecules in thermodynamic equilibrium has long been identified as an obstacle to life's emergence, and determining the specific nonequilibrium conditions that might have given rise to life is challenging. To address these issues, we investigate how the concentrations of life's building blocks change as a function of the distance from equilibrium on average, in two example settings: (i) the synthesis of heavy amino acids and (ii) their polymerization into peptides. We find that relative concentrations of the heaviest amino acids can be boosted by four orders of magnitude, and concentrations of the longest peptide chains can be increased by hundreds of orders of magnitude. The average nonequilibrium distribution does not depend on the details of how the system was driven from equilibrium, indicating that environments might not have to be fine-tuned to support life.

Physical limitations of work extraction from temporal correlations.

Recently proposed information-exploiting systems extract work from a single heat bath by using temporal correlations on an input tape. We study how enforcing time-continuous dynamics, which is necessary to ensure that the device is physically realizable, constrains possible designs and drastically diminishes efficiency. We show that these problems can be circumvented by means of applying an external, time-varying protocol, which turns the device from a "passive," free-running machine into an "actively" driven one.

Thermodynamics of prediction.

A system responding to a stochastic driving signal can be interpreted as computing, by means of its dynamics, an implicit model of the environmental variables. The system's state retains information about past environmental fluctuations, and a fraction of this information is predictive of future ones. The remaining nonpredictive information reflects model complexity that does not improve predictive power, and thus represents the ineffectiveness of the model. We expose the fundamental equivalence between this model inefficiency and thermodynamic inefficiency, measured by dissipation. Our results hold arbitrarily far from thermodynamic equilibrium and are applicable to a wide range of systems, including biomolecular machines. They highlight a profound connection between the effective use of information and efficient thermodynamic operation: any system constructed to keep memory about its environment and to operate with maximal energetic efficiency has to be predictive.

An information-theoretic approach to curiosity-driven reinforcement learning.

We provide a fresh look at the problem of exploration in reinforcement learning, drawing on ideas from information theory. First, we show that Boltzmann-style exploration, one of the main exploration methods used in reinforcement learning, is optimal from an information-theoretic point of view, in that it optimally trades expected return for the coding cost of the policy. Second, we address the problem of curiosity-driven learning. We propose that, in addition to maximizing the expected return, a learner should choose a policy that also maximizes the learner's predictive power. This makes the world both interesting and exploitable. Optimal policies then have the form of Boltzmann-style exploration with a bonus, containing a novel exploration-exploitation trade-off which emerges naturally from the proposed optimization principle. Importantly, this exploration-exploitation trade-off persists in the optimal deterministic policy, i.e., when there is no exploration due to randomness. As a result, exploration is understood as an emerging behavior that optimizes information gain, rather than being modeled as pure randomization of action choices.

Optimal causal inference: estimating stored information and approximating causal architecture.

We introduce an approach to inferring the causal architecture of stochastic dynamical systems that extends rate-distortion theory to use causal shielding--a natural principle of learning. We study two distinct cases of causal inference: optimal causal filtering and optimal causal estimation. Filtering corresponds to the ideal case in which the probability distribution of measurement sequences is known, giving a principled method to approximate a system's causal structure at a desired level of representation. We show that in the limit in which a model-complexity constraint is relaxed, filtering finds the exact causal architecture of a stochastic dynamical system, known as the causal-state partition. From this, one can estimate the amount of historical information the process stores. More generally, causal filtering finds a graded model-complexity hierarchy of approximations to the causal architecture. Abrupt changes in the hierarchy, as a function of approximation, capture distinct scales of structural organization. For nonideal cases with finite data, we show how the correct number of the underlying causal states can be found by optimal causal estimation. A previously derived model-complexity control term allows us to correct for the effect of statistical fluctuations in probability estimates and thereby avoid overfitting.

Neuromorphic walking gait control.

We present a neuromorphic pattern generator for controlling the walking gaits of four-legged robots which is inspired by central pattern generators found in the nervous system and which is implemented as a very large scale integrated (VLSI) chip. The chip contains oscillator circuits that mimic the output of motor neurons in a strongly simplified way. We show that four coupled oscillators can produce rhythmic patterns with phase relationships that are appropriate to generate all four-legged animal walking gaits. These phase relationships together with frequency and duty cycle of the oscillators determine the walking behavior of a robot driven by the chip, and they depend on a small set of stationary bias voltages. We give analytic expressions for these dependencies. This chip reduces the complex, dynamic inter-leg control problem associated with walking gait generation to the problem of setting a few stationary parameters. It provides a compact and low power solution for walking gait control in robots.

How many clusters? An information-theoretic perspective.

Clustering provides a common means of identifying structure in complex data, and there is renewed interest in clustering as a tool for the analysis of large data sets in many fields. A natural question is how many clusters are appropriate for the description of a given system. Traditional approaches to this problem are based on either a framework in which clusters of a particular shape are assumed as a model of the system or on a two-step procedure in which a clustering criterion determines the optimal assignments for a given number of clusters and a separate criterion measures the goodness of the classification to determine the number of clusters. In a statistical mechanics approach, clustering can be seen as a trade-off between energy- and entropy-like terms, with lower temperature driving the proliferation of clusters to provide a more detailed description of the data. For finite data sets, we expect that there is a limit to the meaningful structure that can be resolved and therefore a minimum temperature beyond which we will capture sampling noise. This suggests that correcting the clustering criterion for the bias that arises due to sampling errors will allow us to find a clustering solution at a temperature that is optimal in the sense that we capture maximal meaningful structure--without having to define an external criterion for the goodness or stability of the clustering. We show that in a general information-theoretic framework, the finite size of a data set determines an optimal temperature, and we introduce a method for finding the maximal number of clusters that can be resolved from the data in the hard clustering limit.

Network information and connected correlations.

Entropy and information provide natural measures of correlation among elements in a network. We construct here the information theoretic analog of connected correlation functions: irreducible N-point correlation is measured by a decrease in entropy for the joint distribution of N variables relative to the maximum entropy allowed by all the observed N-1 variable distributions. We calculate the "connected information" terms for several examples and show that it also enables the decomposition of the information that is carried by a population of elements about an outside source.