Quantifying Long-Range Interactions and Coherent Structure in Multi-Agent Dynamics.

We develop and apply several novel methods quantifying dynamic multi-agent team interactions. These interactions are detected information-theoretically and captured in two ways: via (i) directed networks (interaction diagrams) representing significant coupled dynamics between pairs of agents, and (ii) state-space plots (coherence diagrams) showing coherent structures in Shannon information dynamics. This model-free analysis relates, on the one hand, the information transfer to responsiveness of the agents and the team, and, on the other hand, the information storage within the team to the team's rigidity and lack of tactical flexibility. The resultant interaction and coherence diagrams reveal implicit interactions, across teams, that may be spatially long-range. The analysis was verified with a statistically significant number of experiments (using simulated football games, produced during RoboCup 2D Simulation League matches), identifying the zones of the most intense competition, the extent and types of interactions, and the correlation between the strength of specific interactions and the results of the matches.

Complex cells decrease errors for the Müller-Lyer illusion in a model of the visual ventral stream.

To improve robustness in object recognition, many artificial visual systems imitate the way in which the human visual cortex encodes object information as a hierarchical set of features. These systems are usually evaluated in terms of their ability to accurately categorize well-defined, unambiguous objects and scenes. In the real world, however, not all objects and scenes are presented clearly, with well-defined labels and interpretations. Visual illusions demonstrate a disparity between perception and objective reality, allowing psychophysicists to methodically manipulate stimuli and study our interpretation of the environment. One prominent effect, the Müller-Lyer illusion, is demonstrated when the perceived length of a line is contracted (or expanded) by the addition of arrowheads (or arrow-tails) to its ends. HMAX, a benchmark object recognition system, consistently produces a bias when classifying Müller-Lyer images. HMAX is a hierarchical, artificial neural network that imitates the "simple" and "complex" cell layers found in the visual ventral stream. In this study, we perform two experiments to explore the Müller-Lyer illusion in HMAX, asking: (1) How do simple vs. complex cell operations within HMAX affect illusory bias and precision? (2) How does varying the position of the figures in the input image affect classification using HMAX? In our first experiment, we assessed classification after traversing each layer of HMAX and found that in general, kernel operations performed by simple cells increase bias and uncertainty while max-pooling operations executed by complex cells decrease bias and uncertainty. In our second experiment, we increased variation in the positions of figures in the input images that reduced bias and uncertainty in HMAX. Our findings suggest that the Müller-Lyer illusion is exacerbated by the vulnerability of simple cell operations to positional fluctuations, but ameliorated by the robustness of complex cell responses to such variance.

The Müller-Lyer Illusion in a computational model of biological object recognition.

Studying illusions provides insight into the way the brain processes information. The Müller-Lyer Illusion (MLI) is a classical geometrical illusion of size, in which perceived line length is decreased by arrowheads and increased by arrowtails. Many theories have been put forward to explain the MLI, such as misapplied size constancy scaling, the statistics of image-source relationships and the filtering properties of signal processing in primary visual areas. Artificial models of the ventral visual processing stream allow us to isolate factors hypothesised to cause the illusion and test how these affect classification performance. We trained a feed-forward feature hierarchical model, HMAX, to perform a dual category line length judgment task (short versus long) with over 90% accuracy. We then tested the system in its ability to judge relative line lengths for images in a control set versus images that induce the MLI in humans. Results from the computational model show an overall illusory effect similar to that experienced by human subjects. No natural images were used for training, implying that misapplied size constancy and image-source statistics are not necessary factors for generating the illusion. A post-hoc analysis of response weights within a representative trained network ruled out the possibility that the illusion is caused by a reliance on information at low spatial frequencies. Our results suggest that the MLI can be produced using only feed-forward, neurophysiological connections.

Information processing in echo state networks at the edge of chaos.

We investigate information processing in randomly connected recurrent neural networks. It has been shown previously that the computational capabilities of these networks are maximized when the recurrent layer is close to the border between a stable and an unstable dynamics regime, the so called edge of chaos. The reasons, however, for this maximized performance are not completely understood. We adopt an information-theoretical framework and are for the first time able to quantify the computational capabilities between elements of these networks directly as they undergo the phase transition to chaos. Specifically, we present evidence that both information transfer and storage in the recurrent layer are maximized close to this phase transition, providing an explanation for why guiding the recurrent layer toward the edge of chaos is computationally useful. As a consequence, our study suggests self-organized ways of improving performance in recurrent neural networks, driven by input data. Moreover, the networks we study share important features with biological systems such as feedback connections and online computation on input streams. A key example is the cerebral cortex, which was shown to also operate close to the edge of chaos. Consequently, the behavior of model systems as studied here is likely to shed light on reasons why biological systems are tuned into this specific regime.

Relating Fisher information to order parameters.

We study phase transitions and relevant order parameters via statistical estimation theory using the Fisher information matrix. The assumptions that we make limit our analysis to order parameters representable as a negative derivative of thermodynamic potential over some thermodynamic variable. Nevertheless, the resulting representation is sufficiently general and explicitly relates elements of the Fisher information matrix to the rate of change in the corresponding order parameters. The obtained relationships allow us to identify, in particular, second-order phase transitions via divergences of individual elements of the Fisher information matrix. A computational study of random Boolean networks supports the derived relationships, illustrating that Fisher information of the magnetization bias (that is, activity level) is peaked in finite-size networks at the critical points, and the maxima increase with the network size. The framework presented here reveals the basic thermodynamic reasons behind similar empirical observations reported previously. The study highlights the generality of Fisher information as a measure that can be applied to a broad range of systems, particularly those where the determination of order parameters is cumbersome.

Initialization and self-organized optimization of recurrent neural network connectivity.

Reservoir computing (RC) is a recent paradigm in the field of recurrent neural networks. Networks in RC have a sparsely and randomly connected fixed hidden layer, and only output connections are trained. RC networks have recently received increased attention as a mathematical model for generic neural microcircuits to investigate and explain computations in neocortical columns. Applied to specific tasks, their fixed random connectivity, however, leads to significant variation in performance. Few problem-specific optimization procedures are known, which would be important for engineering applications, but also in order to understand how networks in biology are shaped to be optimally adapted to requirements of their environment. We study a general network initialization method using permutation matrices and derive a new unsupervised learning rule based on intrinsic plasticity (IP). The IP-based learning uses only local learning, and its aim is to improve network performance in a self-organized way. Using three different benchmarks, we show that networks with permutation matrices for the reservoir connectivity have much more persistent memory than the other methods but are also able to perform highly nonlinear mappings. We also show that IP-based on sigmoid transfer functions is limited concerning the output distributions that can be achieved.

Patterns and costs of printed and online journal usage.

This study in an academic medical sciences library setting examines the correlation of usage of a matched set of print and online titles, the validity of e-journals usage statistics and the impact of online journals on print journal usage. The print and online usage was determined for 270 journals, both versions of which were available. Print usage was determined annually since 1997 using the reshelving and the error-copies method. Online usage statistics were delivered by five publishers and corrected for redundant multiple accesses. Print journal usage decreased by 22.3 and 30.2% respectively over each of 2 years after the introduction of online journals. Journals published both in print and online lost 30.4% of their print usage within 2 years. The total loss of usage of print-only titles in the same period was somewhat higher, at 45.8%. The average correlation between online and print usage is 0.60 and 0.67 respectively. For the examined titles, users accessed the online versions ten times as often as the print version. Two clearly distinguishable groupings emerged: while with Academic Press and Elsevier, e-journal usage exceeded print usage by a factor of 3 or 4, the e-journals of Blackwell, HighWire and Springer were used on average 14.6 times as frequently as the corresponding print journals. Each usage of a print article cost 2.79-50.82 Euro, each usage of an online article 0.31-15.10 Euro, depending on the publisher. On average, the usage of an online article was 5.4 times cheaper. Within 2-3 years the usage of online journals has outstripped that of print titles by a factor of ten, but the specific spectrum of usage remains much the same as when only print journals alone existed. Print titles not available online suffer a greater decline in usage compared with print/online journals. This confirms that what is read or purchased is determined primarily by ease of access and that there is a steady tendency to reduce the multiplicity of access modes to a manageable few. The availability of journals online seems to have created a new clientele, at least in the case of the German-language Springer journals. The connection between supplier and supply is much less clear with e-journals than it is with print titles. Therefore it is very important to stress and encourage the role of the library as the supplier of this sort of information in the university environment. Collection building issues are discussed in the light of the results.