NMDA-driven dendritic modulation enables multitask representation learning in hierarchical sensory processing pathways.

While sensory representations in the brain depend on context, it remains unclear how such modulations are implemented at the biophysical level, and how processing layers further in the hierarchy can extract useful features for each possible contextual state. Here, we demonstrate that dendritic N-Methyl-D-Aspartate spikes can, within physiological constraints, implement contextual modulation of feedforward processing. Such neuron-specific modulations exploit prior knowledge, encoded in stable feedforward weights, to achieve transfer learning across contexts. In a network of biophysically realistic neuron models with context-independent feedforward weights, we show that modulatory inputs to dendritic branches can solve linearly nonseparable learning problems with a Hebbian, error-modulated learning rule. We also demonstrate that local prediction of whether representations originate either from different inputs, or from different contextual modulations of the same input, results in representation learning of hierarchical feedforward weights across processing layers that accommodate a multitude of contexts.

Mermin-Wagner fluctuations in 2D amorphous solids.

In a recent commentary, J. M. Kosterlitz described how D. Thouless and he got motivated to investigate melting and suprafluidity in two dimensions [Kosterlitz JM (2016) J Phys Condens Matter 28:481001]. It was due to the lack of broken translational symmetry in two dimensions-doubting the existence of 2D crystals-and the first computer simulations foretelling 2D crystals (at least in tiny systems). The lack of broken symmetries proposed by D. Mermin and H. Wagner is caused by long wavelength density fluctuations. Those fluctuations do not only have structural impact, but additionally a dynamical one: They cause the Lindemann criterion to fail in 2D in the sense that the mean squared displacement of atoms is not limited. Comparing experimental data from 3D and 2D amorphous solids with 2D crystals, we disentangle Mermin-Wagner fluctuations from glassy structural relaxations. Furthermore, we demonstrate with computer simulations the logarithmic increase of displacements with system size: Periodicity is not a requirement for Mermin-Wagner fluctuations, which conserve the homogeneity of space on long scales.

Strain Pattern in Supercooled Liquids.

Investigations of strain correlations at the glass transition reveal unexpected phenomena. The shear strain fluctuations show an Eshelby-strain pattern [∼cos(4θ)/r^{2}], characteristic of elastic response, even in liquids, at long times. We address this using a mode-coupling theory for the strain fluctuations in supercooled liquids and data from both video microscopy of a two-dimensional colloidal glass former and simulations of Brownian hard disks. We show that the long-ranged and long-lived strain signatures follow a scaling law valid close to the glass transition. For large enough viscosities, the Eshelby-strain pattern is visible even on time scales longer than the structural relaxation time τ and after the shear modulus has relaxed to zero.

Biologically plausible deep learning - But how far can we go with shallow networks?

Training deep neural networks with the error backpropagation algorithm is considered implausible from a biological perspective. Numerous recent publications suggest elaborate models for biologically plausible variants of deep learning, typically defining success as reaching around 98% test accuracy on the MNIST data set. Here, we investigate how far we can go on digit (MNIST) and object (CIFAR10) classification with biologically plausible, local learning rules in a network with one hidden layer and a single readout layer. The hidden layer weights are either fixed (random or random Gabor filters) or trained with unsupervised methods (Principal/Independent Component Analysis or Sparse Coding) that can be implemented by local learning rules. The readout layer is trained with a supervised, local learning rule. We first implement these models with rate neurons. This comparison reveals, first, that unsupervised learning does not lead to better performance than fixed random projections or Gabor filters for large hidden layers. Second, networks with localized receptive fields perform significantly better than networks with all-to-all connectivity and can reach backpropagation performance on MNIST. We then implement two of the networks - fixed, localized, random & random Gabor filters in the hidden layer - with spiking leaky integrate-and-fire neurons and spike timing dependent plasticity to train the readout layer. These spiking models achieve >98.2% test accuracy on MNIST, which is close to the performance of rate networks with one hidden layer trained with backpropagation. The performance of our shallow network models is comparable to most current biologically plausible models of deep learning. Furthermore, our results with a shallow spiking network provide an important reference and suggest the use of data sets other than MNIST for testing the performance of future models of biologically plausible deep learning.