Learning dynamic representations of the functional connectome in neurobiological networks.

The static synaptic connectivity of neuronal circuits stands in direct contrast to the dynamics of their function. As in changing community interactions, different neurons can participate actively in various combinations to effect behaviors at different times. We introduce an unsupervised approach to learn the dynamic affinities between neurons in live, behaving animals, and to reveal which communities form among neurons at different times. The inference occurs in two major steps. First, pairwise non-linear affinities between neuronal traces from brain-wide calcium activity are organized by non-negative tensor factorization (NTF). Each factor specifies which groups of neurons are most likely interacting for an inferred interval in time, and for which animals. Finally, a generative model that allows for weighted community detection is applied to the functional motifs produced by NTF to reveal a dynamic functional connectome. Since time codes the different experimental variables (e.g., application of chemical stimuli), this provides an atlas of neural motifs active during separate stages of an experiment (e.g., stimulus application or spontaneous behaviors). Results from our analysis are experimentally validated, confirming that our method is able to robustly predict causal interactions between neurons to generate behavior.

Population encoding of stimulus features along the visual hierarchy.

The retina and primary visual cortex (V1) both exhibit diverse neural populations sensitive to diverse visual features. Yet it remains unclear how neural populations in each area partition stimulus space to span these features. One possibility is that neural populations are organized into discrete groups of neurons, with each group signaling a particular constellation of features. Alternatively, neurons could be continuously distributed across feature-encoding space. To distinguish these possibilities, we presented a battery of visual stimuli to the mouse retina and V1 while measuring neural responses with multi-electrode arrays. Using machine learning approaches, we developed a manifold embedding technique that captures how neural populations partition feature space and how visual responses correlate with physiological and anatomical properties of individual neurons. We show that retinal populations discretely encode features, while V1 populations provide a more continuous representation. Applying the same analysis approach to convolutional neural networks that model visual processing, we demonstrate that they partition features much more similarly to the retina, indicating they are more like big retinas than little brains.

Population encoding of stimulus features along the visual hierarchy.

The retina and primary visual cortex (V1) both exhibit diverse neural populations sensitive to diverse visual features. Yet it remains unclear how neural populations in each area partition stimulus space to span these features. One possibility is that neural populations are organized into discrete groups of neurons, with each group signaling a particular constellation of features. Alternatively, neurons could be continuously distributed across feature-encoding space. To distinguish these possibilities, we presented a battery of visual stimuli to mouse retina and V1 while measuring neural responses with multi-electrode arrays. Using machine learning approaches, we developed a manifold embedding technique that captures how neural populations partition feature space and how visual responses correlate with physiological and anatomical properties of individual neurons. We show that retinal populations discretely encode features, while V1 populations provide a more continuous representation. Applying the same analysis approach to convolutional neural networks that model visual processing, we demonstrate that they partition features much more similarly to the retina, indicating they are more like big retinas than little brains.

IAN: Iterated Adaptive Neighborhoods for Manifold Learning and Dimensionality Estimation.

Invoking the manifold assumption in machine learning requires knowledge of the manifold's geometry and dimension, and theory dictates how many samples are required. However, in most applications, the data are limited, sampling may not be uniform, and the manifold's properties are unknown; this implies that neighborhoods must adapt to the local structure. We introduce an algorithm for inferring adaptive neighborhoods for data given by a similarity kernel. Starting with a locally conservative neighborhood (Gabriel) graph, we sparsify it iteratively according to a weighted counterpart. In each step, a linear program yields minimal neighborhoods globally, and a volumetric statistic reveals neighbor outliers likely to violate manifold geometry. We apply our adaptive neighborhoods to nonlinear dimensionality reduction, geodesic computation, and dimension estimation. A comparison against standard algorithms using, for example, k-nearest neighbors, demonstrates the usefulness of our approach.

From boundaries to bumps: When closed (extremal) contours are critical.

Invariants underlying shape inference are elusive: A variety of shapes can give rise to the same image, and a variety of images can be rendered from the same shape. The occluding contour is a rare exception: It has both image salience, in terms of isophotes, and surface meaning, in terms of surface normal. We relax the notion of occluding contour and, more accurately, the rim on the object that projects to it, to define closed extremal curves. This new shape descriptor is invariant over different renderings. It exists at the topological level, which guarantees an image-based counterpart. It surrounds bumps and dents, as well as common interior shape components, and formalizes the qualitative nature of bump perception. The invariants are biologically computable, unify shape inferences from shading and specular materials, and predict new phenomena in bump and dent perception. Most important, working at the topological level allows us to capture the elusive aspect of bump boundaries.

Flow stimuli reveal ecologically appropriate responses in mouse visual cortex.

Assessments of the mouse visual system based on spatial-frequency analysis imply that its visual capacity is low, with few neurons responding to spatial frequencies greater than 0.5 cycles per degree. However, visually mediated behaviors, such as prey capture, suggest that the mouse visual system is more precise. We introduce a stimulus class-visual flow patterns-that is more like what the mouse would encounter in the natural world than are sine-wave gratings but is more tractable for analysis than are natural images. We used 128-site silicon microelectrodes to measure the simultaneous responses of single neurons in the primary visual cortex (V1) of alert mice. While holding temporal-frequency content fixed, we explored a class of drifting patterns of black or white dots that have energy only at higher spatial frequencies. These flow stimuli evoke strong visually mediated responses well beyond those predicted by spatial-frequency analysis. Flow responses predominate in higher spatial-frequency ranges (0.15-1.6 cycles per degree), many are orientation or direction selective, and flow responses of many neurons depend strongly on sign of contrast. Many cells exhibit distributed responses across our stimulus ensemble. Together, these results challenge conventional linear approaches to visual processing and expand our understanding of the mouse's visual capacity to behaviorally relevant ranges.

Colour, contours, shading and shape: flow interactions reveal anchor neighbourhoods.

Two dilemmas arise in inferring shape information from shading. First, depending on the rendering physics, images can change significantly with (even) small changes in lighting or viewpoint, while the percept frequently does not. Second, brightness variations can be induced by material effects-such as pigmentation-as well as by shading effects. Improperly interpreted, material effects would confound shading effects. We show how these dilemmas are coupled by reviewing recent developments in shape inference together with a role for colour in separating material from shading effects. Aspects of both are represented in a common geometric (flow) framework, and novel displays of hue/shape interaction demonstrate a global effect with interactions limited to localized regions. Not all parts of an image are perceptually equal; shape percepts appear to be constructed from image anchor regions.

Conduction in the Heart Wall: Helicoidal Fibers Minimize Diffusion Bias.

The mammalian heart must function as an efficient pump while simultaneously conducting electrical signals to drive the contraction process. In the ventricles, electrical activation begins at the insertion points of the Purkinje network in the endocardium. How does the diffusion component of the subsequent excitation wave propagate from the endocardium in a healthy heart wall without creating directional biases? We show that this is a consequence of the particular geometric organization of myocytes in the heart wall. Using a generalized helicoid to model fiber orientation, we treat the myocardium as a curved space via Riemannian geometry, and then use stochastic calculus to model local signal diffusion. Our analysis shows that the helicoidal arrangement of myocytes minimizes the directional biases that could lead to aberrant propagation, thereby explaining how electrophysiological principles are consistent with local measurements of cardiac fiber geometry. We discuss our results in the context of the need to balance electrical and mechanical requirements for heart function.

Feedback between motion and sensation provides nonlinear boost in run-and-tumble navigation.

Many organisms navigate gradients by alternating straight motions (runs) with random reorientations (tumbles), transiently suppressing tumbles whenever attractant signal increases. This induces a functional coupling between movement and sensation, since tumbling probability is controlled by the internal state of the organism which, in turn, depends on previous signal levels. Although a negative feedback tends to maintain this internal state close to adapted levels, positive feedback can arise when motion up the gradient reduces tumbling probability, further boosting drift up the gradient. Importantly, such positive feedback can drive large fluctuations in the internal state, complicating analytical approaches. Previous studies focused on what happens when the negative feedback dominates the dynamics. By contrast, we show here that there is a large portion of physiologically-relevant parameter space where the positive feedback can dominate, even when gradients are relatively shallow. We demonstrate how large transients emerge because of non-normal dynamics (non-orthogonal eigenvectors near a stable fixed point) inherent in the positive feedback, and further identify a fundamental nonlinearity that strongly amplifies their effect. Most importantly, this amplification is asymmetric, elongating runs in favorable directions and abbreviating others. The result is a "ratchet-like" gradient climbing behavior with drift speeds that can approach half the maximum run speed of the organism. Our results thus show that the classical drawback of run-and-tumble navigation-wasteful runs in the wrong direction-can be mitigated by exploiting the non-normal dynamics implicit in the run-and-tumble strategy.

Local field potentials and border ownership: A conjecture about computation in visual cortex.

Border ownership is an intermediate-level visual task: it must integrate (upward flowing) image information about edges with (downward flowing) shape information. This highlights the familiar local-to-global aspect of border formation (linking of edge elements to form contours) with the much less studied global-to-local aspect (which edge elements form part of the same shape). To address this task we show how to incorporate certain high-level notions of distance and geometric arrangement into a form that can influence image-based edge information. The center of the argument is a reaction-diffusion equation that reveals how (global) aspects of the distance map (that is, shape) can be "read out" locally, suggesting a solution to the border ownership problem. Since the reaction-diffusion equation defines a field, a possible information processing role for the local field potential can be defined. We argue that such fields also underlie the Gestalt notion of closure, especially when it is refined using modern experimental techniques. An important implication of this theoretical argument is that, if true, then network modeling must be extended to include the substrate surrounding spiking neurons, including glia.

Heart wall myofibers are arranged in minimal surfaces to optimize organ function.

Heart wall myofibers wind as helices around the ventricles, strengthening them in a manner analogous to the reinforcement of concrete cylindrical columns by spiral steel cables [Richart FE, et al. (1929) Univ of Illinois, Eng Exp Stn Bull 190]. A multitude of such fibers, arranged smoothly and regularly, contract and relax as an integrated functional unit as the heart beats. To orchestrate this motion, fiber tangling must be avoided and pumping should be efficient. Current models of myofiber orientation across the heart wall suggest groupings into sheets or bands, but the precise geometry of bundles of myofibers is unknown. Here we show that this arrangement takes the form of a special minimal surface, the generalized helicoid [Blair DE, Vanstone JR (1978) Minimal Submanifolds and Geodesics 13-16], closing the gap between individual myofibers and their collective wall structure. The model holds across species, with a smooth variation in its three curvature parameters within the myocardial wall providing tight fits to diffusion magnetic resonance images from the rat, the dog, and the human. Mathematically it explains how myofibers are bundled in the heart wall while economizing fiber length and optimizing ventricular ejection volume as they contract. The generalized helicoid provides a unique foundation for analyzing the fibrous composite of the heart wall and should therefore find applications in heart tissue engineering and in the study of heart muscle diseases.

Differential geometric inference in surface stereo.

Many traditional two-view stereo algorithms explicitly or implicitly use the frontal parallel plane assumption when exploiting contextual information since, e.g., the smoothness prior biases toward constant disparity (depth) over a neighborhood. This introduces systematic errors to the matching process for slanted or curved surfaces. These errors are nonnegligible for detailed geometric modeling of natural objects such as a human face. We show how to use contextual information geometrically to avoid such errors. A differential geometric study of smooth surfaces allows contextual information to be encoded in Cartan's moving frame model over local quadratic approximations, providing a framework of geometric consistency for both depth and surface normals; the accuracy of our reconstructions argues for the sufficiency of the approximation. In effect, Cartan's model provides the additional constraint necessary to move beyond the frontal parallel plane assumption in stereo reconstruction. It also suggests how geometry can extend surfaces to account for unmatched points due to partial occlusion.

Boundaries, shading, and border ownership: A cusp at their interaction.

The association of borders with "figure" rather than "background" provides a topological organizing principle for early vision. Such global influences have recently been shown to have local effects, with neuronal activity modulated by stimulus properties from well outside the classical receptive field. We extend the theoretical analysis of such phenomena by developing the geometry of interaction between shading, boundaries, and boundary ownership for smooth surfaces. The purely exterior edges of smooth objects enjoy a fold-type relationship between shading and boundary, due to foreshortening, while the background is cut off transversely. However, at cusp points in the image mapping the exterior boundary ends abruptly. Since such singular points are notoriously unstable, we conjecture that this process is regularized by a natural quantization of suggestive contours due to physiological boundary-detection mechanisms. The result extends a theorem about how contours must end to one that characterizes surface (Gaussian) curvature in the neighborhood of where they appear to end. Apparent contours and their interaction with local shading thus provide important monocular shape cues.

Attention, segregation, and textons: bridging the gap between object-based attention and texton-based segregation.

Studies of object-based attention (OBA) have suggested that attentional selection is intimately associated with discrete objects. However, the relationship of this association to the basic visual features ('textons') which guide the segregation of visual scenes into 'objects' remains largely unexplored. Here we study this hypothesized relationship for one of the most conspicuous features of early vision: orientation. To do so we examine how attention spreads through uniform (one 'object') orientation-defined textures (ODTs), and across texture-defined boundaries in discontinuous (two 'objects') ODTs. Using the divided-attention paradigm we find that visual events that are known to trigger orientation-based texture segregation, namely perceptual boundaries defined by high orientation and/or curvature gradients, also induce a significant cost on attentional selection. At the same time we show that no effect is incurred by the absolute value of the textons, i.e., by the general direction (or, the 'grain') of the texture-in conflict with previous findings in the OBA literature. Collectively these experiments begin to reveal the link between object-based attention and texton-based segregation, a link which also offers important cross-disciplinary methodological advantages.

A constant production hypothesis guides leaf venation patterning.

We propose a theoretical mechanism that enables the elaboration of veins to supply distant cells during leaf development. In contrast to the more standard view that a signal (e.g., auxin) is produced at isolated sites to stimulate growth, we determine the consequences of the hypothesis that auxin is produced at a constant rate in every cell. High concentration sites for auxin emerge naturally in a reaction-diffusion model, together with global information about leaf shape and existing venation. Because the global information is encoded as auxin concentration and its gradient, those signals provide individual cells with sufficient information to determine their own fate. Unlike other models, a single substance suffices for the reaction-diffusion at early, but not initial, stages of development. Neither complex interactions nor predetermination are necessary. We predict angiosperm areolation patterns in simulation, and our model further implies the Sachs Canalization Hypothesis and resolves a dilemma regarding the role of auxin in cell growth.

Geometric diffusions for the analysis of data from sensor networks.

Harmonic analysis on manifolds and graphs has recently led to mathematical developments in the field of data analysis. The resulting new tools can be used to compress and analyze large and complex data sets, such as those derived from sensor networks or neuronal activity datasets, obtained in the laboratory or through computer modeling. The nature of the algorithms (based on diffusion maps and connectivity strengths on graphs) possesses a certain analogy with neural information processing, and has the potential to provide inspiration for modeling and understanding biological organization in perception and memory formation.

Indexing hierarchical structures using graph spectra.

Hierarchical image structures are abundant in computer vision and have been used to encode part structure, scale spaces, and a variety of multiresolution features. In this paper, we describe a framework for indexing such representations that embeds the topological structure of a directed acyclic graph (DAG) into a low-dimensional vector space. Based on a novel spectral characterization of a DAG, this topological signature allows us to efficiently retrieve a promising set of candidates from a database of models using a simple nearest-neighbor search. We establish the insensitivity of the signature to minor perturbation of graph structure due to noise, occlusion, or node split/merge. To accommodate large-scale occlusion, the DAG rooted at each nonleaf node of the query "votes" for model objects that share that "part," effectively accumulating local evidence in a model DAG's topological subspaces. We demonstrate the approach with a series of indexing experiments in the domain of view-based 3D object recognition using shock graphs.

Hue geometry and horizontal connections.

Primate visual systems support an elaborate specialization for processing color information. Concentrating on the hue component, we observe that, contrary to Mondrian-like assumptions, hue varies in a smooth manner for ecologically important natural imagery. To represent these smooth variations, and to support those information processing tasks that utilize hue, a piecewise smooth hue field is postulated. The geometry of hue-patch interactions is developed analogously to orientation-patch interactions in texture. The result is a model for long-range (horizontal) interactions in the color domain, the power of which is demonstrated on a number of examples. Implications for computer image processing, computer vision, visual neurophysiology and psychophysics are discussed.

Sensitivity to curvatures in orientation-based texture segmentation.

Texture segregation has long been attributed to changes in the distribution of elementary features across the visual field [Nature 290 (12) (1981) 91; Biol. Cybernet. 54 (1986) 245]. The study of orientation, a conspicuous feature, has led to models of orientation-based texture segmentation (OBTS) that depend on the magnitude of one or two orientation gradients [Vis. Res. 31 (4) (1991) 679; Vis. Res. 31 (6) (1991) 1073] and influenced further by the relative configuration between the orientation textons and the global orientation edge [Percept. Psychophys. 52 (4) (1992) 255; Vis. Res. 35 (20) (1995) 2863]. Here we show that these models are at best partial and that the notion of orientation gradient has been incompletely used in the study of OBTS. To do so, we first study the behavior of orientation in orientation-defined texture patches. Geometrical analysis identifies two texture curvatures and reveals the incompleteness of previous stimuli. Psychophysical experimentation then demonstrates that segmentation is strongly affected by discontinuities in these curvatures. Importantly, we show that this sensitivity to curvature is independent of the orientation gradients and inconsistent with the simple configural considerations proposed in the past.