GOOD CONTINUATION IN 3D: THE NEUROGEOMETRY OF STEREO VISION.

Classical good continuation for image curves is based on 2D position and orientation. It is supported by the columnar organization of cortex, by psychophysical experiments, and by rich models of (differential) geometry. Here we extend good continuation to stereo. We introduce a neurogeometric model, in which the parametrizations involve both spatial and orientation disparities. Our model provides insight into the neurobiology, suggesting an implicit organization for neural interactions and a well-defined 3D association field. Our model sheds light on the computations underlying the correspondence problem, and illustrates how good continuation in the world generalizes good continuation in the plane.

Spatial self-organization resolves conflicts between individuality and collective migration.

Collective behavior can spontaneously emerge when individuals follow common rules of interaction. However, the behavior of each individual differs due to existing genetic and non-genetic variation within the population. It remains unclear how this individuality is managed to achieve collective behavior. We quantify individuality in bands of clonal Escherichia coli cells that migrate collectively along a channel by following a self-generated gradient of attractant. We discover that despite substantial differences in individual chemotactic abilities, the cells are able to migrate as a coherent group by spontaneously sorting themselves within the moving band. This sorting mechanism ensures that differences between individual chemotactic abilities are compensated by differences in the local steepness of the traveling gradient each individual must navigate, and determines the minimum performance required to travel with the band. By resolving conflicts between individuality and collective migration, this mechanism enables populations to maintain advantageous diversity while on the move.

Geometric diffusions as a tool for harmonic analysis and structure definition of data: multiscale methods.

In the companion article, a framework for structural multiscale geometric organization of subsets of R(n) and of graphs was introduced. Here, diffusion semigroups are used to generate multiscale analyses in order to organize and represent complex structures. We emphasize the multiscale nature of these problems and build scaling functions of Markov matrices (describing local transitions) that lead to macroscopic descriptions at different scales. The process of iterating or diffusing the Markov matrix is seen as a generalization of some aspects of the Newtonian paradigm, in which local infinitesimal transitions of a system lead to global macroscopic descriptions by integration. This article deals with the construction of fast-order N algorithms for data representation and for homogenization of heterogeneous structures.

Geometric diffusions as a tool for harmonic analysis and structure definition of data: diffusion maps.

We provide a framework for structural multiscale geometric organization of graphs and subsets of R(n). We use diffusion semigroups to generate multiscale geometries in order to organize and represent complex structures. We show that appropriately selected eigenfunctions or scaling functions of Markov matrices, which describe local transitions, lead to macroscopic descriptions at different scales. The process of iterating or diffusing the Markov matrix is seen as a generalization of some aspects of the Newtonian paradigm, in which local infinitesimal transitions of a system lead to global macroscopic descriptions by integration. We provide a unified view of ideas from data analysis, machine learning, and numerical analysis.

On the psychophysics of the shape triangle.

We earlier introduced an approach to categorical shape description based on the singularities (shocks) of curve evolution equations. We now consider the simplest compositions of shocks, and show that they lead to three classes of parametrically ordered shape sequences, organized along the sides of a shape triangle. By conducting several psychophysical experiments we demonstrate that shock-based descriptions are predictive of performance in shape perception. Most significantly, the experiments reveal a fundamental difference between perceptual effects dominated by when shocks form with respect to one another, versus those dominated by where they form. The shock-based theory provides a foundation for unifying tasks as diverse as shape bisection, recognition, and categorization.

Computing with self-excitatory cliques: A model and an application to hyperacuity-scale computation in visual cortex.

We present a model of visual computation based on tightly inter-connected cliques of pyramidal cells. It leads to a formal theory of cell assemblies, a specific relationship between correlated firing patterns and abstract functionality, and a direct calculation relating estimates of cortical cell counts to orientation hyperacuity. Our network architecture is unique in that (1) it supports a mode of computation that is both reliable and efficient; (2) the current-spike relations are modeled as an analog dynamical system in which the requisite computations can take place on the time scale required for an early stage of visual processing; and (3) the dynamics are triggered by the spatiotemporal response of cortical cells. This final point could explain why moving stimuli improve vernier sensitivity.

Evidence for boundary-specific grouping.

A prerequisite for higher-level visual tasks such as object recognition is a segmentation of the image into distinct two-dimensional regions. While it has long been assumed that the human visual system jointly exploits region and boundary cues for image segmentation, we report the results of psychophysical experiments which suggest that the visual system relies on geometric properties of bounding contours such as closure and not on the texture of the two-dimensional regions they partition. These findings suggest that the visual system may code and links contours into coherent shapes before surface properties are conjoined.

Area and length minimizing flows for shape segmentation.

A number of active contour models have been proposed that unify the curve evolution framework with classical energy minimization techniques for segmentation, such as snakes. The essential idea is to evolve a curve (in two dimensions) or a surface (in three dimensions) under constraints from image forces so that it clings to features of interest in an intensity image. The evolution equation has been derived from first principles as the gradient flow that minimizes a modified length functional, tailored to features such as edges. However, because the flow may be slow to converge in practice, a constant (hyperbolic) term is added to keep the curve/surface moving in the desired direction. We derive a modification of this term based on the gradient flow derived from a weighted area functional, with image dependent weighting factor. When combined with the earlier modified length gradient flow, we obtain a partial differential equation (PDE) that offers a number of advantages, as illustrated by several examples of shape segmentation on medical images. In many cases the weighted area flow may be used on its own, with significant computational savings.

The discriminability of smooth stereoscopic surfaces.

In this study the sensitivity of human vision to the smoothness of stereoscopic surface structure was investigated. In experiments 1 and 2 random-dot stereograms were used to evaluate the discrimination of smooth versus 'noisy' sinusoidal surfaces differing in the percentages of points on a single smooth surface. Fully coherent smooth surfaces were found to be much more discriminable than other less smooth randomly perturbed surfaces. In experiment 3 the discrimination between discontinuous triangle-wave surfaces and similarly shaped smoothly curved surfaces obtained from the addition of the fundamental and the third harmonic of the corresponding triangle-wave surface was evaluated. The triangle-wave surfaces were found to be more accurately discriminated from the smoothly curved surfaces than would be predicted from the detectability of the difference in their Fourier power spectra. This superior discriminability was attributed to differences between the curvature and/or discontinuity of the two surfaces. In experiment 3 the effects of incoherent 'noise' points on the discrimination between the two surface types were also evaluated. These randomly positioned noise points had a relatively small effect on the discrimination between the two surfaces. In general, the results of these experiments indicate that smooth surfaces are salient for stereopsis and that isolated local violations of smoothness are highly discriminable.

Texture fields and texture flows: sensitivity to differences.

There are two large classes of textures, those with an overall orientation structure (texture flows) and those without (texture fields). We investigate human sensitivity to detecting a patch of texture field within a texture flow psychophysically by using random not Moiré patterns. The resultant sensitivity, as a function of patch-size and path-length, is then related to a computational model of orientation selection, which reveals a connection between texture structure and the estimation of curvature. Finally, the connection back to curvature is confirmed by demonstrating a similarity between the patch sensitivity data and previous data on sensitivity to corners in flow patterns.

Endstopping and curvature.

Hypercomplex or endstopped visual cortical neurons are usually supposed to be concerned with length or end point analysis. However, recent evidence demonstrates that endstopped neurons are curvature-selective, a connection that we explore here in some detail. A model of endstopped simple cells is developed and a variety of computational simulations examine the connection of the model to the reported length and orientation responses of endstopped neurons. Even and odd versions of the model are described, both of which are shown to be curvature-selective. Even-symmetric instances of the model respond well to thin curves over a range of curve orientation and curvature, independent of sign of curvature. In contrast, odd-symmetric instances respond to both thin and thick curves while exhibiting a more complex curvature-sign dependence--responding in a sign-selective fashion to curved lines but not to curved edges. Finally, the response of the endstopped model to curve singularities is explored, and the possible role of nonendstopped and endstopped cells in building curve descriptions is discussed.

Corner detection in curvilinear dot grouping.

Corners, or discontinuities in orientation, are one of the most salient and useful properties of contours. But how sensitive are we in detecting them, and what does this sensitivity imply about the processes by which corners can be detected. In this paper we address both of these questions, starting with the observation that changing the sampling phase of a curve changes the geometry of its discrete trace, or the set of discrete (retinotopic) points onto which the curve projects. This motivates our stimuli--dotted curves--and our experimental design: if curves are represented by dots, the placement of the dots effects whether or not corners are perceived. Specifically, we present quantitative data on sensitivity to discontinuities as a function of dot phase, and address its theoretical explanation within a two-stage model of orientation selection. Curvature plays a key role in this model, and, finally, the model and experimental data are brought together by showing that a very coarse approximation to change in curvature (or differences in local curvature estimates) is sufficient to account for the psychophysical data on sensitivity to discontinuities.

Points and endpoints: a size/spacing constraint for dot grouping.

One-dimensional arrangements of dots immediately group into contours. It is reported that, when these contours participate in certain larger arrangements, there is an abrupt point at which the percept changes as a function of dot spacing (or density along the contour). Closely spaced arrangements give rise to subjective effects involving apparent brightness and depth, whereas sparsely spaced ones do not. The effects are most clear in configurations that involve endpoints and possible occlusions. For these configurations, densely dotted contours are perceptually equivalent to solid ones, but sparse ones are not. This change in percept occurs abruptly and consistently at a dot to space ratio of 1:5, when the dot density is normalized by dot size, and this point is called the size/spacing constraint. It holds only for dots of the order of 1 min visual angle in diameter when small to modest contrast values are used. The subjective effects are not present for dotted contours (or even for solid ones) that are smaller (less than 0.5 min), and differ for contours that are larger (greater than 10 min). To demonstrate the significance of size/spacing constraints for early vision, a framework for grouping consisting of processes at many different levels is outlined, and the requirements for the earliest one (orientation selection) are sketched in greater detail. The size/spacing constraint follows directly from one of these requirements--receptive field structure--and seems to indicate a switch from early orientation-selection processes to later ones.

Endstopped neurons in the visual cortex as a substrate for calculating curvature.

Neurons in the visual cortex typically respond selectively to the orientation, and velocity and direction of movement, of moving-bar stimuli. These responses are generally thought to provide information about the orientation and position of lines and edges in the visual field. Some cells are also endstopped, that is selective for bars of specific lengths. Hubel and Wiesel first observed that endstopped hypercomplex cells could respond to curved stimuli and suggested they might be involved in detection of curvature, but the exact relationship between endstopping and curvature has never been determined. We present here a mathematical model relating endstopping to curvature in which the difference in response of two simple cells gives rise to endstopping and varies in proportion to curvature. We also provide physiological evidence that endstopped cells in area 17 of the cat visual cortex are selective for curvature, whereas non-endstopped cells are not, and that some are selective for the sign of curvature. The prevailing view of edge and curve determination is that orientations are selected locally by the class of simple cortical cells and then integrated to form global curves. We have developed a computational theory of orientation selection which shows that measurements of orientation obtained by simple cells are not sufficient because there will be strong, incorrect responses from cells whose receptive fields (RFs) span distinct curves (Fig. 1). If estimates of curvature are available, however, these inappropriate responses can be eliminated. Curvature provides the key to structuring the network that underlies our theory and distinguishes it from previous lateral inhibition schemes.

Sensitivity to corners in flow patterns.

Flow patterns are two-dimensional orientation structures that arise from the projection of certain kinds of surface coverings (such as fur) onto images. Detecting orientation changes within them is an important task, since the changes often correspond to significant events such as corners, occluding edges, or surface creases. We model such patterns as random-dot Moiré patterns, and examine sensitivity to change in orientation within them. We show that the amount of structure available from which orientation and curvature can be estimated is critical, and introduce a path-length parameter to model it. For short path lengths many discontinuities are smoothed over, which has further implications for computational modeling of orientation selectivity.

The local structure of image discontinuities in one dimension.

The detailed structure of intensities in the local neighborhood of an edge can often indicate the nature of the physical event givinig rise to that edge. We argue that the limit, as we approach arbitrarily close to either side of an edge, of such image parameters as type of texture, texture gradient, color, appropriate directional derivatives of intensity, etc., is a key aspect of this structure. However, the general problem of capturing this local structure is surprisingly complex. Thus, we restrict ourselves in this paper to a relatively simple domain¿one-dimensional cuts through idealized images modeled by piecewise smooth (C1) functions corrupted by Gaussian noise. Within this domain, we define local structure to be the limit of the uncorrupted intensity and of its derivatives as we approach arbitrarily close to either side of a discontinuity. We develop a technique that captures this local structure while simultaneously locating the discontinuities, and demonstrate that these tasks are in fact inseparable. The technique is an extension, using estimation theory, of the classical definition of discontinuity. It handles, in a consistent fashion, both jump discontinuities in the function and jump discontinuities in its first derivative (so-called step-edges are a special case of the former and roof-edges of the latter). It also integrates, again in a consistent fashion, information derived from a number of different neighborhood sizes.

Receptive fields and the representation of visual information.

Receptive fields in the retina indicate the first measurements taken over the (discrete) visual image. Why are they circular surround with an excitatory/inhibitory structure? We hypothesize that this provides a representation of the visual information in a form suitable for transmission over the optic nerve, a rather limited channel, that can then be extended into a variety of representations at the cortex. These cortical representations span a range of sizes and functionally separate positive and negative contrast data, precisely as is required for further processing. Our scheme is both physiologically and psychophysically plausible. In particular, we derive an explicit formula for constructing large receptive fields from small ones, and introduce the notion of de-blurring to derive interpolation filters for hyperacuity. A mathematical requirement of our scheme is a form of separation between positive and negative contrast data, a nonlinearity that we predict will agree with observations. Furthermore, the mathematics that we utilize are more naturally applicable to physiological models based on Gaussian than are (Fourier) spatial frequencies.

Osteoclast enlargement in endstage renal disease.

In normal subjects and patients with endstage renal disease, osteoclast cell volumes were constructed using serial 2-micron thick plastic embedded sections from iliac crest bone biopsy specimens. Four cells randomly selected from each of the subjects were analyzed to give both the cell volume from the cumulative areas and thickness of each cell slice and also the cell axes taking the vector along the bone face as width or Y, thickness from the bone surface as breadth or X and vertical dimension length or Z. The mean cell volume was 6,230 microns3 in the control subjects and was significantly larger being 11,730 microns3 and 13,680 microns3 in the two patient groups. The cells showed polarity with the largest axes, Y and Z, being those in apposition to the bone surface. Howship's lacunae were enlarged in the patients and the cross-sectional area of an individual lacuna corresponded to the area of the contiguous osteoclast, r = 0.62, P less than 0.001. All patients had secondary hyperparathyroidism and osteoclast numbers were increased. There was no correlation between osteoclast size and duration of renal failure, previous vitamin D intake, or aluminum exposure. It is suggested that in ESRD, osteoclasts undergo both an increase in number and size and that these cells, being larger, remove more bone than the smaller cells in normal subjects.

Subjective figures and texture perception.

A texture discrimination task using the Ehrenstein illusion demonstrates that subjective brightness effects can play an essential role in early vision. The subjectively bright regions of the Ehrenstein can be organized either as discs or as stripes, depending on orientation. The accuracy of discrimination between variants of the Ehrenstein and control patterns was a direct function of the presence of the illusory brightness stripes, being high when they were present and low otherwise. It is argued that neither receptive field structure nor spatial-frequency content can adequately account for these results. We suggest that the subjective brightness illusions, rather than being a high-level, cognitive aspect of vision, are in fact the result of an early visual process.