For deep networks, the whole equals the sum of the parts.

Deep convolutional networks exceed humans in sensitivity to local image properties, but unlike biological vision systems, do not discover and encode abstract relations that capture important properties of objects and events in the world. Coupling network architectures with additional machinery for encoding abstract relations will make deep networks better models of human abilities and more versatile and capable artificial devices.

Persistent perceptual grouping effects in the evaluation of simple arithmetic expressions.

Landy and Goldstone (2007a, 2010) demonstrated that an explicit rule, operator precedence for simple arithmetic expressions, is enforced in part by perceptual processes like unit formation and attention. When perceptual grouping competes with operator precedence, errors increase. We replicated this result (Exp. 1) and investigated whether perceptual grouping effects persist when the visual stimulus is presented briefly and then masked (Exp. 2) and when verbal recoding is encouraged through vocal expression (Exp. 3). We found that perceptual-grouping effects persisted in the masking condition, suggesting that the mental representations of arithmetic expressions retain visuospatial characteristics. Similarly, verbalization of the expressions did not eliminate perceptual-grouping effects, suggesting that participants were not verbally recoding. In sum, the persistent effects of unit formation and spatial attention emphasize the importance of perceptual processing in the development of human expertise in this domain.

Configural relations in humans and deep convolutional neural networks.

Deep convolutional neural networks (DCNNs) have attracted considerable interest as useful devices and as possible windows into understanding perception and cognition in biological systems. In earlier work, we showed that DCNNs differ dramatically from human perceivers in that they have no sensitivity to global object shape. Here, we investigated whether those findings are symptomatic of broader limitations of DCNNs regarding the use of relations. We tested learning and generalization of DCNNs (AlexNet and ResNet-50) for several relations involving objects. One involved classifying two shapes in an otherwise empty field as same or different. Another involved enclosure. Every display contained a closed figure among contour noise fragments and one dot; correct responding depended on whether the dot was inside or outside the figure. The third relation we tested involved a classification that depended on which of two polygons had more sides. One polygon always contained a dot, and correct classification of each display depended on whether the polygon with the dot had a greater number of sides. We used DCNNs that had been trained on the ImageNet database, and we used both restricted and unrestricted transfer learning (connection weights at all layers could change with training). For the same-different experiment, there was little restricted transfer learning (82.2%). Generalization tests showed near chance performance for new shapes. Results for enclosure were at chance for restricted transfer learning and somewhat better for unrestricted (74%). Generalization with two new kinds of shapes showed reduced but above-chance performance (≈66%). Follow-up studies indicated that the networks did not access the enclosure relation in their responses. For the relation of more or fewer sides of polygons, DCNNs showed successful learning with polygons having 3-5 sides under unrestricted transfer learning, but showed chance performance in generalization tests with polygons having 6-10 sides. Experiments with human observers showed learning from relatively few examples of all of the relations tested and complete generalization of relational learning to new stimuli. These results using several different relations suggest that DCNNs have crucial limitations that derive from their lack of computations involving abstraction and relational processing of the sort that are fundamental in human perception.

Design of a trichromatic cone array.

Cones with peak sensitivity to light at long (L), medium (M) and short (S) wavelengths are unequal in number on the human retina: S cones are rare (<10%) while increasing in fraction from center to periphery, and the L/M cone proportions are highly variable between individuals. What optical properties of the eye, and statistical properties of natural scenes, might drive this organization? We found that the spatial-chromatic structure of natural scenes was largely symmetric between the L, M and S sensitivity bands. Given this symmetry, short wavelength attenuation by ocular media gave L/M cones a modest signal-to-noise advantage, which was amplified, especially in the denser central retina, by long-wavelength accommodation of the lens. Meanwhile, total information represented by the cone mosaic remained relatively insensitive to L/M proportions. Thus, the observed cone array design along with a long-wavelength accommodated lens provides a selective advantage: it is maximally informative.

Perceptual learning depends on perceptual constancy.

Perceptual learning refers to experience-induced improvements in the pick-up of information. Perceptual constancy describes the fact that, despite variable sensory input, perceptual representations typically correspond to stable properties of objects. Here, we show evidence of a strong link between perceptual learning and perceptual constancy: Perceptual learning depends on constancy-based perceptual representations. Perceptual learning may involve changes in early sensory analyzers, but such changes may in general be constrained by categorical distinctions among the high-level perceptual representations to which they contribute. Using established relations of perceptual constancy and sensory inputs, we tested the ability to discover regularities in tasks that dissociated perceptual and sensory invariants. We found that human subjects could learn to classify based on a perceptual invariant that depended on an underlying sensory invariant but could not learn the identical sensory invariant when it did not correlate with a perceptual invariant. These results suggest that constancy-based representations, known to be important for thought and action, also guide learning and plasticity.

Constant curvature segments as building blocks of 2D shape representation.

How the visual system represents shape, and how shape representations might be computed by neural mechanisms, are fundamental and unanswered questions. Here, we investigated the hypothesis that 2-dimensional (2D) contour shapes are encoded structurally, as sets of connected constant curvature segments. We report 3 experiments investigating constant curvature segments as fundamental units of contour shape representations in human perception. Our results showed better performance in a path detection paradigm for constant curvature targets, as compared with locally matched targets that lacked this global regularity (Experiment 1), and that participants can learn to segment contours into constant curvature parts with different curvature values, but not into similarly different parts with linearly increasing curvatures (Experiment 2). We propose a neurally plausible model of contour shape representation based on constant curvature, built from oriented units known to exist in early cortical areas, and we confirmed the model's prediction that changes to the angular extent of a segment will be easier to detect than changes to relative curvature (Experiment 3). Together, these findings suggest the human visual system is specially adapted to detect and encode regions of constant curvature and support the notion that constant curvature segments are the building blocks from which abstract contour shape representations are composed. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

The Synergy of Passive and Active Learning Modes in Adaptive Perceptual Learning.

Adaptive learning systems that generate spacing intervals based on learner performance enhance learning efficiency and retention (Mettler, Massey & Kellman, 2016). Recent research in factual learning suggests that initial blocks of passive trials, where learners observe correct answers without overtly responding, produce greater learning than passive or active trials alone (Mettler, Massey, Burke, Garrigan & Kellman, 2018). Here we tested whether this passive + active advantage generalizes beyond factual learning to perceptual learning. Participants studied and classified images of butterfly genera using either: 1) Passive Only presentations, 2) Passive Initial Blocks followed by active, adaptive scheduling, 3) Passive Initial Category Exemplar followed by active, adaptive scheduling, or 4) Active Only learning. We found an advantage for combinations of active and passive presentations over Passive Only or Active Only presentations. Passive trials presented in initial blocks showed the best performance, paralleling earlier findings in factual learning. Combining active and passive learning produces greater learning gains than either alone, and these effects occur for diverse forms of learning, including perceptual learning.

Interpolation processes in object perception: reply to Anderson (2007).

P. J. Kellman, P. Garrigan, & T. F. Shipley presented a theory of 3-D interpolation in object perception. Along with results from many researchers, this work supports an emerging picture of how the visual system connects separate visible fragments to form objects. In his commentary, B. L. Anderson challenges parts of that view, especially the idea of a common underlying interpolation component in modal and amodal completion (the identity hypothesis). Here the authors analyze Anderson's evidence and argue that he neither provides any reason to abandon the identity hypothesis nor offers a viable alternative theory. The authors offer demonstrations and analyses indicating that interpolated contours can appear modally despite absence of the luminance relations, occlusion geometry, and surface attachment that Anderson claims to be necessary. The authors elaborate crossing interpolations as key cases in which modal and amodal appearance must be consequences of interpolation. Finally, the authors dispute Anderson's assertion that vision researchers are misguided in using objective performance methods, and they argue that his challenges to relatability fail because contour and surface processes, as well as local and global influences, have been distinguished experimentally.

Object interpolation in three dimensions.

Perception of objects in ordinary scenes requires interpolation processes connecting visible areas across spatial gaps. Most research has focused on 2-D displays, and models have been based on 2-D, orientation-sensitive units. The authors present a view of interpolation processes as intrinsically 3-D and producing representations of contours and surfaces spanning all 3 spatial dimensions. The authors propose a theory of 3-D relatability that indicates for a given edge which orientations and positions of other edges in 3 dimensions may be connected to it, and they summarize the empirical evidence for 3-D relatability. The theory unifies and illuminates a number of fundamental issues in object formation, including the identity hypothesis in visual completion, the relations of contour and surface processes, and the separation of local and global processing. The authors suggest that 3-D interpolation and 3-D relatability have major implications for computational and neural models of object perception.

3-d interpolation in object perception: evidence from an objective performance paradigm.

Object perception requires interpolation processes that connect visible regions despite spatial gaps. Some research has suggested that interpolation may be a 3-D process, but objective performance data and evidence about the conditions leading to interpolation are needed. The authors developed an objective performance paradigm for testing 3-D interpolation and tested a new theory of 3-D contour interpolation, termed 3-D relatability. The theory indicates for a given edge which orientations and positions of other edges in space may be connected to it by interpolation. Results of 5 experiments showed that processing of orientation relations in 3-D relatable displays was superior to processing in 3-D nonrelatable displays and that these effects depended on object formation. 3-D interpolation and 3-D relatabilty are discussed in terms of their implications for computational and neural models of object perception, which have typically been based on 2-D-orientation-sensitive units.

Coherence of visual representations: attention and integration of contour shape information.

Change blindness demonstrations illustrate the limited detail of visual representations. These demonstrations typically require disruption to the visual input when the change occurs or changes that occur very slowly. With sustained viewing or faster changes to the scenes, changes are more easily detected because attention can be effectively allocated to the part of the scene that is changing. Here, we investigate the interaction of visual attention and memory in the domain of 2-D contour shapes. We show, using a novel combination of established change blindness paradigms, that changes can go unnoticed even when they occur on isolated 2-D contour shapes. The effect appears to be due to involuntary updating of stored shape information. This involuntary updating process, however, is constrained so that previously attended shape information is updated only when attention is reallocated to qualitatively similar shape information.

Forensic comparison and matching of fingerprints: using quantitative image measures for estimating error rates through understanding and predicting difficulty.

Latent fingerprint examination is a complex task that, despite advances in image processing, still fundamentally depends on the visual judgments of highly trained human examiners. Fingerprints collected from crime scenes typically contain less information than fingerprints collected under controlled conditions. Specifically, they are often noisy and distorted and may contain only a portion of the total fingerprint area. Expertise in fingerprint comparison, like other forms of perceptual expertise, such as face recognition or aircraft identification, depends on perceptual learning processes that lead to the discovery of features and relations that matter in comparing prints. Relatively little is known about the perceptual processes involved in making comparisons, and even less is known about what characteristics of fingerprint pairs make particular comparisons easy or difficult. We measured expert examiner performance and judgments of difficulty and confidence on a new fingerprint database. We developed a number of quantitative measures of image characteristics and used multiple regression techniques to discover objective predictors of error as well as perceived difficulty and confidence. A number of useful predictors emerged, and these included variables related to image quality metrics, such as intensity and contrast information, as well as measures of information quantity, such as the total fingerprint area. Also included were configural features that fingerprint experts have noted, such as the presence and clarity of global features and fingerprint ridges. Within the constraints of the overall low error rates of experts, a regression model incorporating the derived predictors demonstrated reasonable success in predicting objective difficulty for print pairs, as shown both in goodness of fit measures to the original data set and in a cross validation test. The results indicate the plausibility of using objective image metrics to predict expert performance and subjective assessment of difficulty in fingerprint comparisons.

The effect of contour closure on shape recognition.

Recent research on the Gestalt principle of closure has focused on how the presence of closure affects the ability to detect contours hidden in cluttered visual arrays. Some of the earliest research on closure, however, dealt with encoding and recognizing closed and open shapes, rather than detection. This research re-addresses the relation between closure and shape memory, focusing on how contour closure affects the ability to learn to recognize novel contour shapes. Of particular interest is whether closed contour shapes are easier to learn to recognize and, if so, whether this benefit is due to better encoding of closed contour shapes or easier comparison of closed contour shapes to already learned shapes. The results show that closed contours are indeed easier to recognize and, further, that this advantage appears to be related to better encoding.

The role of constant curvature in 2-D contour shape representations.

In early cortex, visual information is encoded by retinotopic orientation-selective units. Higher-level representations of abstract properties, such as shape, require encodings that are invariant to changes in size, position, and orientation. Within the domain of open, 2-D contours, we consider how an economical representation that supports viewpoint-invariant shape comparisons can be derived from early encodings. We explore the idea that 2-D contour shapes are encoded as joined segments of constant curvature. We report three experiments in which participants compared sequentially presented 2-D contour shapes comprised of constant curvature (CC) or non-constant curvature (NCC) segments. We show that, when shapes are compared across viewpoint or for a retention interval of 1000 ms, performance is better for CC shapes. Similar recognition performance is observed for both shape types, however, if they are compared at the same viewpoint and the retention interval is reduced to 500 ms. These findings are consistent with a symbolic encoding of 2-D contour shapes into CC parts when the retention intervals over which shapes must be stored exceed the duration of initial, transient, visual representations.

Natural images from the birthplace of the human eye.

Here we introduce a database of calibrated natural images publicly available through an easy-to-use web interface. Using a Nikon D70 digital SLR camera, we acquired about six-megapixel images of Okavango Delta of Botswana, a tropical savanna habitat similar to where the human eye is thought to have evolved. Some sequences of images were captured unsystematically while following a baboon troop, while others were designed to vary a single parameter such as aperture, object distance, time of day or position on the horizon. Images are available in the raw RGB format and in grayscale. Images are also available in units relevant to the physiology of human cone photoreceptors, where pixel values represent the expected number of photoisomerizations per second for cones sensitive to long (L), medium (M) and short (S) wavelengths. This database is distributed under a Creative Commons Attribution-Noncommercial Unported license to facilitate research in computer vision, psychophysics of perception, and visual neuroscience.

A unified model of illusory and occluded contour interpolation.

Models of contour interpolation have been proposed for illusory contour interpolation but seldom for interpolation of occluded contours. The identity hypothesis (Kellman & Loukides, 1987; Kellman & Shipley, 1991) posits that an early interpolation mechanism is shared by interpolated contours that are ultimately perceived as either illusory or occluded. Here we propose a model of such a unified interpolation mechanism for illusory and occluded contours, building on the framework established in Heitger, von der Heydt, Peterhans, Rosenthaler, and Kubler (1998). We show that a single, neurally plausible mechanism that is consistent with the identity hypothesis also generates contour interpolations in agreement with perception for cases of transparency, self-splitting objects, interpolation with mixed boundary assignment, and "quasimodal" interpolations. Limiting cases for this local, feed-forward approach are presented, demonstrating that both early, local interpolation mechanisms and non-local scene constraints are necessary for describing the perception of interpolated contours.

Perceptual learning and human expertise.

We consider perceptual learning: experience-induced changes in the way perceivers extract information. Often neglected in scientific accounts of learning and in instruction, perceptual learning is a fundamental contributor to human expertise and is crucial in domains where humans show remarkable levels of attainment, such as language, chess, music, and mathematics. In Section 2, we give a brief history and discuss the relation of perceptual learning to other forms of learning. We consider in Section 3 several specific phenomena, illustrating the scope and characteristics of perceptual learning, including both discovery and fluency effects. We describe abstract perceptual learning, in which structural relationships are discovered and recognized in novel instances that do not share constituent elements or basic features. In Section 4, we consider primary concepts that have been used to explain and model perceptual learning, including receptive field change, selection, and relational recoding. In Section 5, we consider the scope of perceptual learning, contrasting recent research, focused on simple sensory discriminations, with earlier work that emphasized extraction of invariance from varied instances in more complex tasks. Contrary to some recent views, we argue that perceptual learning should not be confined to changes in early sensory analyzers. Phenomena at various levels, we suggest, can be unified by models that emphasize discovery and selection of relevant information. In a final section, we consider the potential role of perceptual learning in educational settings. Most instruction emphasizes facts and procedures that can be verbalized, whereas expertise depends heavily on implicit pattern recognition and selective extraction skills acquired through perceptual learning. We consider reasons why perceptual learning has not been systematically addressed in traditional instruction, and we describe recent successful efforts to create a technology of perceptual learning in areas such as aviation, mathematics, and medicine. Research in perceptual learning promises to advance scientific accounts of learning, and perceptual learning technology may offer similar promise in improving education.