An edge-simplicity bias in the visual input to young infants.

The development of sparse edge coding in the mammalian visual cortex depends on early visual experience. In humans, there are multiple indicators that the statistics of early visual experiences has unique properties that may support these developments. However, there are no direct measures of the edge statistics of infant daily-life experience. Using head-mounted cameras to capture egocentric images of young infants and adults in the home, we found infant images to have distinct edge statistics relative to adults. For infants, scenes with sparse edge patterns-few edges and few orientations-dominate. The findings implicate biased early input at the scale of daily life that is likely specific to the early months after birth and provide insights into the quality, amount, and timing of the visual experiences during the foundational developmental period for human vision.

What image features guide lightness perception?

Lightness constancy is the ability to perceive black and white surface colors under a wide range of lighting conditions. This fundamental visual ability is not well understood, and current theories differ greatly on what image features are important for lightness perception. Here we measured classification images for human observers and four models of lightness perception to determine which image regions influenced lightness judgments. The models were a high-pass-filter model, an oriented difference-of-Gaussians model, an anchoring model, and an atmospheric-link-function model. Human and model observers viewed three variants of the argyle illusion (Adelson, 1993) and judged which of two test patches appeared lighter. Classification images showed that human lightness judgments were based on local, anisotropic stimulus regions that were bounded by regions of uniform lighting. The atmospheric-link-function and anchoring models predicted the lightness illusion perceived by human observers, but the high-pass-filter and oriented-difference-of-Gaussians models did not. Furthermore, all four models produced classification images that were qualitatively different from those of human observers, meaning that the model lightness judgments were guided by different image regions than human lightness judgments. These experiments provide a new test of models of lightness perception, and show that human observers' lightness computations can be highly local, as in low-level models, and nevertheless depend strongly on lighting boundaries, as suggested by midlevel models.

Multisensory Integration in Short-term Memory: Musicians do Rock.

Demonstrated interactions between seeing and hearing led us to assess the link between music training and short-term memory for auditory, visual and audiovisual sequences of rapidly presented, quasi-random components. Visual sequences' components varied in luminance; auditory sequences' components varied in frequency. Concurrent components in audiovisual sequences were either congruent (the frequency of an auditory item increased monotonically with the luminance of the visual item it accompanied), or incongruent (an item's frequency was uncorrelated with luminance of the item it accompanied). Subjects judged whether the last four items in a sequence replicated its first four items. With audiovisual sequences, subjects were instructed to ignore the sequence's auditory components, basing their judgments solely on the visual input. Subjects with prior instrumental training significantly outperformed their untrained counterparts, with both auditory and visual sequences, and with sequences of correlated auditory and visual items. Reverse correlation showed that the presence of a correlated, concurrent auditory stream altered subjects' reliance on particular visual items in a sequence. Moreover, congruence between auditory and visual items produced performance above what would be predicted from simple summation of information from the two modalities, a result that might reflect a contribution from special-purpose, multimodal neural mechanisms.

Biased guessing in a complete-identification visual-working-memory task: Further evidence for mixed-state models.

Research is reported that provides evidence for a significant role of mixed states and guessing processes in tasks of visual working memory (VWM). Subjects engaged in a complete-identification VWM task. The stimulus set consisted of 16 colors roughly equally spaced around a color circle. On each trial, a memory-set drawn from the colors was briefly presented, followed by a location probe. Subjects attempted to reproduce the color of the probed item by clicking on the appropriate response button of a discrete color wheel. The key manipulation was to vary payoffs for alternative correct responses across trials. Analysis of the resulting matrices of individual-subject identification-confusion data provided evidence for a systematic guessing process: On trials in which subjects had no memory for the probed stimulus, they guessed with high probability using the high-payoff response. Formal modeling corroborated this interpretation. Mixed-state models that assumed that performance involved a combination of memory-based responding and biased guessing yielded accurate and easy-to-interpret accounts of the identification data; by comparison, variable-resources (VR) models without a guessing state struggled to account for the data, including versions with bias parameters for the high-payoff response. The authors argue that the work adds to recent converging sources of evidence that point to a significant role of discrete, mixed states in VWM. The authors also suggest directions for development of extended VR models with sophisticated knowledge-rich decision rules for the complete-identification task. (PsycINFO Database Record

Efficiencies for parts and wholes in biological-motion perception.

People can reliably infer the actions, intentions, and mental states of fellow humans from body movements (Blake & Shiffrar, 2007). Previous research on such biological-motion perception has suggested that the movements of the feet may play a particularly important role in making certain judgments about locomotion (Chang & Troje, 2009; Troje & Westhoff, 2006). One account of this effect is that the human visual system may have evolved specialized processes that are efficient for extracting information carried by the feet (Troje & Westhoff, 2006). Alternatively, the motion of the feet may simply be more discriminable than that of other parts of the body. To dissociate these two possibilities, we measured people's ability to discriminate the walking direction of stimuli in which individual body parts (feet, hands) were removed or shown in isolation. We then compared human performance to that of a statistically optimal observer (Gold, Tadin, Cook, & Blake, 2008), giving us a measure of humans' discriminative ability independent of the information available (a quantity known as efficiency). We found that efficiency was highest when the hands and the feet were shown in isolation. A series of follow-up experiments suggested that observers were relying on a form-based cue with the isolated hands (specifically, the orientation of their path through space) and a motion-based cue with the isolated feet to achieve such high efficiencies. We relate our findings to previous proposals of a distinction between form-based and motion-based mechanisms in biological-motion perception.

Memory and learning for visual signals in time and space.

Vision is often characterized as a spatial sense, but what does that characterization imply about the relative ease of processing visual information distributed over time rather than over space? Three experiments addressed this question, using stimuli comprising random luminances. For some stimuli, individual items were presented sequentially, at 8 Hz; for other stimuli, individual items were presented simultaneously, as horizontal spatial arrays. For temporal sequences, subjects judged whether each of the last four luminances matched the corresponding luminance in the first four; for spatial arrays, they judged whether each of the right-hand four luminances matched the corresponding left-hand luminance. Overall, performance was far better with spatial presentations, even when the entire spatial array was presented for just tens of milliseconds. Experiment 2 demonstrated that there was no gain in performance from combining spatial and temporal information within a single stimulus. In a final experiment, particular spatial arrays or temporal sequences were made to recur intermittently, interspersed among, non-recurring stimuli. Performance improved steadily as particular stimulus exemplars recurred, with spatial and temporal stimuli being learned at equivalent rates. Logistic regression identified several shortcut strategies that subjects may have exploited while performing our task.

The Impact of Symmetry on the Efficiency of Human Face Perception.

The presence of symmetric properties in a stimulus has been shown to often exert an influence on perception and information processing. Investigations into symmetry have given rise to the notion that it is processed easily and efficiently by the human visual system. However, only a handful of studies have attempted to actually measure symmetry's role in the efficiency of information use. We explored the impact of symmetry on the perception of human faces, a domain where it has been thought to play a particularly important role. Specifically, we measured information processing efficiency, defined as human performance relative to that of an ideal observer, for the detection, discrimination, and identification of symmetric and asymmetric faces. Surprisingly, we found no evidence for significant differences in efficiency between these two classes of stimuli. Training yielded significant improvements in overall efficiency, but had no significant effect on the relative efficiency of asymmetric and symmetric face identification. Our results indicate that although symmetry may be important to other aspects of face perception (e.g., perceived beauty), it has no discernible impact upon the efficiency with which information is used when detecting, discriminating, and identifying faces.

The impact of configural superiority on the processing of spatial information.

The impact of context on perception has been well documented for over a century. In some cases, the introduction of context to a set of target features may produce a unified percept, leading to a quicker and more accurate classification; a configural superiority effect (Pomerantz, Sager, & Stoever, 1977). Although this effect has been well characterized in terms of the stimulus features that produce the effect, the specific impact context has on the spatial strategies adopted by observers when making perceptual judgments remains unclear. Here, we sought to address this question by using the methods of response classification and ideal observer analysis. In our main experiment, we used a stimulus set known to produce the configural superiority effect and found that although observers were faster in the presence of context, they were actually less efficient at extracting stimulus information. This surprising result was attributable to the use of a spatial strategy in which observers relied on redundant, noninformative features in the presence of context. A control experiment ruled out the possibility that the mere presence of added context led to these strategic shifts. Our results support previous notions about the nature of the perceptual shifts that are induced by the configural superiority effect. However, they also show that configural processing is more nuanced than originally thought: Although observers may be faster at making judgments when context induces the percept of a configural whole, there appears to be a hidden cost in terms of the efficiency with which information is used. (PsycINFO Database Record

Integration of Social Information by Human Groups.

We consider a situation in which individuals search for accurate decisions without direct feedback on their accuracy, but with information about the decisions made by peers in their group. The "wisdom of crowds" hypothesis states that the average judgment of many individuals can give a good estimate of, for example, the outcomes of sporting events and the answers to trivia questions. Two conditions for the application of wisdom of crowds are that estimates should be independent and unbiased. Here, we study how individuals integrate social information when answering trivia questions with answers that range between 0% and 100% (e.g., "What percentage of Americans are left-handed?"). We find that, consistent with the wisdom of crowds hypothesis, average performance improves with group size. However, individuals show a consistent bias to produce estimates that are insufficiently extreme. We find that social information provides significant, albeit small, improvement to group performance. Outliers with answers far from the correct answer move toward the position of the group mean. Given that these outliers also tend to be nearer to 50% than do the answers of other group members, this move creates group polarization away from 50%. By looking at individual performance over different questions we find that some people are more likely to be affected by social influence than others. There is also evidence that people differ in their competence in answering questions, but lack of competence is not significantly correlated with willingness to change guesses. We develop a mathematical model based on these results that postulates a cognitive process in which people first decide whether to take into account peer guesses, and if so, to move in the direction of these guesses. The size of the move is proportional to the distance between their own guess and the average guess of the group. This model closely approximates the distribution of guess movements and shows how outlying incorrect opinions can be systematically removed from a group resulting, in some situations, in improved group performance. However, improvement is only predicted for cases in which the initial guesses of individuals in the group are biased.

Redundant visual information enhances group decisions.

Important perceptual judgments are often made by combining the opinions of several individuals to make a collective decision, such as when teams of physicians make diagnoses based on medical images. Although group-level decisions are generally superior to the decisions made by individuals, it remains unclear whether collective decision making is most effective when information is redundantly provided to all individuals within a group, or when each individual is responsible for only a portion of the total information. Here, we test this idea by having individuals and groups of different sizes make perceptual judgments about the presence of a weak visual signal. We found that groups viewing the entirety of information significantly outperformed groups that viewed limited portions of information, and that this difference in performance could be accounted for by a simple internal noise-averaging model. However, noise averaging alone was insufficient to account for improvements in individual and group-level performance as group size varied. These results indicate that sharing redundant information can enhance the quality of individual perceptual judgments and lead to better group decision making than dividing information across members of a group.

A perceptually completed whole is less than the sum of its parts.

How efficiently do people integrate the disconnected image fragments that fall on their eyes when they view partly occluded objects? In the present study, I used a psychophysical summation-at-threshold technique to address this question by measuring discrimination performance with both isolated and combined features of physically fragmented but perceptually complete objects. If visual completion promotes superior integration efficiency, performance with a visually completed object should exceed what would be expected from performance with the individual object parts shown in isolation. Contrary to this prediction, results showed that discrimination performance with both static and moving versions of physically fragmented but perceptually complete objects was significantly worse than would be expected from performance with their constituent parts. These results present a challenge for future theories of visual completion.

The perception of a familiar face is no more than the sum of its parts.

Why do faces become easier to recognize with repeated exposure? Previous research has suggested that familiarity may induce a qualitative shift in visual processing from an independent analysis of individual facial features to analysis that includes information about the relationships among features (Farah, Wilson, Drain, & Tanaka Psychological Review, 105, 482-498, 1998; Maurer, Grand, & Mondloch Trends in Cognitive Science, 6, 255-260, 2002). We tested this idea by using a "summation-at-threshold" technique (Gold, Mundy, & Tjan Psychological Science, 23, 427-434, 2012; Nandy & Tjan Journal of Vision, 8, 3.1-20, 2008), in which an observer's ability to recognize each individual facial feature shown independently is used to predict their ability to recognize all of the features shown in combination. We find that, although people are better overall at recognizing familiar as opposed to unfamiliar faces, their ability to integrate information across features is similar for unfamiliar and highly familiar faces and is well predicted by their ability to recognize each of the facial features shown in isolation. These results are consistent with the idea that familiarity has a quantitative effect on the efficiency with which information is extracted from individual features, rather than a qualitative effect on the process by which features are combined.

Information processing correlates of a size-contrast illusion.

Perception is often influenced by context. A well-known class of perceptual context effects is perceptual contrast illusions, in which proximate stimulus regions interact to alter the perception of various stimulus attributes, such as perceived brightness, color and size. Although the phenomenal reality of contrast effects is well documented, in many cases the connection between these illusions and how information is processed by perceptual systems is not well understood. Here, we use noise as a tool to explore the information processing correlates of one such contrast effect: the Ebbinghaus-Titchener size-contrast illusion. In this illusion, the perceived size of a central dot is significantly altered by the sizes of a set of surrounding dots, such that the presence of larger surrounding dots tends to reduce the perceived size of the central dot (and vise versa). In our experiments, we first replicated previous results that have demonstrated the subjective reality of the Ebbinghaus-Titchener illusion. We then used visual noise in a detection task to probe the manner in which observers processed information when experiencing the illusion. By correlating the noise with observers' classification decisions, we found that the sizes of the surrounding contextual elements had a direct influence on the relative weight observers assigned to regions within and surrounding the central element. Specifically, observers assigned relatively more weight to the surrounding region and less weight to the central region in the presence of smaller surrounding contextual elements. These results offer new insights into the connection between the subjective experience of size-contrast illusions and their associated information processing correlates.

Memory and incidental learning for visual frozen noise sequences.

Five experiments explored short-term memory and incidental learning for random visual spatio-temporal sequences. In each experiment, human observers saw samples of 8 Hz temporally-modulated 1D or 2D contrast noise sequences whose members were either uncorrelated across an entire 1-s long stimulus sequence, or comprised two frozen noise sequences that repeated identically between a stimulus' first and second 500 ms halves ("Repeated" noise). Presented with randomly intermixed stimuli of both types, observers judged whether each sequence repeated or not. Additionally, a particular exemplar of Repeated noise (a frozen or "Fixed Repeated" noise) was interspersed multiple times within a block of trials. As previously shown with auditory frozen noise stimuli (Agus, Thorpe, & Pressnitzer, 2010) recognition performance (d') increased with successive presentations of a Fixed Repeated stimulus, and exceeded performance with regular Repeated noise. However, unlike the case with auditory stimuli, learning of random visual stimuli was slow and gradual, rather than fast and abrupt. Reverse correlation revealed that contrasts occupying particular temporal positions within a sequence had disproportionately heavy weight in observers' judgments. A subsequent experiment suggested that this result arose from observers' uncertainty about the temporal mid-point of the noise sequences. Additionally, discrimination performance fell dramatically when a sequence of contrast values was repeated, but in reverse ("mirror image") order. This poor performance with temporal mirror images is strikingly different from vision's exquisite sensitivity to spatial mirror images.

Discrete-slots models of visual working-memory response times.

Much recent research has aimed to establish whether visual working memory (WM) is better characterized by a limited number of discrete all-or-none slots or by a continuous sharing of memory resources. To date, however, researchers have not considered the response-time (RT) predictions of discrete-slots versus shared-resources models. To complement the past research in this field, we formalize a family of mixed-state, discrete-slots models for explaining choice and RTs in tasks of visual WM change detection. In the tasks under investigation, a small set of visual items is presented, followed by a test item in 1 of the studied positions for which a change judgment must be made. According to the models, if the studied item in that position is retained in 1 of the discrete slots, then a memory-based evidence-accumulation process determines the choice and the RT; if the studied item in that position is missing, then a guessing-based accumulation process operates. Observed RT distributions are therefore theorized to arise as probabilistic mixtures of the memory-based and guessing distributions. We formalize an analogous set of continuous shared-resources models. The model classes are tested on individual subjects with both qualitative contrasts and quantitative fits to RT-distribution data. The discrete-slots models provide much better qualitative and quantitative accounts of the RT and choice data than do the shared-resources models, although there is some evidence for "slots plus resources" when memory set size is very small.

The efficiency of dynamic and static facial expression recognition.

Unlike frozen snapshots of facial expressions that we often see in photographs, natural facial expressions are dynamic events that unfold in a particular fashion over time. But how important are the temporal properties of expressions for our ability to reliably extract information about a person's emotional state? We addressed this question experimentally by gauging human performance in recognizing facial expressions with varying temporal properties relative to that of a statistically optimal ("ideal") observer. We found that people recognized emotions just as efficiently when viewing them as naturally evolving dynamic events, temporally reversed events, temporally randomized events, or single images frozen in time. Our results suggest that the dynamic properties of human facial movements may play a surprisingly small role in people's ability to infer the emotional states of others from their facial expressions.

Inversion effects in face-selective cortex with combinations of face parts.

The face inversion effect has been used as a basis for claims about the specialization of face-related perceptual and neural processes. One of these claims is that the fusiform face area (FFA) is the site of face-specific feature-based and/or configural/holistic processes that are responsible for producing the face inversion effect. However, the studies on which these claims were based almost exclusively used stimulus manipulations of whole faces. Here, we tested inversion effects using single, discrete features and combinations of multiple discrete features, in addition to whole faces, using both behavioral and fMRI measurements. In agreement with previous studies, we found behavioral inversion effects with whole faces and no inversion effects with a single eye stimulus or the two eyes in combination. However, we also found behavioral inversion effects with feature combination stimuli that included features in the top and bottom halves (eyes-mouth and eyes-nose-mouth). Activation in the FFA showed an inversion effect for the whole-face stimulus only, which did not match the behavioral pattern. Instead, a pattern of activation consistent with the behavior was found in the bilateral inferior frontal gyrus, which is a component of the extended face-preferring network. The results appear inconsistent with claims that the FFA is the site of face-specific feature-based and/or configural/holistic processes that are responsible for producing the face inversion effect. They are more consistent with claims that the FFA shows a stimulus preference for whole upright faces.

The response of face-selective cortex with single face parts and part combinations.

A critical issue in object recognition research is how the parts of an object are analyzed by the visual system and combined into a perceptual whole. However, most of the previous research has examined how changes to object parts influence recognition of the whole, rather than recognition of the parts themselves. This is particularly true of the research on face recognition, and especially with questions related to the neural substrates. Here, we investigated patterns of BOLD fMRI brain activation with internal face parts (features) presented singly and in different combinations. A preference for single features over combinations was found in the occipital face area (OFA) as well as a preference for the two-eyes combination stimulus over other combination stimulus types. The fusiform face area (FFA) and lateral occipital cortex (LO) showed no preferences among the single feature and combination stimulus types. The results are consistent with a growing view that the OFA represents processes involved in early, feature-based analysis.

The perception of a face is no more than the sum of its parts.

When you see a person's face, how do you go about combining his or her facial features to make a decision about who that person is? Most current theories of face perception assert that the ability to recognize a human face is not simply the result of an independent analysis of individual features, but instead involves a holistic coding of the relationships among features. This coding is thought to enhance people's ability to recognize a face beyond what would be expected if each feature were shown in isolation. In the study reported here, we explicitly tested this idea by comparing human performance on facial-feature integration with that of an optimal Bayesian integrator. Contrary to the predictions of most current notions of face perception, our findings showed that human observers integrate facial features in a manner that is no better than would be predicted by their ability to use each individual feature when shown in isolation. That is, a face is perceived no better than the sum of its individual parts.

Pattern recognition in correlated and uncorrelated noise.

This study examined how correlated, or filtered, noise affected efficiency for recognizing two types of signal patterns, Gabor patches and three-dimensional objects. In general, compared with the ideal observer, human observers were most efficient at performing tasks in low-pass noise, followed by white noise; they were least efficient in high-pass noise. Simulations demonstrated that contrast-dependent internal noise was likely to have limited human performance in the high-pass conditions for both signal types. Classification images showed that observers were likely adopting different strategies in the presence of low-pass versus white noise. However, efficiencies were underpredicted by the linear classification images and asymmetries were present in the classification subimages, indicating the influence of nonlinear processes. Response consistency analyses indicated that lower contrast-dependent internal noise contributed somewhat to higher efficiencies in low-pass noise for Gabor patches but not objects. Taken together, the results of these experiments suggest a complex interaction among signals, external noise spectra, and internal noise in determining efficiency in correlated and uncorrelated noise.