Discrimination of natural scenes in central and peripheral vision.

We conducted suprathreshold discrimination experiments to compare how natural-scene information is processed in central and peripheral vision (16° eccentricity). Observers' ratings of the perceived magnitude of changes in naturalistic scenes were lower for peripheral than for foveal viewing, and peripheral orientation changes were rated less than peripheral colour changes. A V1-based Visual Difference Predictor model of the magnitudes of perceived foveal change was adapted to match the sinusoidal grating sensitivities of peripheral vision, but it could not explain why the ratings for changes in peripheral stimuli were so reduced. Perceived magnitude ratings for peripheral stimuli were further reduced by simultaneous presentation of flanking patches of naturalistic images, a phenomenon that could not be replicated foveally, even after M-scaling the foveal stimuli to reduce their size and the distances from the flankers. The effects of the peripheral flankers are very reminiscent of crowding phenomena demonstrated with letters or Gabor patches.

A general rule for sensory cue summation: evidence from photographic, musical, phonetic and cross-modal stimuli.

The Euclidean and MAX metrics have been widely used to model cue summation psychophysically and computationally. Both rules happen to be special cases of a more general Minkowski summation rule , where m = 2 and ∞, respectively. In vision research, Minkowski summation with power m = 3-4 has been shown to be a superior model of how subthreshold components sum to give an overall detection threshold. Recently, we have previously reported that Minkowski summation with power m = 2.84 accurately models summation of suprathreshold visual cues in photographs. In four suprathreshold discrimination experiments, we confirm the previous findings with new visual stimuli and extend the applicability of this rule to cue combination in auditory stimuli (musical sequences and phonetic utterances, where m = 2.95 and 2.54, respectively) and cross-modal stimuli (m = 2.56). In all cases, Minkowski summation with power m = 2.5-3 outperforms the Euclidean and MAX operator models. We propose that this reflects the summation of neuronal responses that are not entirely independent but which show some correlation in their magnitudes. Our findings are consistent with electrophysiological research that demonstrates signal correlations (r = 0.1-0.2) between sensory neurons when these are presented with natural stimuli.

Summation of perceptual cues in natural visual scenes.

Natural visual scenes are rich in information, and any neural system analysing them must piece together the many messages from large arrays of diverse feature detectors. It is known how threshold detection of compound visual stimuli (sinusoidal gratings) is determined by their components' thresholds. We investigate whether similar combination rules apply to the perception of the complex and suprathreshold visual elements in naturalistic visual images. Observers gave magnitude estimations (ratings) of the perceived differences between pairs of images made from photographs of natural scenes. Images in some pairs differed along one stimulus dimension such as object colour, location, size or blur. But, for other image pairs, there were composite differences along two dimensions (e.g. both colour and object-location might change). We examined whether the ratings for such composite pairs could be predicted from the two ratings for the respective pairs in which only one stimulus dimension had changed. We found a pooling relationship similar to that proposed for simple stimuli: Minkowski summation with exponent 2.84 yielded the best predictive power (r=0.96), an exponent similar to that generally reported for compound grating detection. This suggests that theories based on detecting simple stimuli can encompass visual processing of complex, suprathreshold stimuli.

Stability of the color-opponent signals under changes of illuminant in natural scenes.

Illumination varies greatly both across parts of a natural scene and as a function of time, whereas the spectral reflectance function of surfaces remains more stable and is of much greater relevance when searching for specific targets. This study investigates the functional properties of postreceptoral opponent-channel responses, in particular regarding their stability against spatial and temporal variation in illumination. We studied images of natural scenes obtained in UK and Uganda with digital cameras calibrated to produce estimated L-, M-, and S-cone responses of trichromatic primates (human) and birds (starling). For both primates and birds we calculated luminance and red-green opponent (RG) responses. We also calculated a primate blue-yellow-opponent (BY) response. The BY response varies with changes in illumination, both across time and across the image, rendering this factor less invariant. The RG response is much more stable than the BY response across such changes in illumination for primates, less so for birds. These differences between species are due to the greater separation of bird L and M cones in wavelength and the narrower bandwidth of the cone action spectra. This greater separation also produces a larger chromatic signal for a given change in spectral reflectance. Thus bird vision seems to suffer a greater degree of spatiotemporal "clutter" than primate vision, but also enhances differences between targets and background. Therefore, there may be a trade-off between the degree of chromatic clutter in a visual system versus the degree of chromatic difference between a target and its background. Primate and bird visual systems have found different solutions to this trade-off.

The effects of amplitude-spectrum statistics on foveal and peripheral discrimination of changes in natural images, and a multi-resolution model.

Psychophysical thresholds were measured for discriminating small changes in spatial features of naturalistic scenes (morph sequences), for foveal and peripheral vision, and under M-scaling. Sensitivity was greatest for scenes with near natural Fourier amplitude slope, perhaps implying that human vision is optimised for natural scene statistics. A low-level model calculated differences in local contrast between pairs of images within a few spatial frequency channels with bandwidth like neurons in V1. The model was "customised" to each observer's contrast sensitivity function for sinusoidal gratings, and it could replicate the "U-shaped" relationships between discrimination threshold and spectral slope, and many differences between picture sets and observers. A single-channel model and an ideal-observer analysis both failed to capture the U-shape.

Visual search in Alzheimer's disease: a deficiency in processing conjunctions of features.

Human vision often needs to encode multiple characteristics of many elements of the visual field, for example their lightness and orientation. The paradigm of visual search allows a quantitative assessment of the function of the underlying mechanisms. It measures the ability to detect a target element among a set of distractor elements. We asked whether Alzheimer's disease (AD) patients are particularly affected in one type of search, where the target is defined by a conjunction of features (orientation and lightness) and where performance depends on some shifting of attention. Two non-conjunction control conditions were employed. The first was a pre-attentive, single-feature, "pop-out" task, detecting a vertical target among horizontal distractors. The second was a single-feature, partly attentive task in which the target element was slightly larger than the distractors-a "size" task. This was chosen to have a similar level of attentional load as the conjunction task (for the control group), but lacked the conjunction of two features. In an experiment, 15 AD patients were compared to age-matched controls. The results suggested that AD patients have a particular impairment in the conjunction task but not in the single-feature size or pre-attentive tasks. This may imply that AD particularly affects those mechanisms which compare across more than one feature type, and spares the other systems and is not therefore simply an 'attention-related' impairment. Additionally, these findings show a double dissociation with previous data on visual search in Parkinson's disease (PD), suggesting a different effect of these diseases on the visual pathway.

Age-related changes in the preattentional detection of visual change.

The ability to detect changes in the environment which occur outside the focus of current awareness is essential if the individual is to be able to divert attention to biologically salient stimuli. The preattentional mechanism underlying the automatic detection of stimulus change in the auditory modality has been extensively studied by recording an event-related potential known as the mismatch negativity. Recently a homologous response from the visual cortex has also been described. Ageing has been shown to affect the efficiency of preattentional processing in the auditory modality, a factor which may contribute to cognitive changes in the elderly. It is unclear whether a similar effect occurs in the visual system. To investigate this issue the visual mismatch negativity was recorded from 12 older adults and 24 younger adults. Whereas the younger adults displayed a robust visual MMN, that evoked in the older adults was significantly reduced in amplitude. The results are indicative of age-related deficits in automatic visual processing.

Spatiochromatic properties of natural images and human vision.

The human visual system shows a relatively greater response to low spatial frequencies of chromatic spatial modulation than to luminance spatial modulation. However, previous work has shown that this differential sensitivity to low spatial frequencies is not reflected in any differential luminance and chromatic content of general natural scenes. This is contrary to the prevailing assumption that the spatial properties of human vision ought to reflect the structure of natural scenes. Now, colorimetric measures of scenes suggest that red-green (and perhaps blue-yellow) color discrimination in primates is particularly suited to the encoding of specific scenes: reddish or yellowish objects on a background of leaves. We therefore ask whether the spatial, as well as chromatic, properties of such scenes are matched to the different spatial-encoding properties of color and luminance modulation in human vision. We show that the spatiochromatic properties of a wide class of scenes, which contain reddish objects (e.g., fruit) on a background of leaves, correspond well to the properties of the red-green (but not blue-yellow) systems in human vision, at viewing distances commensurate with typical grasping distance. This implies that the red-green system is particularly suited to encoding both the spatial and the chromatic structure of such scenes.

The pupillary light reflex in aging and Alzheimer's disease.

The pupillary light reflex is reported to be reduced in amplitude in Alzheimer's disease (AD). The purpose of this study was to determine whether this effect is measurable under conditions typical of clinical rather than laboratory settings. A head-mounted infra-red videopupillometer was used to measure the amplitude of pupillary constriction in 12 patients with probable AD, 12 healthy age-matched older adults and 12 young adults. The constriction to the onset of bright light relative to the resting amplitude was significantly reduced in AD compared with both control groups. This result is consistent with an acetylcholine-related deficit in AD and supports the findings of Prettyman et al. and Fotiou et al. The impairment is likely to be caused by degeneration in relays in the midbrain but cholinergic deficits in the peripheral parasympathetic pathway cannot be excluded. The variation in pupillary response between individuals may preclude its use for diagnostic purposes. However, if the changes in pupillary response in AD are related to change in neurotransmitter status, then the value of such a technique may be in its use in providing an objective, non-invasive monitor of physiological abnormality with which to follow disease progression and treatment efficacy.

The human visual system is optimised for processing the spatial information in natural visual images.

A fundamental tenet of visual science is that the detailed properties of visual systems are not capricious accidents, but are closely matched by evolution and neonatal experience to the environments and lifestyles in which those visual systems must work. This has been shown most convincingly for fish and insects. For mammalian vision, however, this tenet is based more upon theoretical arguments than upon direct observations. Here, we describe experiments that require human observers to discriminate between pictures of slightly different faces or objects. These are produced by a morphing technique that allows small, quantifiable changes to be made in the stimulus images. The independent variable is designed to give increasing deviation from natural visual scenes, and is a measure of the Fourier composition of the image (its second-order statistics). Performance in these tests was best when the pictures had natural second-order spatial statistics, and degraded when the images were made less natural. Furthermore, performance can be explained with a simple model of contrast coding, based upon the properties of simple cells in the mammalian visual cortex. The findings thus provide direct empirical support for the notion that human spatial vision is optimised to the second-order statistics of the optical environment.

A comparison between human and machine labelling of image regions.

In previous work (Campbell et al, 1997 Pattern Recognition 30 555-563) a vision system was developed which is capable of classifying objects in outdoor scenes. The approach involves segmenting the image into regions, obtaining a feature-based description of each region, and then passing this description on to an artificial neural network (ANN) which has been trained to label the region with one of eleven possible object types. The question addressed here is: how important is each of these features to overall performance, both in human and machine vision? A set of experiments was conducted in which human subjects were trained in the same labelling task as the ANN. The stimuli, each depicting a single image region, were generated from a large database of urban and rural images. The subjects were then tested on both intact and degraded stimuli. The results suggest that certain features are particularly influential in mediating overall labelling performance. An equivalent experiment was carried out with the ANN. A method is presented which allows individual features to be corrupted in such a way as to simulate the loss of certain forms of visual information. The results, which are broadly similar to those found in the previous experiment, imply that the ANN can provide a useful model of human image region labelling. It is anticipated that the methodology, which draws on both computational and psychophysical techniques, will be of use to other areas of investigation.

Influence of sensitivity on response bias in taste and audition.

A detection theoretic analysis was employed to examine sensitivity and response bias in two modalities. In Experiment 1, 6 tasters made same-different judgments about the concentration of either sucrose or quinine in pairs of tonic water samples. The beverages were colored, but color was not predictive of the concentration of the sweet or bitter ingredient. When same-different ratings were collapsed to approximate the outcome of a categorical decision, tasters with poorer sensitivity appear to have adopted more extreme response criteria than did tasters with greater sensitivity, irrespective of taste quality, color, or whether pairs of solutions comprised the same or different colors. In Experiment 2, 3 individuals discriminated pairs of 1000-Hz sinusoids differing in amplitude. Six amplitude differences were tested. Rating-scale versions of two paradigms: The single-interval yes-no task and the two-interval same-different task were used to measure sensitivity and bias. There was a preponderance of "same" responses in the same-different task. Estimates of bias obtained from collapsed ratings in both tasks were unaffected by sensitivity, but a consideration of the range over which sets of criteria were spread suggested a general tendency toward more conservative response biases as sensitivity declined.

Mismatch negativity in the visual modality.

In the auditory system, the automatic detection of stimulus change provides a mechanism for switching attention to biologically significant events. It gives rise to the mismatch negativity (MMN) event related potential. It is unclear whether a similar mechanism exists in vision. To investigate this issue, evoked potentials were recorded to target stimuli in the centre of the visual field, and to frequent standard and infrequent deviant stimuli presented outside the focus of attention, in the peripheral field. Deviants evoked a more negative potential than standards 250-400 ms after the stimulus. The negativity, distributed over supplementary visual areas of occipital and posterior temporal cortex, was associated with the rarity of the deviants and not the physical features which distinguished them from standards. This negativity shares a number of characteristics with auditory MMN.

Color and luminance information in natural scenes.

The spatial filtering applied by the human visual system appears to be low pass for chromatic stimuli and band pass for luminance stimuli. Here we explore whether this observed difference in contrast sensitivity reflects a real difference in the components of chrominance and luminance in natural scenes. For this purpose a digital set of 29 hyperspectral images of natural scenes was acquired and its spatial frequency content analyzed in terms of chrominance and luminance defined according to existing models of the human cone responses and visual signal processing. The statistical 1/f amplitude spatial-frequency distribution is confirmed for a variety of chromatic conditions across the visible spectrum. Our analysis suggests that natural scenes are relatively rich in high-spatial-frequency chrominance information that does not appear to be transmitted by the human visual system. This result is unlikely to have arisen from errors in the original measurements. Several reasons may combine to explain a failure to transmit high-spatial-frequency chrominance: (a) its minor importance for primate visual tasks, (b) its removal by filtering applied to compensate for chromatic aberration of the eye's optics, and (c) a biological bottleneck blocking its transmission. In addition, we graphically compare the ratios of luminance to chrominance measured by our hyperspectral camera and those measured psychophysically over an equivalent spatial-frequency range.

Complete sparing of high-contrast color input to motion perception in cortical color blindness.

It is widely held that color and motion are processed by separate parallel pathways in the visual system, but this view is difficult to reconcile with the fact that motion can be detected in equiluminant stimuli that are defined by color alone. To examine the relationship between color and motion, we tested three patients who had lost their color vision following cortical damage (central achromatopsia). Despite their profound loss in the subjective experience of color and their inability to detect the motion of faint colors, all three subjects showed surprisingly strong responses to high-contrast, moving color stimuli--equal in all respects to the performance of subjects with normal color vision. The pathway from opponent-color detectors in the retina to the motion analysis areas must therefore be independent of the damaged color centers in the occipitotemporal area. It is probably also independent of the motion analysis area MT/V5, because the contribution of color to motion detection in these patients is much stronger than the color response of monkey area MT.

Automatic segmentation and classification of outdoor images using neural networks.

The paper describes how neural networks may be used to segment and label objects in images. A self-organising feature map is used for the segmentation phase, and we quantify the quality of the segmentations produced as well as the contribution made by colour and texture features. A multi-layer perception is trained to label the regions produced by the segmentation process. It is shown that 91.1% of the image area is correctly classified into one of eleven categories which include cars, houses, fences, roads, vegetation and sky.

Human colour discrimination based on a non-parvocellular pathway.

BACKGROUND: Traditionally, colour information is assumed to be carried by neural channels in the parvocellular pathway and to be encoded in an opponent manner, while other, non-parvocellular, spectrally non-opponent channels are thought to play no part in colour vision. But is the parvocellular pathway the only way that colours can be discriminated in human vision? We studied two patients with cerebral achromatopsia, who lack conscious colour perception but are nevertheless able to make use of colour information. In particular, we investigated whether, in these patients, colour discrimination is mediated by the parvocellular pathway. RESULTS: The achromatopsic patients carried out a forced-choice colour- and luminance-discrimination task, and showed clear evidence of unconscious colour processing, consistent with previous studies. We added different types of luminance noise to see when this unconscious colour information could be masked. The results of the colour-discrimination-with-noise and the brightness-non-additivity experiments showed a double-dissociation between patients. This indicates that, in one patient, unconscious colour discrimination may be subserved by a spectrally non-opponent mechanism, which does not have the characteristics of the parvocellular pathway and which is responsive to fast flicker. Spectral sensitivity, contrast sensitivity and motion perception experiments confirmed that this patient lacks a working opponent parvocellular system. The second achromatopsic patient showed evidence of a residual parvocellular system. CONCLUSIONS: Our results show that chromatic discrimination need not be mediated by neural mechanisms, the parvocellular system in particular, normally assumed to subserve conscious colour perception. Such discrimination may be mediated by a neural subsystem which responds to fast flicker, is spectrally non-opponent, and supports normal motion perception.

Perception of emotion from dynamic point-light displays represented in dance.

It is well known that biological motion, as produced by point-light displays on a human body, gives a good representation of the represented body-eg its gender and the nature of the task which it is engaged in. The question is whether it is possible to judge the emotional state of a human body from motion information alone. An ability to make this kind of judgment may imply that people are able to perceive emotion from patterns of movement without having to compute the detailed shape first. Subjects were shown brief video clips of two trained dancers (one male, one female). The dancers were aiming to convey the following emotions: fear, anger, grief, joy, surprise, and disgust. The video clips portrayed fully lit scenes and point-light scenes, with thirteen small points of light attached to the body of each dancer. Half the stimuli were presented the right way up, while half were inverted. The subjects' task was to judge which emotion was being portrayed. Full-body clips gave good recognition of emotionality (88% correct), but the results for upright biological-motion displays were also significantly above chance (63% correct). Inversion of the display reduced biological-motion (but not full-body) performance to close to chance but still significantly above chance. A space-time analysis of the motion of the points of light was carried out, and was related to the discriminability of the different emotions. Biological-motion displays, which convey no information while static, are able to give a rich description of the subject matter, including the ability to judge emotional state. This ability is disrupted when the image is inverted.

Is the richness of our visual world an illusion? Transsaccadic memory for complex scenes.

Our construction of a stable visual world, despite the presence of saccades, is discussed. A computer-graphics method was used to explore transsaccadic memory for complex images. Images of real-life scenes were presented under four conditions: they stayed still or moved in an unpredictable direction (forcing an eye movement), while simultaneously changing or staying the same. Changes were the appearance, disappearance, or rotation of an object in the scene. Subjects detected the changes easily when the image did not move but when it moved their performance fell to chance. A grey-out period was introduced to mimic that which occurs during a saccade. This also reduced performance but not to chance levels. These results reveal the poverty of transsaccadic memory for real-life complex scenes. They are discussed with respect to Dennett's view that much less information is available in vision than our subjective impression leads us to believe. Our stable visual world may be constructed out of a brief retinal image and a very sketchy, higher-level representation along with a pop-out mechanism to redirect attention. The richness of our visual world is, to this extent, an illusion.