Natural scenes reveal diverse representations of 2D and 3D body pose in the human brain.

Human pose, defined as the spatial relationships between body parts, carries instrumental information supporting the understanding of motion and action of a person. A substantial body of previous work has identified cortical areas responsive to images of bodies and different body parts. However, the neural basis underlying the visual perception of body part relationships has received less attention. To broaden our understanding of body perception, we analyzed high-resolution fMRI responses to a wide range of poses from over 4,000 complex natural scenes. Using ground-truth annotations and an application of three-dimensional (3D) pose reconstruction algorithms, we compared similarity patterns of cortical activity with similarity patterns built from human pose models with different levels of depth availability and viewpoint dependency. Targeting the challenge of explaining variance in complex natural image responses with interpretable models, we achieved statistically significant correlations between pose models and cortical activity patterns (though performance levels are substantially lower than the noise ceiling). We found that the 3D view-independent pose model, compared with two-dimensional models, better captures the activation from distinct cortical areas, including the right posterior superior temporal sulcus (pSTS). These areas, together with other pose-selective regions in the LOTC, form a broader, distributed cortical network with greater view-tolerance in more anterior patches. We interpret these findings in light of the computational complexity of natural body images, the wide range of visual tasks supported by pose structures, and possible shared principles for view-invariant processing between articulated objects and ordinary, rigid objects.

The effect of illumination on the visibility of steps and ramps for people with low vision.

SIGNIFICANCE: Poor visibility of indoor features such as steps and ramps can pose mobility hazards for people with low vision. For purposes of architectural design, it is important to understand how design parameters such as the illumination level of an indoor space affect the visibility of steps and ramps. PURPOSE: This study was aimed to examine the effect of typical variation in photopic illumination level in an indoor space on the visibility of steps and ramps for individuals with low vision. METHODS: Steps and ramps were constructed in a large windowless room illuminated by overhead lights. Subjects with low vision completed a 5-alternative forced choice task to recognize the targets at three levels of photopic illumination, i.e., 800, 80, and 8 lux, and gave confidence ratings about their judgments on a 5-point scale. Acuities and contrast sensitivities of the subjects were also measured at each illumination level. For comparison, a group of normally sighted subjects with simulated acuity reduction also completed the step-and-ramp recognition task. RESULTS: For both groups of subjects, recognition accuracy was not affected by illumination level. For subjects with low vision, however, there was a significant effect of illumination level on confidence rating: subjects became more confident about their judgments with increasing illumination. There was also a weak effect of illumination level on acuity and contrast sensitivity, both worsening with decreasing illumination. Recognition performance was best predicted by contrast sensitivity, whereas confidence was best predicted by visual acuity. CONCLUSIONS: Illumination variation over a typical photopic range in an indoor space had minimal effect on the objective visibility of steps and ramps for people with low vision. However, illumination level affected subjects' confidence in hazard recognition. Design decisions on parameters such as illumination should consider the consequences on both the objective and the subjective accessibility of a space.

Effect of expansive optic flow and lateral motion parallax on depth estimation with normal and artificially reduced acuity.

When an observer moves in space, the retinal projection of a stationary object either expands if the motion is toward the object or shifts horizontally if the motion contains a lateral component. This study examined the impact of expansive optic flow and lateral motion parallax on the accuracy of depth perception for observers with normal or artificially reduced acuity and asked whether any benefit is due to the continuous motion or to the discrete object image displacement. Stationary participants viewed a virtual room on a computer screen. They used an on-screen slider to estimate the depth of a target object relative to a reference object after seeing 2-second videos simulating five conditions: static viewing, expansive optic flow, and lateral motion parallax in either continuous motion or image displacement. Ten participants viewed the stimuli with normal acuity in Experiment 1 and 11 with three levels of artificially reduced acuity in Experiment 2. Linear regression models represented the relationship between the depth estimates of participants and the ground truth. Lateral motion parallax produced more accurate depth estimates than expansive optic flow and static viewing. Depth perception with continuous motion was more accurate than that with displacement under mild and moderate, but not severe, acuity reduction. For observers with both normal and artificially reduced acuity, lateral motion parallax was more helpful for object depth estimation than expansive optic flow, and continuous motion parallax was more helpful than object image displacement.

Visual adaptation selective for individual limbs reveals hierarchical human body representation.

The spatial relationships between body parts are a rich source of information for person perception, with even simple pairs of parts providing highly valuable information. Computation of these relationships would benefit from a hierarchical representation, where body parts are represented individually. We hypothesized that the human visual system makes use of such representations. To test this hypothesis, we used adaptation to determine whether observers were sensitive to changes in the length of one body part relative to another. Observers viewed forearm/upper arm pairs where the forearm had been either lengthened or shortened, judging the perceived length of the forearm. Observers then adapted to a variety of different stimuli (e.g., arms, objects, etc.) in different orientations and visual field locations. We found that following adaptation to distorted limbs, but not non-limb objects, observers experienced a shift in perceived forearm length. Furthermore, this effect partially transferred across different orientations and visual field locations. Taken together, these results suggest the effect arises in high level mechanisms specialized for specific body parts, providing evidence for a representation of bodies based on parts and their relationships.

Temporally flexible feedback signal to foveal cortex for peripheral object recognition.

Recent studies have shown that information from peripherally presented images is present in the human foveal retinotopic cortex, presumably because of feedback signals. We investigated this potential feedback signal by presenting noise in fovea at different object-noise stimulus onset asynchronies (SOAs), whereas subjects performed a discrimination task on peripheral objects. Results revealed a selective impairment of performance when foveal noise was presented at 250-ms SOA, but only for tasks that required comparing objects' spatial details, suggesting a task- and stimulus-dependent foveal processing mechanism. Critically, the temporal window of foveal processing was shifted when mental rotation was required for the peripheral objects, indicating that the foveal retinotopic processing is not automatically engaged at a fixed time following peripheral stimulation; rather, it occurs at a stage when detailed information is required. Moreover, fMRI measurements using multivoxel pattern analysis showed that both image and object category-relevant information of peripheral objects was represented in the foveal cortex. Taken together, our results support the hypothesis of a temporally flexible feedback signal to the foveal retinotopic cortex when discriminating objects in the visual periphery.

Simulating visibility under reduced acuity and contrast sensitivity.

Architects and lighting designers have difficulty designing spaces that are accessible to those with low vision, since the complex nature of most architectural spaces requires a site-specific analysis of the visibility of mobility hazards and key landmarks needed for navigation. We describe a method that can be utilized in the architectural design process for simulating the effects of reduced acuity and contrast on visibility. The key contribution is the development of a way to parameterize the simulation using standard clinical measures of acuity and contrast sensitivity. While these measures are known to be imperfect predictors of visual function, they provide a way of characterizing general levels of visual performance that is familiar to both those working in low vision and our target end-users in the architectural and lighting-design communities. We validate the simulation using a letter-recognition task.

The perceptual dimensions of natural dynamic flow.

We measured perceptual judgments of category, material attributes, affordances, and similarity to investigate the perceptual dimensions underlying the visual representation of a broad class of natural dynamic flows (sea waves, smoke, and windblown foliage). The dynamic flows were looped 3-s movies windowed with circular apertures of two sizes to manipulate the level of spatial context. In low levels of spatial context (smaller apertures), human observers' judgments of material attributes and affordances were inaccurate, with estimates biased toward assumptions that the flows resulted from objects that were rigid, "pick-up-able," and not penetrable. The similarity arrangements showed dynamic flow clusters based partly on material, but dominated by color appearance. In high levels of spatial context (large apertures), observers reliably estimated material categories and their attributes. The similarity arrangements were based primarily on categories related to external, physical causes. Representational similarity analysis suggests that while shallow dimensions like color sometimes account for inferences of physical causes in the low-context condition, shallow dimensions cannot fully account for these inferences in the high-context condition. For the current broad data set of dynamic flows, the perceptual dimensions that best account for the similarity arrangements in the high-context condition are related to the intermolecular bond strength of a material's underlying physical structure. These arrangements are also best related to affordances that underlie common motor activities. Thus, the visual system appears to use an efficient strategy to resolve flow ambiguity; vision will sometimes rely on local, image-based, statistical properties that can support reliable inference of external physical causes, and other times it uses deeper causal knowledge to interpret and use flow information to the extent that it is useful for everyday action decisions.

Responses in early visual areas to contour integration are context dependent.

It has been shown that early visual areas are involved in contour processing. However, it is not clear how local and global context interact to influence responses in those areas, nor has the interarea coordination that yields coherent structural percepts been fully studied, especially in human observers. In this study, we used functional magnetic resonance imaging (fMRI) to measure activity in early visual cortex while observers performed a contour detection task in which alignment of Gabor elements and background clutter were manipulated. Six regions of interest (two regions, containing either the cortex representing the target or the background clutter, in each of areas V1, V2, and V3) were predefined using separate target versus background functional localizer scans. The first analysis using a general linear model showed that in the presence of background clutter, responses in V1 and V2 target regions of interest were significantly stronger to aligned than unaligned contours, whereas when background clutter was absent, no significant difference was observed. The second analysis using interarea correlations showed that with background clutter, there was an increase in V1-V2 coordination within the target regions when perceiving aligned versus unaligned contours; without clutter, however, correlations between V1 and V2 were similar no matter whether aligned contours were present or not. Both the average response magnitude and the connectivity analysis suggest different mechanisms support contour processing with or without background distractors. Coordination between V1 and V2 may play a major role in coherent structure perception, especially with complex scene organization.

Scene coherence can affect the local response to natural images in human V1.

Neurons in primary visual cortex (V1) can be indirectly affected by visual stimulation positioned outside their receptive fields. Although this contextual modulation has been intensely studied, we have little notion of how it manifests with naturalistic stimulation. Here, we investigated how the V1 response to a natural image fragment is affected by spatial context that is consistent or inconsistent with the scene from which it was extracted. Using functional magnetic resonance imaging at 7 T, we measured the blood oxygen level-dependent signal in human V1 (n = 8) while participants viewed an array of apertures. Most apertures showed fragments from a single scene, yielding a dominant perceptual interpretation which participants were asked to categorize, and the remaining apertures each showed fragments drawn from a set of 20 scenes. We find that the V1 response was significantly increased for apertures showing image structure that was coherent with the dominant scene relative to the response to the same image structure when it was non-coherent. Additional analyses suggest that this effect was mostly evident for apertures in the periphery of the visual field, that it peaked towards the centre of the aperture, and that it peaked in the middle to superficial regions of the cortical grey matter. These findings suggest that knowledge of typical spatial relationships is embedded in the circuitry of contextual modulation. Such mechanisms, possibly augmented by contributions from attentional factors, serve to increase the local V1 activity under conditions of contextual consistency.

Differences in perceptual learning transfer as a function of training task.

A growing body of research--including results from behavioral psychology, human structural and functional imaging, single-cell recordings in nonhuman primates, and computational modeling--suggests that perceptual learning effects are best understood as a change in the ability of higher-level integration or association areas to read out sensory information in the service of particular decisions. Work in this vein has argued that, depending on the training experience, the "rules" for this read-out can either be applicable to new contexts (thus engendering learning generalization) or can apply only to the exact training context (thus resulting in learning specificity). Here we contrast learning tasks designed to promote either stimulus-specific or stimulus-general rules. Specifically, we compare learning transfer across visual orientation following training on three different tasks: an orientation categorization task (which permits an orientation-specific learning solution), an orientation estimation task (which requires an orientation-general learning solution), and an orientation categorization task in which the relevant category boundary shifts on every trial (which lies somewhere between the two tasks above). While the simple orientation-categorization training task resulted in orientation-specific learning, the estimation and moving categorization tasks resulted in significant orientation learning generalization. The general framework tested here--that task specificity or generality can be predicted via an examination of the optimal learning solution--may be useful in building future training paradigms with certain desired outcomes.

Regions of mid-level human visual cortex sensitive to the global coherence of local image patches.

The global structural arrangement and spatial layout of the visual environment must be derived from the integration of local signals represented in the lower tiers of the visual system. This interaction between the spatially local and global properties of visual stimulation underlies many of our visual capacities, and how this is achieved in the brain is a central question for visual and cognitive neuroscience. Here, we examine the sensitivity of regions of the posterior human brain to the global coordination of spatially displaced naturalistic image patches. We presented observers with image patches in two circular apertures to the left and right of central fixation, with the patches drawn from either the same (coherent condition) or different (noncoherent condition) extended image. Using fMRI at 7T (n = 5), we find that global coherence affected signal amplitude in regions of dorsal mid-level cortex. Furthermore, we find that extensive regions of mid-level visual cortex contained information in their local activity pattern that could discriminate coherent and noncoherent stimuli. These findings indicate that the global coordination of local naturalistic image information has important consequences for the processing in human mid-level visual cortex.

Attention modulates neuronal correlates of interhemispheric integration and global motion perception.

In early retinotopic areas of the human visual system, information from the left and right visual hemifields (VHFs) is processed contralaterally in two hemispheres. Despite this segregation, we have the perceptual experience of a unified, coherent, and uninterrupted single visual field. How exactly the visual system integrates information from the two VHFs and achieves this perceptual experience still remains largely unknown. In this study using fMRI, we explored candidate areas that are involved in interhemispheric integration and the perceptual experience of a unified, global motion across VHFs. Stimuli were two-dimensional, computer-generated objects with parts in both VHFs. The retinal image in the left VHF always remained stationary, but in the experimental condition, it appeared to have local motion because of the perceived global motion of the object. This perceptual effect could be weakened by directing the attention away from the global motion through a demanding fixation task. Results show that lateral occipital areas, including the medial temporal complex, play an important role in the process of perceptual experience of a unified global motion across VHFs. In early areas, including the lateral geniculate nucleus and V1, we observed correlates of this perceptual experience only when attention is not directed away from the object. These findings reveal effects of attention on interhemispheric integration in motion perception and imply that both the bilateral activity of higher-tier visual areas and feedback mechanisms leading to bilateral activity of early areas play roles in the perceptual experience of a unified visual field.

Estimating lighting direction in scenes with multiple objects.

To recover the reflectance and shape of an object in a scene, the human visual system must account for the properties of the light illuminating the object. Here, we examine the extent to which multiple objects within a scene are utilised to estimate the direction of lighting in a scene. In Experiment 1, we presented participants with rendered scenes that contained 1, 9, or 25 unfamiliar blob-like objects and measured their capacity to discriminate whether a directional light source was left or right of the participants' vantage point. Trends reported for ensemble perception suggest that the number of utilised objects-and, consequently, discrimination sensitivity-would increase with set size. However, we find little indication that increasing the number of objects in a scene increased discrimination sensitivity. In Experiment 2, an equivalent noise analysis was used to measure participants' internal noise and the number of objects used to judge the average light source direction in a scene, finding that participants relied on 1 or 2 objects to make their judgement regardless of whether 9 or 25 objects were present. In Experiment 3, participants completed a shape identification task that required an implicit judgement of light source direction, rather than an explicit judgement as in Experiments 1 and 2. We find that sensitivity for identifying surface shape was comparable for scenes containing 1, 9, and 25 objects. Our results suggest that the visual system relied on a small number of objects to estimate the direction of lighting in our rendered scenes.

Consequences of polar form coherence for fMRI responses in human visual cortex.

Relevant features in the visual image are often spatially extensive and have complex orientation structure. Our perceptual sensitivity to such spatial form is demonstrated by polar Glass patterns, in which an array of randomly-positioned dot pairs that are each aligned with a particular polar displacement (rotation, for example) yield a salient impression of spatial structure. Such patterns are typically considered to be processed in two main stages: local spatial filtering in low-level visual cortex followed by spatial pooling and complex form selectivity in mid-level visual cortex. However, it remains unclear both whether reciprocal interactions within the cortical hierarchy are involved in polar Glass pattern processing and which mid-level areas identify and communicate polar Glass pattern structure. Here, we used functional magnetic resonance imaging (fMRI) at 7T to infer the magnitude of neural response within human low-level and mid-level visual cortex to polar Glass patterns of varying coherence (proportion of signal elements). The activity within low-level visual areas V1 and V2 was not significantly modulated by polar Glass pattern coherence, while the low-level area V3, dorsal and ventral mid-level areas, and the human MT complex each showed a positive linear coherence response functions. The cortical processing of polar Glass patterns thus appears to involve primarily feedforward communication of local signals from V1 and V2, with initial polar form selectivity reached in V3 and distributed to multiple pathways in mid-level visual cortex.

Segmentation decreases the magnitude of the tilt illusion.

In the tilt illusion, the perceived orientation of a target grating depends strongly on the orientation of a surround. When the orientations of the center and surround gratings differ by a small angle, the center grating appears to tilt away from the surround orientation (repulsion), whereas for a large difference in angle, the center appears to tilt toward the surround orientation (attraction). In order to understand how segmentation/perceptual grouping of the center and surround affect the magnitude of the tilt illusion, we conducted three psychophysical experiments in which we measured observers' perception of center orientation as a function of center-surround relative contrast, relative disparity depth, and geometric features such as occlusion and collinearity. All of these manipulations affected the strength of perceived orientation bias in the center. Our results suggest that if stronger segmentation/perceptual grouping is induced between the center and surround, the tilt repulsion bias decreases/increases. A grouping-dependent tilt illusion plays an important role in visual search and detection by enhancing the sensitivity of our visual system to feature discrepancies, especially in relatively homogenous environments.

General lighting can overcome accidental viewing.

When seeing an object in a scene, the presumption of seeing that object from a general viewpoint (as opposed to an accidental viewpoint) is a useful heuristic to decide which of many interpretations of this object is correct. Similar heuristic assumptions on illumination quality might also be used for scene interpretation. Here we tested that assumption and asked if illumination information helps determine object properties when seen from an accidental viewpoint. Test objects were placed on a flat surface and illumination was varied while keeping the objects' images constant. Observers judged the shape or rigidity of static or moving simple objects presented in accidental view. They also chose which of two seemingly very similar faces was familiar. We found: (1) Objects might appear flat without shadow information but were perceived to be volumetric objects or non-planar in the presence of cast shadows. (2) Apparently non-rigid objects became rigid with shadow information. (3) Shading and shadows helped to infer which of two face was the familiar one. Previous results had shown that cast shadows help determine spatial layout of objects. Our study shows that other properties of objects like rigidity or 3D-shape can be disambiguated by shadow information.

How haptic size sensations improve distance perception.

Determining distances to objects is one of the most ubiquitous perceptual tasks in everyday life. Nevertheless, it is challenging because the information from a single image confounds object size and distance. Though our brains frequently judge distances accurately, the underlying computations employed by the brain are not well understood. Our work illuminates these computions by formulating a family of probabilistic models that encompass a variety of distinct hypotheses about distance and size perception. We compare these models' predictions to a set of human distance judgments in an interception experiment and use Bayesian analysis tools to quantitatively select the best hypothesis on the basis of its explanatory power and robustness over experimental data. The central question is: whether, and how, human distance perception incorporates size cues to improve accuracy. Our conclusions are: 1) humans incorporate haptic object size sensations for distance perception, 2) the incorporation of haptic sensations is suboptimal given their reliability, 3) humans use environmentally accurate size and distance priors, 4) distance judgments are produced by perceptual "posterior sampling". In addition, we compared our model's estimated sensory and motor noise parameters with previously reported measurements in the perceptual literature and found good correspondence between them. Taken together, these results represent a major step forward in establishing the computational underpinnings of human distance perception and the role of size information.

Cardiac conduction disturbances in rheumatologic disease: a cross-sectional study.

BACKGROUND: Cardiovascular abnormalities are common in patients with rheumatologic disorders. Tachy-arrhythmias occur more frequently in these patients than the general population; however, the prevalence of bradyarrhythmias in this group is less clear. This investigation aimed to analyze the incidence and predictors of bradyarrhythmias and conduction disturbances, as well as the presence and influence of cardiologist management, in patients with rheumatologic disorders. METHODS: We performed a retrospective chart review of 57 consecutive patients with rheumatologic conditions evaluated at a tertiary-care safety-net hospital. Conduction disturbances were defined by any electrocardiogram (ECG) finding indicating: bradycardia, sinoatrial block of any degree, atrioventricular nodal block of any degree, left anterior or posterior fascicular block, non-specific intraventricular conduction delay, complete or incomplete right bundle branch block, left bundle branch block, or paced rhythm. Univariate and multivariate analyses were used to assess the association of relative predictors of conduction disturbance, the primary outcome of this investigation, as well as the secondary outcome of cardiologist involvement in patient care. Statistical significance was defined as P<0.05. Variables found to be statistically significant in a univariate analysis were included in a multivariable logistic regression analysis. RESULTS: The most common rheumatologic condition in our patient population was systemic lupus erythematous (21 patients, 36.8%) followed by gout (15 patients, 26.3%), rheumatoid arthritis (13 patients, 22.8%), sarcoidosis (6 patients, 10.5%), and two patients (3.5%) with other autoimmune diseases. A total of 31.6% of patients in this study were found to have conduction disturbances, higher than the prevalence of conduction disturbances in the general population. Multivariate logistic regression analysis showed significantly increased odds for conduction disturbances with increased age (odds ratio (OR): 1.05, 95% confidence interval (CI): 1.01-1.10, P<0.05). Similar analysis for the involvement of a cardiologist in the care of a patient with a rheumatologic disorder found increased odds for cardiologist involvement with advanced age (OR: 1.05, 95% CI: 1.002-1.09, P<0.05) and cardiovascular disease (OR: 5.0, 95% CI 1.24-21.90, P<0.05). CONCLUSION: Prevalence of conduction abnormalities is greater in rheumatologic patients than the general population. Odds for conduction abnormalities increased with age; and the odds of cardiologist involvement increased with age and cardiovascular disease.

Evaluating the Visibility of Architectural Features for People with Low Vision -A Quantitative Approach.

Most people with low vision rely on their remaining functional vision for mobility. Our goal is to provide tools to help design architectural spaces in which safe and effective mobility is possible by those with low vision---spaces that we refer to as visually accessible. We describe an approach that starts with a 3D CAD model of a planned space and produces labeled images indicating whether or not structures that are potential mobility hazards are visible at a particular level of low vision. There are two main parts to the analysis. The first, previously described, represents low-vision status by filtering a calibrated luminance image generated from the CAD model and associated lighting and materials information to produce a new image with unseen detail removed. The second part, described in this paper, uses both these filtered images and information about the geometry of the space obtained from the CAD model and related lighting and surface material specifications to produce a quantitative estimate of the likelihood of particular hazards being visible. We provide examples of the workflow required, a discussion of the novelty and implications of the approach, and a short discussion of needed future work.

A link between visual disambiguation and visual memory.

Sensory information in the retinal image is typically too ambiguous to support visual object recognition by itself. Theories of visual disambiguation posit that to disambiguate, and thus interpret, the incoming images, the visual system must integrate the sensory information with previous knowledge of the visual world. However, the underlying neural mechanisms remain unclear. Using functional magnetic resonance imaging (fMRI) of human subjects, we have found evidence for functional specialization for storing disambiguating information in memory versus interpreting incoming ambiguous images. Subjects viewed two-tone, "Mooney" images, which are typically ambiguous when seen for the first time but are quickly disambiguated after viewing the corresponding unambiguous color images. Activity in one set of regions, including a region in the medial parietal cortex previously reported to play a key role in Mooney image disambiguation, closely reflected memory for previously seen color images but not the subsequent disambiguation of Mooney images. A second set of regions, including the superior temporal sulcus, showed the opposite pattern, in that their responses closely reflected the subjects' percepts of the disambiguated Mooney images on a stimulus-to-stimulus basis but not the memory of the corresponding color images. Functional connectivity between the two sets of regions was stronger during those trials in which the disambiguated percept was stronger. This functional interaction between brain regions that specialize in storing disambiguating information in memory versus interpreting incoming ambiguous images may represent a general mechanism by which previous knowledge disambiguates visual sensory information.