How small changes to one eye's retinal image can transform the perceived shape of a very familiar object.

Vision can provide useful cues about the geometric properties of an object, like its size, distance, pose, and shape. But how the brain merges these properties into a complete sensory representation of a three-dimensional object is poorly understood. To address this gap, we investigated a visual illusion in which humans misperceive the shape of an object due to a small change in one eye's retinal image. We first show that this illusion affects percepts of a highly familiar object under completely natural viewing conditions. Specifically, people perceived their own rectangular mobile phone to have a trapezoidal shape. We then investigate the perceptual underpinnings of this illusion by asking people to report both the perceived shape and pose of controlled stimuli. Our results suggest that the shape illusion results from distorted cues to object pose. In addition to yielding insights into object perception, this work informs our understanding of how the brain combines information from multiple visual cues in natural settings. The shape illusion can occur when people wear everyday prescription spectacles; thus, these findings also provide insight into the cue combination challenges that some spectacle wearers experience on a regular basis.

Transformations of sensory information in the brain suggest changing criteria for optimality.

Neurons throughout the brain modulate their firing rate lawfully in response to sensory input. Theories of neural computation posit that these modulations reflect the outcome of a constrained optimization in which neurons aim to robustly and efficiently represent sensory information. Our understanding of how this optimization varies across different areas in the brain, however, is still in its infancy. Here, we show that neural sensory responses transform along the dorsal stream of the visual system in a manner consistent with a transition from optimizing for information preservation towards optimizing for perceptual discrimination. Focusing on the representation of binocular disparities-the slight differences in the retinal images of the two eyes-we re-analyze measurements characterizing neuronal tuning curves in brain areas V1, V2, and MT (middle temporal) in the macaque monkey. We compare these to measurements of the statistics of binocular disparity typically encountered during natural behaviors using a Fisher Information framework. The differences in tuning curve characteristics across areas are consistent with a shift in optimization goals: V1 and V2 population-level responses are more consistent with maximizing the information encoded about naturally occurring binocular disparities, while MT responses shift towards maximizing the ability to support disparity discrimination. We find that a change towards tuning curves preferring larger disparities is a key driver of this shift. These results provide new insight into previously-identified differences between disparity-selective areas of cortex and suggest these differences play an important role in supporting visually-guided behavior. Our findings emphasize the need to consider not just information preservation and neural resources, but also relevance to behavior, when assessing the optimality of neural codes.

A paradigm for characterizing motion misperception in people with typical vision and low vision.

PURPOSE: We aimed to develop a paradigm that can efficiently characterize motion percepts in people with low vision and compare their responses with well-known misperceptions made by people with typical vision when targets are hard to see. METHODS: We recruited a small cohort of individuals with reduced acuity and contrast sensitivity (n = 5) as well as a comparison cohort with typical vision (n = 5) to complete a psychophysical study. Study participants were asked to judge the motion direction of a tilted rhombus that was either high or low contrast. In a series of trials, the rhombus oscillated vertically, horizontally, or diagonally. Participants indicated the perceived motion direction using a number wheel with 12 possible directions, and statistical tests were used to examine response biases. RESULTS: All participants with typical vision showed systematic misperceptions well predicted by a Bayesian inference model. Specifically, their perception of vertical or horizontal motion was biased toward directions orthogonal to the long axis of the rhombus. They had larger biases for hard-to-see (low contrast) stimuli. Two participants with low vision had a similar bias, but with no difference between high- and low-contrast stimuli. The other participants with low vision were unbiased in their percepts or biased in the opposite direction. CONCLUSIONS: Our results suggest that some people with low vision may misperceive motion in a systematic way similar to people with typical vision. However, we observed large individual differences. Future work will aim to uncover reasons for such differences and identify aspects of vision that predict susceptibility.

Low Vision Impairs Implicit Sensorimotor Adaptation in Response to Small Errors, But Not Large Errors.

Successful goal-directed actions require constant fine-tuning of the motor system. This fine-tuning is thought to rely on an implicit adaptation process that is driven by sensory prediction errors (e.g., where you see your hand after reaching vs. where you expected it to be). Individuals with low vision experience challenges with visuomotor control, but whether low vision disrupts motor adaptation is unknown. To explore this question, we assessed individuals with low vision and matched controls with normal vision on a visuomotor task designed to isolate implicit adaptation. We found that low vision was associated with attenuated implicit adaptation only for small visual errors, but not for large visual errors. This result highlights important constraints underlying how low-fidelity visual information is processed by the sensorimotor system to enable successful implicit adaptation.

Using augmented reality to cue obstacles for people with low vision.

Detecting and avoiding obstacles while navigating can pose a challenge for people with low vision, but augmented reality (AR) has the potential to assist by enhancing obstacle visibility. Perceptual and user experience research is needed to understand how to craft effective AR visuals for this purpose. We developed a prototype AR application capable of displaying multiple kinds of visual cues for obstacles on an optical see-through head-mounted display. We assessed the usability of these cues via a study in which participants with low vision navigated an obstacle course. The results suggest that 3D world-locked AR cues were superior to directional heads-up cues for most participants during this activity.

The art of the float.

For more than 2000 years, artists have exploited cast shadows to influence how objects appear to be positioned in a scene. A contact cast shadow can anchor an object to the ground and a detached cast shadow can make an object appear to float. However, there is a period of approximately 1000 years when there were virtually no cast shadows in art. How were states of contact versus floating depicted by artists without cast shadows? Here, we survey various techniques used by artists to anchor relative position with and without cast shadows. We then conduct experimental tests of the hypothesized surface attraction principles that underlie these techniques. In the absence of cast shadows, an object (a wooden box) was often seen as resting on a surface as long as that surface offered information about ground orientation and support (a tiled floor). When the ground surface was ambiguous and cloud-like (1/f noise), the box was more likely to be seen to float. The presence of cast shadows made the box appear to contact the ground whether it was well-defined or ambiguous. Both shadows and surface support also increased the accuracy with which participants detected when the box was tilted up from the ground. These results indicate that artists long ago discovered the important power of support relationships to anchor objects to surfaces in the absence of shadows.

The contribution of image minification to discomfort experienced in wearable optics.

Wearable optics have a broad range of uses, for example, in refractive spectacles and augmented/virtual reality devices. Despite the long-standing and widespread use of wearable optics in vision care and technology, user discomfort remains an enduring mystery. Some of this discomfort is thought to derive from optical image minification and magnification. However, there is limited scientific data characterizing the full range of physical and perceptual symptoms caused by minification or magnification during daily life. In this study, we aimed to evaluate sensitivity to changes in retinal image size introduced by wearable optics. Forty participants wore 0%, 2%, and 4% radially symmetric optical minifying lenses binocularly (over both eyes) and monocularly (over just one eye). Physical and perceptual symptoms were measured during tasks that required head movement, visual search, and judgment of world motion. All lens pairs except the controls (0% binocular) were consistently associated with increased discomfort along some dimension. Greater minification tended to be associated with greater discomfort, and monocular minification was often-but not always-associated with greater symptoms than binocular minification. Furthermore, our results suggest that dizziness and visual motion were the most reported physical and perceptual symptoms during naturalistic tasks. This work establishes preliminary guidelines for tolerances to binocular and monocular image size distortion in wearable optics.

Longitudinal Trends in Case Histories and Rehabilitative Device Assessments at Low Vision Examinations.

SIGNIFICANCE: Understanding longitudinal changes in why individuals frequent low-vision clinics is crucial for ensuring that patient care keeps current with changing technology and changing lifestyles. Among other findings, our results suggest that reading remains a prevailing patient complaint, with shifting priorities toward technology-related topics. PURPOSE: This study aimed to understand changes in patient priorities and patient care in low vision over the past decade. METHODS: We conducted a retrospective study of examination records (2009 to 2019, 3470 examinations) from two U.S. low-vision clinics. Automated word searches summarized two properties of the records: topics discussed during the case history and types of rehabilitative devices assessed. Logistic regression was used to model the effects of examination year, patient age, patient sex, and level of visual impairment. RESULTS: Collapsing across all years, the most common topic discussed was reading (78%), followed by light-related topics (71%) and technology (59%). Whereas the odds of discussing reading trended downward over the decade (odds ratio, 0.57; P = .03), technology, social interaction, mobility, and driving trended upward (odds ratios, 4.53, 3.31, 2.71, and 1.95; all P 's < 0.001). The most frequently assessed devices were tinted lenses (95%). Over time, video magnifier and spectacle assessments trended downward (odds ratios, 0.64 and 0.72; P = .004, 0.04), whereas assessments of other optical aids increased. The data indicate several consistent differences among patient demographics. CONCLUSIONS: Reading is likely to remain a prevailing patient complaint, but an increase in technology-related topics suggests shifting priorities, particularly in younger demographics. "Low-tech" optical aids have remained prominent in low-vision care even as "high-tech" assistive devices in the marketplace continue to advance.

The effect of spatial structure on binocular contrast perception.

To obtain a single percept of the world, the visual system must combine inputs from the two eyes. Understanding the principles that govern this binocular combination process has important real-world clinical and technological applications. However, most research examining binocular combination has relied on relatively simple visual stimuli and it is unclear how well the findings apply to real-world scenarios. For example, it is well-known that, when the two eyes view sine wave gratings with differing contrast (dichoptic stimuli), the binocular percept often matches the higher contrast grating. Does this winner-take-all property of binocular contrast combination apply to more naturalistic imagery, which include broadband structure and spatially varying contrast? To better understand binocular combination during naturalistic viewing, we conducted psychophysical experiments characterizing binocular contrast perception for a range of visual stimuli. In two experiments, we measured the binocular contrast perception of dichoptic sine wave gratings and naturalistic stimuli, and asked how the contrast of the surrounding context affected percepts. Binocular contrast percepts were close to winner-take-all across many of the stimuli when the surrounding context was the average contrast of the two eyes. However, we found that changing the surrounding context modulated the binocular percept of some patterns and not others. We show evidence that this contextual effect may be due to the spatial orientation structure of the stimuli. These findings provide a step toward understanding binocular combination in the natural world and highlight the importance of considering the effect of the spatial interactions in complex stimuli.

Efficient sensory coding of multidimensional stimuli.

According to the efficient coding hypothesis, sensory systems are adapted to maximize their ability to encode information about the environment. Sensory neurons play a key role in encoding by selectively modulating their firing rate for a subset of all possible stimuli. This pattern of modulation is often summarized via a tuning curve. The optimally efficient distribution of tuning curves has been calculated in variety of ways for one-dimensional (1-D) stimuli. However, many sensory neurons encode multiple stimulus dimensions simultaneously. It remains unclear how applicable existing models of 1-D tuning curves are for neurons tuned across multiple dimensions. We describe a mathematical generalization that builds on prior work in 1-D to predict optimally efficient multidimensional tuning curves. Our results have implications for interpreting observed properties of neuronal populations. For example, our results suggest that not all tuning curve attributes (such as gain and bandwidth) are equally useful for evaluating the encoding efficiency of a population.

A perceptual eyebox for near-eye displays.

In near-eye display systems that support three-dimensional (3D) augmented and virtual reality, a central factor in determining the user experience is the size of the eyebox. The eyebox refers to a volume where the eye receives an acceptable view of the image with respect to a set of criteria and thresholds. The size and location of this volume are primarily driven by optical architecture choices in which designers trade-off a number of constraints, such as field of view, image quality, and product design. It is thus important to clearly quantify how design decisions affect the properties of the eyebox. Recent work has started evaluating the eyebox in 3D based purely on optical criteria. However, such analyses do not incorporate perceptual criteria that determine visual quality, which are particularly important for binocular 3D systems. To address this limitation, we introduce the framework of a perceptual eyebox. The perceptual eyebox is the volume where the eye(s) must be located for the user to experience a visual percept falling within a perceptually-defined criterion. We combine optical and perceptual data to characterize an example perceptual eyebox for display visibility in augmented reality. The key contributions in this paper include: comparing the perceptual eyebox for monocular and binocular display designs, modeling the effects of user eye separation, and examining the effects of eye rotation on the eyebox volume.

Optimizing virtual reality for all users through gaze-contingent and adaptive focus displays.

From the desktop to the laptop to the mobile device, personal computing platforms evolve over time. Moving forward, wearable computing is widely expected to be integral to consumer electronics and beyond. The primary interface between a wearable computer and a user is often a near-eye display. However, current generation near-eye displays suffer from multiple limitations: they are unable to provide fully natural visual cues and comfortable viewing experiences for all users. At their core, many of the issues with near-eye displays are caused by limitations in conventional optics. Current displays cannot reproduce the changes in focus that accompany natural vision, and they cannot support users with uncorrected refractive errors. With two prototype near-eye displays, we show how these issues can be overcome using display modes that adapt to the user via computational optics. By using focus-tunable lenses, mechanically actuated displays, and mobile gaze-tracking technology, these displays can be tailored to correct common refractive errors and provide natural focus cues by dynamically updating the system based on where a user looks in a virtual scene. Indeed, the opportunities afforded by recent advances in computational optics open up the possibility of creating a computing platform in which some users may experience better quality vision in the virtual world than in the real one.

Natural statistics of depth edges modulate perceptual stability.

Binocular fusion relies on matching points in the two eyes that correspond to the same physical feature in the world; however, not all world features are binocularly visible. Near depth edges, some regions of a scene are often visible to only one eye (so-called half occlusions). Accurate detection of these monocularly visible regions is likely to be important for stable visual perception. If monocular regions are not detected as such, the visual system may attempt to binocularly fuse non-corresponding points, which can result in unstable percepts. We investigated the hypothesis that the visual system capitalizes on statistical regularities associated with depth edges in natural scenes to aid binocular fusion and facilitate perceptual stability. By sampling from a large set of stereoscopic natural images with co-registered distance information, we found evidence that monocularly visible regions near depth edges primarily result from background occlusions. Accordingly, monocular regions tended to be more visually similar to the adjacent binocularly visible background region than to the adjacent binocularly visible foreground. Consistent with our hypothesis, perceptual experiments showed that perception tended to be more stable when the image properties of the depth edge were statistically more likely given the probability of occurrence in natural scenes (i.e., when monocular regions were more visually similar to the binocular background). The generality of these results was supported by a parametric study with simulated environments. Exploiting regularities in natural environments may allow the visual system to facilitate fusion and perceptual stability when both binocular and monocular regions are visible.

Using an Augmented Reality Device as a Distance-based Vision Aid-Promise and Limitations.

SIGNIFICANCE: For people with limited vision, wearable displays hold the potential to digitally enhance visual function. As these display technologies advance, it is important to understand their promise and limitations as vision aids. PURPOSE: The aim of this study was to test the potential of a consumer augmented reality (AR) device for improving the functional vision of people with near-complete vision loss. METHODS: An AR application that translates spatial information into high-contrast visual patterns was developed. Two experiments assessed the efficacy of the application to improve vision: an exploratory study with four visually impaired participants and a main controlled study with participants with simulated vision loss (n = 48). In both studies, performance was tested on a range of visual tasks (identifying the location, pose and gesture of a person, identifying objects, and moving around in an unfamiliar space). Participants' accuracy and confidence were compared on these tasks with and without augmented vision, as well as their subjective responses about ease of mobility. RESULTS: In the main study, the AR application was associated with substantially improved accuracy and confidence in object recognition (all P < .001) and to a lesser degree in gesture recognition (P < .05). There was no significant change in performance on identifying body poses or in subjective assessments of mobility, as compared with a control group. CONCLUSIONS: Consumer AR devices may soon be able to support applications that improve the functional vision of users for some tasks. In our study, both artificially impaired participants and participants with near-complete vision loss performed tasks that they could not do without the AR system. Current limitations in system performance and form factor, as well as the risk of overconfidence, will need to be overcome.

Systematic misperceptions of 3-D motion explained by Bayesian inference.

People make surprising but reliable perceptual errors. Here, we provide a unified explanation for systematic errors in the perception of three-dimensional (3-D) motion. To do so, we characterized the binocular retinal motion signals produced by objects moving through arbitrary locations in 3-D. Next, we developed a Bayesian model, treating 3-D motion perception as optimal inference given sensory noise in the measurement of retinal motion. The model predicts a set of systematic perceptual errors, which depend on stimulus distance, contrast, and eccentricity. We then used a virtual-reality headset as well as a standard 3-D desktop stereoscopic display to test these predictions in a series of perceptual experiments. As predicted, we found evidence that errors in 3-D motion perception depend on the contrast, viewing distance, and eccentricity of a stimulus. These errors include a lateral bias in perceived motion direction and a surprising tendency to misreport approaching motion as receding and vice versa. In sum, we present a Bayesian model that provides a parsimonious account for a range of systematic misperceptions of motion in naturalistic environments.

Predicting cortical dark/bright asymmetries from natural image statistics and early visual transforms.

The nervous system has evolved in an environment with structure and predictability. One of the ubiquitous principles of sensory systems is the creation of circuits that capitalize on this predictability. Previous work has identified predictable non-uniformities in the distributions of basic visual features in natural images that are relevant to the encoding tasks of the visual system. Here, we report that the well-established statistical distributions of visual features--such as visual contrast, spatial scale, and depth--differ between bright and dark image components. Following this analysis, we go on to trace how these differences in natural images translate into different patterns of cortical input that arise from the separate bright (ON) and dark (OFF) pathways originating in the retina. We use models of these early visual pathways to transform natural images into statistical patterns of cortical input. The models include the receptive fields and non-linear response properties of the magnocellular (M) and parvocellular (P) pathways, with their ON and OFF pathway divisions. The results indicate that there are regularities in visual cortical input beyond those that have previously been appreciated from the direct analysis of natural images. In particular, several dark/bright asymmetries provide a potential account for recently discovered asymmetries in how the brain processes visual features, such as violations of classic energy-type models. On the basis of our analysis, we expect that the dark/bright dichotomy in natural images plays a key role in the generation of both cortical and perceptual asymmetries.

Gender representation in the vision sciences: A longitudinal study.

Understanding the current status and historical trends of gender representation within a research field is an important component of fostering a diverse and inclusive scientific community. Here, we report on the gender representation of a large sample of the vision science research community--the attendees of the Annual Meeting of the Vision Sciences Society (VSS). Our analysis shows that the majority of scientists at all career levels in our sample are male. This imbalance is most pronounced for the senior scientists, whereas predoctoral students are nearly balanced between the genders. Historically, the gender imbalance was larger than it is at present, and it has followed a slow-but-steady trend toward gender parity over the past decade. A longitudinal analysis based on tracking individual attendees shows a larger dropout rate for female than male predoctoral trainees. However, among the trainees who continue in the vision science field after graduate school, evidence suggests that career advancement is quite similar between the genders. In an additional analysis, we found that the VSS Young Investigator awardees and the abstract review committee members reflect substantial gender imbalances, suggesting that these recognitions have yet to catch up with the greater gender balance of the rising generation of junior vision scientists. We hope that this report will encourage awareness of issues of diversity in the scientific community and further promote the development of a research field in which all talented scientists are supported to succeed.

Sensitivity and bias in the discrimination of two-dimensional and three-dimensional motion direction.

Sensory systems are faced with an essentially infinite number of possible environmental events but have limited processing resources. Posed with this challenge, it makes sense to allocate these resources to prioritize the discrimination of events with the most behavioral relevance. Here, we asked if such relevance is reflected in the processing and perception of motion. We compared human performance on a rapid motion direction discrimination task, including monocular and binocular viewing. In particular, we determined sensitivity and bias for a binocular motion-in-depth (three-dimensional; 3D) stimulus and for its constituent monocular (two-dimensional; 2D) signals over a broad range of speeds. Consistent with prior work, we found that binocular 3D sensitivity was lower than monocular sensitivity for all speeds. Although overall sensitivity was worse for 3D discrimination, we found that the transformation from 2D to 3D motion processing also incorporated a pattern of potentially advantageous biases. One such bias is reflected by a criterion shift that occurs at the level of 3D motion processing and results in an increased hit rate for motion toward the head. We also observed an increase in sensitivity for 3D motion trajectories presented on crossed rather than uncrossed disparity pedestals, privileging motion trajectories closer to the observer. We used these measurements to determine the range of real-world trajectories for which rapid 3D motion discrimination is most useful. These results suggest that the neural mechanisms that underlie motion perception privilege behaviorally relevant motion and provide insights into the nature of human motion sensitivity in the real world.

The natural statistics of blur.

Blur from defocus can be both useful and detrimental for visual perception: It can be useful as a source of depth information and detrimental because it degrades image quality. We examined these aspects of blur by measuring the natural statistics of defocus blur across the visual field. Participants wore an eye-and-scene tracker that measured gaze direction, pupil diameter, and scene distances as they performed everyday tasks. We found that blur magnitude increases with increasing eccentricity. There is a vertical gradient in the distances that generate defocus blur: Blur below the fovea is generally due to scene points nearer than fixation; blur above the fovea is mostly due to points farther than fixation. There is no systematic horizontal gradient. Large blurs are generally caused by points farther rather than nearer than fixation. Consistent with the statistics, participants in a perceptual experiment perceived vertical blur gradients as slanted top-back whereas horizontal gradients were perceived equally as left-back and right-back. The tendency for people to see sharp as near and blurred as far is also consistent with the observed statistics. We calculated how many observations will be perceived as unsharp and found that perceptible blur is rare. Finally, we found that eye shape in ground-dwelling animals conforms to that required to put likely distances in best focus.

Perceived depth in natural images reflects encoding of low-level luminance statistics.

Sighted animals must survive in an environment that is diverse yet highly structured. Neural-coding models predict that the visual system should allocate its computational resources to exploit regularities in the environment, and that this allocation should facilitate perceptual judgments. Here we use three approaches (natural scenes statistical analysis, a reanalysis of single-unit data from alert behaving macaque, and a behavioral experiment in humans) to address the question of how the visual system maximizes behavioral success by taking advantage of low-level regularities in the environment. An analysis of natural scene statistics reveals that the probability distributions for light increments and decrements are biased in a way that could be exploited by the visual system to estimate depth from relative luminance. A reanalysis of neurophysiology data from Samonds et al. (2012) shows that the previously reported joint tuning of V1 cells for relative luminance and binocular disparity is well matched to a predicted distribution of binocular disparities produced by natural scenes. Finally, we show that a percept of added depth can be elicited in images by exaggerating the correlation between luminance and depth. Together, the results from these three approaches provide further evidence that the visual system allocates its processing resources in a way that is driven by the statistics of the natural environment.