PTVR - A software in Python to make virtual reality experiments easier to build and more reproducible.

Researchers increasingly use virtual reality (VR) to perform behavioral experiments, especially in vision science. These experiments are usually programmed directly in so-called game engines that are extremely powerful. However, this process is tricky and time-consuming as it requires solid knowledge of game engines. Consequently, the anticipated prohibitive effort discourages many researchers who want to engage in VR. This paper introduces the Perception Toolbox for Virtual Reality (PTVR) library, allowing visual perception studies in VR to be created using high-level Python script programming. A crucial consequence of using a script is that an experiment can be described by a single, easy-to-read piece of code, thus improving VR studies' transparency, reproducibility, and reusability. We built our library upon a seminal open-source library released in 2018 that we have considerably developed since then. This paper aims to provide a comprehensive overview of the PTVR software for the first time. We introduce the main objects and features of PTVR and some general concepts related to the three-dimensional (3D) world. This new library should dramatically reduce the difficulty of programming experiments in VR and elicit a whole new set of visual perception studies with high ecological validity.

From Psychoacoustics to Brain Waves: A Longitudinal Approach to Novel Word Learning.

Musical expertise has been shown to positively influence high-level speech abilities such as novel word learning. This study addresses the question whether low-level enhanced perceptual skills causally drives successful novel word learning. We used a longitudinal approach with psychoacoustic procedures to train 2 groups of nonmusicians either on pitch discrimination or on intensity discrimination, using harmonic complex sounds. After short (approximately 3 hr) psychoacoustic training, discrimination thresholds were lower on the specific feature (pitch or intensity) that was trained. Moreover, compared to the intensity group, participants trained on pitch were faster to categorize words varying in pitch. Finally, although the N400 components in both the word learning phase and in the semantic task were larger in the pitch group than in the intensity group, no between-group differences were found at the behavioral level in the semantic task. Thus, these results provide mixed evidence that enhanced perception of relevant features through a few hours of acoustic training with harmonic sounds causally impacts the categorization of speech sounds as well as novel word learning. These results are discussed within the framework of near and far transfer effects from music training to speech processing.

Neural correlates of intra-saccadic motion perception.

Retinal motion of the visual scene is not consciously perceived during ocular saccades in normal everyday conditions. It has been suggested that extra-retinal signals actively suppress intra-saccadic motion perception to preserve stable perception of the visual world. However, using stimuli optimized to preferentially activate the M-pathway, Castet and Masson (2000) demonstrated that motion can be perceived during a saccade. Based on this psychophysical paradigm, we used electroencephalography and eye-tracking recordings to investigate the neural correlates related to the conscious perception of intra-saccadic motion. We demonstrated the effective involvement during saccades of the cortical areas V1-V2 and MT-V5, which convey motion information along the M-pathway. We also showed that individual motion perception was related to retinal temporal frequency.

A Vision Enhancement System to Improve Face Recognition with Central Vision Loss.

SIGNIFICANCE: The overall goal of this work is to validate a low vision aid system that uses gaze as a pointing tool and provides smart magnification. We conclude that smart visual enhancement techniques as well as gaze contingency should improve the efficiency of assistive technology for the visually impaired. PURPOSE: A low vision aid, using gaze-contingent visual enhancement and primarily intended to help reading with central vision loss, was recently designed and tested with simulated scotoma. Here, we present a validation of this system for face recognition in age-related macular degeneration patients. METHODS: Twelve individuals with binocular central vision loss were recruited and tested on a face identification-matching task. Gaze position was measured in real time, thanks to an eye tracker. In the visual enhancement condition, at any time during the screen exploration, the fixated face was segregated from background and considered as a region of interest that could be magnified into a region of augmented vision by the participant, if desired. In the natural exploration condition, participants also performed the matching task but without the visual aid. Response time and accuracy were analyzed with mixed-effects models to (1) compare the performance with and without visual aid and (2) estimate the usability of the system. RESULTS: On average, the percentage of correct response for the natural exploration condition was 41%. This value was significantly increased to 63% with visual enhancement (95% confidence interval, 45 to 78%). For the large majority of our participants (83%), this improvement was accompanied by moderate increase in response time, suggesting a real functional benefit for these individuals. CONCLUSIONS: Without visual enhancement, participants with age-related macular degeneration performed poorly, confirming their struggle for face recognition and the need to use efficient visual aids. Our system significantly improved face identification accuracy by 55%, proving to be helpful under laboratory conditions.

The magnification factor accounts for the greater hypometria and imprecision of larger saccades: Evidence from a parametric human-behavioral study.

Saccades quite systematically undershoot a peripheral visual target by about 10% of its eccentricity while becoming more variable, mainly in amplitude, as the target becomes more peripheral. This undershoot phenomenon has been interpreted as the strategic adjustment of saccadic gain downstream of the superior colliculus (SC), where saccades are programmed. Here, we investigated whether the eccentricity-related increase in saccades' hypometria and imprecision might not instead result from overrepresentation of space closer to the fovea in the SC and visual-cortical areas. To test this magnification-factor (MF) hypothesis, we analyzed four parametric eye-movement data sets, collected while humans made saccades to single eccentric stimuli. We first established that the undershoot phenomenon generalizes to ordinary saccade amplitudes (0.5°-15°) and directions (0°-90°) and that landing-position distributions become not only increasingly elongated but also more skewed toward the fovea as target eccentricity increases. Moreover, we confirmed the MF hypothesis by showing (a) that the linear eccentricity-related increase in undershoot error and negative skewness canceled out when landing positions were log-scaled according to the MF in monkeys' SC and (b) that the spread, proportional to eccentricity outside an extended, 5°, foveal region, became circular and invariant in size in SC space. Yet the eccentricity-related increase in variability, slower near the fovea, yielded progressively larger and more elongated clusters toward foveal and vertical-meridian SC representations. What causes this latter, unexpected, pattern remains undetermined. Nevertheless, our findings clearly suggest that the undershoot phenomenon, and related variability, originate in, or upstream of, the SC, rather than reflecting downstream, adaptive, strategies.

Letter and symbol identification: No evidence for letter-specific crowding mechanisms.

It has been proposed that letters, as opposed to symbols, trigger specialized crowding processes, boosting identification of the first and last letters of words. This hypothesis is based on evidence that single-letter accuracy as a function of within-string position has a W shape (the classic serial position function [SPF] in psycholinguistics) whereas an inverted V shape is obtained when measured with symbols. Our main goal was to test the robustness of the latter result. Our hypothesis was that any letter/symbol difference might result from short-term visual memory processes (due to the partial report [PR] procedures used in SPF studies) rather than from crowding. We therefore removed the involvement of short-term memory by precueing target-item position and compared SPFs with precueing and postcueing. Perimetric complexity was stringently matched between letters and symbols. In postcueing conditions similar to previous studies, we did not reproduce the inverted V shape for symbols: Clear-cut W shapes were observed with an overall smaller accuracy for symbols compared to letters. This letter/symbol difference was dramatically reduced in precueing conditions in keeping with our prediction. Our results are not consistent with the claim that letter strings trigger specialized crowding processes. We argue that PR procedures are not fit to isolate crowding processes.

A New Vessel-Based Method to Estimate Automatically the Position of the Nonfunctional Fovea on Altered Retinography From Maculopathies.

Purpose: The purpose of this study was to validate a new automated method to locate the fovea on normal and pathological fundus images. Compared to the normative anatomic measures (NAMs), our vessel-based fovea localization (VBFL) approach relies on the retina's vessel structure to make predictions. Methods: The spatial relationship between the fovea location and vessel characteristics is learnt from healthy fundus images and then used to predict fovea location in new images. We evaluate the VBFL method on three categories of fundus images: healthy images acquired with different head orientations and fixation locations, healthy images with simulated macular lesions, and pathological images from age-related macular degeneration (AMD). Results: For healthy images taken with the head tilted to the side, the NAM estimation error is significantly multiplied by 4, whereas VBFL yields no significant increase, representing a 73% reduction in prediction error. With simulated lesions, VBFL performance decreases significantly as lesion size increases and remains better than NAM until lesion size reaches 200 degrees2. For pathological images, average prediction error was 2.8 degrees, with 64% of the images yielding an error of 2.5 degrees or less. VBFL was not robust for images showing darker regions and/or incomplete representation of the optic disk. Conclusions: The vascular structure provides enough information to precisely locate the fovea in fundus images in a way that is robust to head tilt, eccentric fixation location, missing vessels, and actual macular lesions. Translational Relevance: The VBFL method should allow researchers and clinicians to assess automatically the eccentricity of a newly developed area of fixation in fundus images with macular lesions.

Similar effect of cueing conditions on attentional and saccadic temporal dynamics.

Spatial attention permits to allocate more processing resources to a restricted portion of the visual space. The influential premotor theory states that the allocation of spatial attention relies on the same processes as those responsible for programming saccadic eye movements. Accordingly, several studies have already demonstrated a close spatial correspondence between attention and saccades. However, the question of the temporal coupling between attention displacements and saccades remains unclear. To address this issue, we compared the saccadic latencies to the temporal dynamics of attentional shifts under various cueing conditions known to affect attentional timing. In Experiment 1, we evaluated the effect of peripheral and central cues, and in Experiment 2, we assessed the influence of peripheral cues of different salience levels. We found that the different cue types had a similar effect on saccade latencies and on the dynamics of attentional shifts, either overt or covert. Moreover, even within a given cueing condition, attentional shift times were strongly correlated with saccadic latencies. These results indicate that, in agreement with the premotor theory, both spatial attention and saccades rely on a common process and that the allocation of spatial attention is tightly time-locked to saccade execution.

Activation time-course of phonological code in silent word recognition in adult readers with and without dyslexia.

In skilled adult readers, reading words is generally assumed to rapidly and automatically activate the phonological code. In adults with dyslexia, despite the main consensus on their phonological processing deficits, little is known about the activation time course of this code. The present study investigated this issue in both populations. Participants' accuracy and eye movements were recorded while they performed a visual lexical decision task in which phonological consistency of written words was manipulated. Readers with dyslexia were affected by phonological consistency during second fixation duration of visual word recognition suggesting a late activation of the phonological code. Regarding skilled readers, no influence of phonological consistency was found when the participants were considered a homogeneous population. However, a different pattern emerged when they were divided into two subgroups according to their phonological and semantic abilities: Those who showed better decoding than semantic skills were affected by phonological consistency at the earliest stage of visual word recognition while those who showed better semantic than decoding skills were not affected by this factor at any processing stage. Overall, the findings suggest that the presence of phonological deficits in readers with dyslexia is associated with a delayed activation of phonological representations during reading. In skilled readers, the contribution of phonology varies with their reading profile, i.e., being phonologically or semantically oriented.

The inhibitory effect of word neighborhood size when reading with central field loss is modulated by word predictability and reading proficiency.

For normally sighted readers, word neighborhood size (i.e., the total number of words that can be formed from a single word by changing only one letter) has a facilitator effect on word recognition. When reading with central field loss (CFL) however, individual letters may not be correctly identified, leading to possible misidentifications and a reverse neighborhood size effect. Here we investigate this inhibitory effect of word neighborhood size on reading performance and whether it is modulated by word predictability and reading proficiency. Nineteen patients with binocular CFL from 32 to 89 years old (mean ± SD = 75 ± 15) read short sentences presented with the self-paced reading paradigm. Accuracy and reading time were measured for each target word read, along with its predictability, i.e., its probability of occurrence following the two preceding words in the sentence using a trigram analysis. Linear mixed effects models were then fit to estimate the individual contributions of word neighborhood size, predictability, frequency and length on accuracy and reading time, while taking patients' reading proficiency into account. For the less proficient readers, who have given up daily reading as a consequence of their visual impairment, we found that the effect of neighborhood size was reversed compared to normally sighted readers and of higher amplitude than the effect of frequency. Furthermore, this inhibitory effect is of greater amplitude (up to 50% decrease in reading speed) when a word is not easily predictable because its chances to occur after the two preceding words in a specific sentence are rather low. Severely impaired patients with CFL often quit reading on a daily basis because this task becomes simply too exhausting. Based on our results, we envision lexical text simplification as a new alternative to promote effective rehabilitation in these patients. By increasing reading accessibility for those who struggle the most, text simplification might be used as an efficient rehabilitation tool and daily reading assistive technology, fostering overall reading ability and fluency through increased practice.

The optimal use of non-optimal letter information in foveal and parafoveal word recognition.

Letters and words across the visual field can be difficult to identify due to limiting visual factors such as acuity, crowding and position uncertainty. Here, we show that when human readers identify words presented at foveal and para-foveal locations, they act like theoretical observers making optimal use of letter identity and letter position information independently extracted from each letter after an unavoidable and non-optimal letter recognition guess. The novelty of our approach is that we carefully considered foveal and parafoveal letter identity and position uncertainties by measuring crowded letter recognition performance in five subjects without any word context influence. Based on these behavioral measures, lexical access was simulated for each subject by an observer making optimal use of each subject's uncertainties. This free-parameter model was able to predict individual behavioral recognition rates of words presented at different positions across the visual field. Importantly, the model was also able to predict individual mislocation and identity letter errors made during behavioral word recognition. These results reinforce the view that human readers recognize foveal and parafoveal words by parts (the word letters) in a first stage, independently of word context. They also suggest a second step where letter identity and position uncertainties are generated based on letter first guesses and positions. During the third lexical access stage, identity and position uncertainties from each letter look remarkably combined together through an optimal word recognition decision process.

Linguistic processes do not beat visuo-motor constraints, but they modulate where the eyes move regardless of word boundaries: Evidence against top-down word-based eye-movement control during reading.

Where readers move their eyes, while proceeding forward along lines of text, has long been assumed to be determined in a top-down word-based manner. According to this classical view, readers of alphabetic languages would invariably program their saccades towards the center of peripheral target words, as selected based on the (expected) needs of ongoing (word-identification) processing, and the variability in within-word landing positions would exclusively result from systematic and random errors. Here we put this predominant hypothesis to a strong test by estimating the respective influences of language-related variables (word frequency and word predictability) and lower-level visuo-motor factors (word length and saccadic launch-site distance to the beginning of words) on both word-skipping likelihood and within-word landing positions. Our eye-movement data were collected while forty participants read 316 pairs of sentences, that differed only by one word, the prime; this was either semantically related or unrelated to a following test word of variable frequency and length. We found that low-level visuo-motor variables largely predominated in determining which word would be fixated next, and where in a word the eye would land. In comparison, language-related variables only had tiny influences. Yet, linguistic variables affected both the likelihood of word skipping and within-word initial landing positions, all depending on the words' length and how far on average the eye landed from the word boundaries, but pending the word could benefit from peripheral preview. These findings provide a strong case against the predominant word-based account of eye-movement guidance during reading, by showing that saccades are primarily driven by low-level visuo-motor processes, regardless of word boundaries, while being overall subject to subtle, one-off, language-based modulations. Our results also suggest that overall distributions of saccades' landing positions, instead of truncated within-word landing-site distributions, should be used for a better understanding of eye-movement guidance during reading.

The influence of word frequency on word reading speed when individuals with macular diseases read text.

People with central field loss (CFL) use peripheral vision to identify words. Eccentric vision provides ambiguous visual inputs to the processes leading to lexical access. Our purpose was to explore the hypothesis that this ambiguity leads to strong influences of inferential processes, our prediction being that increasing word frequency would decrease word reading time. Individuals with bilateral CFL induced by macular diseases read French sentences displayed with a self-paced reading method. Reading time of the last word of each sentence (target word) was recorded. Each target word (in sentence n) was matched with a synonym word (in sentence n+1) of the same length. When using absolute frequency value (Analysis 1), we found that reading time of target words decreased when word frequency increases, even when controlling for word length. The amplitude of this effect is larger than reported in previous investigations of reading with normal subjects. When comparing the effect of relative frequency (low vs. high) within each pair of synonyms (Analysis 2), results show the same pattern as the one observed in Analysis 1. Our results demonstrate clear-cut frequency effects on word reading time and suggest that inferential processes are stronger in CFL readers than in normally sighted observers. These results might also help design text simplification tools tailored for low-vision patients.

Page mode reading with simulated scotomas: oculo-motor patterns.

This study investigated the relationship between reading speed and oculo-motor parameters when normally sighted observers had to read single sentences with an artificial macular scotoma. Using multiple regression analysis, our main result shows that two significant predictors, number of saccades per sentence followed by average fixation duration, account for 94% of reading speed variance: reading speed decreases when number of saccades and fixation duration increase. The number of letters per forward saccade (L/FS), which was measured directly in contrast to previous studies, is not a significant predictor. The results suggest that, independently of the size of saccades, some or all portions of a sentence are temporally integrated across an increasing number of fixations as reading speed is reduced.

Dynamics of distributed 1D and 2D motion representations for short-latency ocular following.

Integrating information is essential to measure the physical 2D motion of a surface from both ambiguous local 1D motion of its elongated edges and non-ambiguous 2D motion of its features such as corners or texture elements. The dynamics of this motion integration shows a complex time course as read from tracking eye movements: first, local 1D motion signals are extracted and pooled to initiate ocular responses, then 2D motion signals are integrated to adjust the tracking direction until it matches the surface motion direction. The nature of these 1D and 2D motion computations are still unclear. One hypothesis is that their different dynamics may be explained from different contrast sensitivities. To test this, we measured contrast-response functions of early, 1D-driven and late, 2D-driven components of ocular following responses to different motion stimuli: gratings, plaids and barberpoles. We found that contrast dynamics of 1D-driven responses are nearly identical across the different stimuli. On the contrary, late 2D-driven components with either plaids or barberpoles have similar latencies but different contrast dynamics. Temporal dynamics of both 1D- and 2D-driven responses demonstrates that the different contrast gains are set very early during the response time course. Running a Bayesian model of motion integration, we show that a large family of contrast-response functions can be predicted from the probability distributions of 1D and 2D motion signals for each stimulus and by the shape of the prior distribution. However, the pure delay (i.e. largely independent upon contrast) observed between 1D- and 2D-motion supports the fact that 1D and 2D probability distributions are computed independently. This two-pathway Bayesian model supports the idea that 1D and 2D mechanisms represent edges and features motion in parallel.

Motion Masking by Stationary Objects: A Study of Simulated Saccades.

Saccades are crucial to visual information intake by re-orienting the fovea to regions of interest in the visual scene. However, they cause drastic disruptions of the retinal input by shifting the retinal image at very high speeds. The resulting motion and smear are barely noticed, a phenomenon known as saccadic omission. Here, we studied the perception of motion during simulated saccades while observers fixated, moving naturalistic visual scenes across the retina with saccadic speed profiles using a very high temporal frequency display. We found that the mere presence of static pre- and post-saccadic images significantly reduces the perceived amplitude of motion but does not eliminate it entirely. This masking of motion perception could make the intra-saccadic stimulus much less salient and thus easier to ignore.

Bayesian modeling of dynamic motion integration.

The quality of the representation of an object's motion is limited by the noise in the sensory input as well as by an intrinsic ambiguity due to the spatial limitation of the visual motion analyzers (aperture problem). Perceptual and oculomotor data demonstrate that motion processing of extended objects is initially dominated by the local 1D motion cues, related to the object's edges and orthogonal to them, whereas 2D information, related to terminators (or edge-endings), takes progressively over and leads to the final correct representation of global motion. A Bayesian framework accounting for the sensory noise and general expectancies for object velocities has proven successful in explaining several experimental findings concerning early motion processing [Weiss, Y., Adelson, E., 1998. Slow and smooth: a Bayesian theory for the combination of local motion signals in human vision. MIT Technical report, A.I. Memo 1624]. In particular, these models provide a qualitative account for the initial bias induced by the 1D motion cue. However, a complete functional model, encompassing the dynamical evolution of object motion perception, including the integration of different motion cues, is still lacking. Here we outline several experimental observations concerning human smooth pursuit of moving objects and more particularly the time course of its initiation phase, which reflects the ongoing motion integration process. In addition, we propose a recursive extension of the Bayesian model, motivated and constrained by our oculomotor data, to describe the dynamical integration of 1D and 2D motion information. We compare the model predictions for object motion tracking with human oculomotor recordings.

Page mode reading with simulated scotomas: a modest effect of interline spacing on reading speed.

Crowding is thought to be one potent limiting factor of reading in peripheral vision. While several studies investigated how crowding between horizontally adjacent letters or words can influence eccentric reading, little attention has been paid to the influence of vertically adjacent lines of text. The goal of this study was to examine the dependence of page mode reading performance (speed and accuracy) on interline spacing. A gaze-contingent visual display was used to simulate a visual central scotoma while normally sighted observers read meaningful French sentences following MNREAD principles. The sensitivity of this new material to low-level factors was confirmed by showing strong effects of perceptual learning, print size and scotoma size on reading performance. In contrast, reading speed was only slightly modulated by interline spacing even for the largest range tested: a 26% gain for a 178% increase in spacing. This modest effect sharply contrasts with the dramatic influence of vertical word spacing found in a recent RSVP study. This discrepancy suggests either that vertical crowding is minimized when reading meaningful sentences, or that the interaction between crowding and other factors such as attention and/or visuo-motor control is dependent on the paradigm used to assess reading speed (page vs. RSVP mode).

Spatiotemporal dynamics of visual attention during saccade preparation: Independence and coupling between attention and movement planning.

During the preparation of a saccadic eye movement, a visual stimulus is more efficiently processed when it is spatially coincident with the saccadic target as compared to when the visual and the saccadic targets are displayed at different locations. We studied the coupling between visual selective attention and saccadic preparation by measuring orientation acuity of human subjects at different locations relative to the saccadic target and at different delays relative to the saccade cue onset. First, we generalized previous results (E. Castet, S. Jeanjean, A. Montagnini, D. Laugier, & G. S. Masson, 2006) revealing that a dramatic perceptual advantage at the saccadic target emerges dynamically within the first 150-200 ms from saccade cue onset. Second, by varying the validity of the spatial cue for the discrimination task, we encouraged subjects to modulate the spatial distribution of attentional resources independently from the automatic deployment to saccadic target. We found that an independent component of attention can be voluntarily deployed away from the saccadic target. The relative weight of the automatic versus the independent component of attention increases across time during saccadic preparation.

Evaluation of a gaze-controlled vision enhancement system for reading in visually impaired people.

People with low vision, especially those with Central Field Loss (CFL), need magnification to read. The flexibility of Electronic Vision Enhancement Systems (EVES) offers several ways of magnifying text. Due to the restricted field of view of EVES, the need for magnification is conflicting with the need to navigate through text (panning). We have developed and implemented a real-time gaze-controlled system whose goal is to optimize the possibility of magnifying a portion of text while maintaining global viewing of the other portions of the text (condition 1). Two other conditions were implemented that mimicked commercially available advanced systems known as CCTV (closed-circuit television systems)-conditions 2 and 3. In these two conditions, magnification was uniformly applied to the whole text without any possibility to specifically select a region of interest. The three conditions were implemented on the same computer to remove differences that might have been induced by dissimilar equipment. A gaze-contingent artificial 10° scotoma (a mask continuously displayed in real time on the screen at the gaze location) was used in the three conditions in order to simulate macular degeneration. Ten healthy subjects with a gaze-contingent scotoma read aloud sentences from a French newspaper in nine experimental one-hour sessions. Reading speed was measured and constituted the main dependent variable to compare the three conditions. All subjects were able to use condition 1 and they found it slightly more comfortable to use than condition 2 (and similar to condition 3). Importantly, reading speed results did not show any significant difference between the three systems. In addition, learning curves were similar in the three conditions. This proof of concept study suggests that the principles underlying the gaze-controlled enhanced system might be further developed and fruitfully incorporated in different kinds of EVES for low vision reading.