Binocular rivalry under naturalistic geometry: Evidence from worlds simulated in virtual reality.

Binocular rivalry is a fascinating, widely studied visual phenomenon in which perception alternates between two competing images. This experience, however, is generally restricted to laboratory settings where two irreconcilable images are presented separately to the two eyes, an implausible geometry where two objects occupy the same physical location. Such laboratory experiences are in stark contrast to everyday visual behavior, where rivalry is almost never encountered, casting doubt on whether rivalry is relevant to our understanding of everyday binocular vision. To investigate the external validity of binocular rivalry, we manipulated the geometric plausibility of rival images using a naturalistic, cue-rich, 3D-corridor model created in virtual reality. Rival stimuli were presented in geometrically implausible, semi-plausible, or plausible layouts. Participants tracked rivalry fluctuations in each of these three layouts and for both static and moving rival stimuli. Results revealed significant and canonical binocular rivalry alternations regardless of geometrical plausibility and stimulus type. Rivalry occurred for layouts that mirrored the unnatural geometry used in laboratory studies and for layouts that mimicked real-world occlusion geometry. In a complementary 3D modeling analysis, we show that interocular conflict caused by geometrically plausible occlusion is a common outcome in a visual scene containing multiple objects. Together, our findings demonstrate that binocular rivalry can reliably occur for both geometrically implausible interocular conflicts and conflicts caused by a common form of naturalistic occlusion. Thus, key features of binocular rivalry are not simply laboratory artifacts but generalize to conditions that match the geometry of everyday binocular vision.

The effect of visual rivalry in peripheral head-mounted displays on mobility.

Recent head-mounted displays and smart glasses use vision multiplexing, an optical approach where two or more views are superimposed on each other. In vision multiplexing, augmented information is presented over an observer's natural field of view, providing field expansion and critical information during mobility situations like walking and driving. Yet despite its utility, vision multiplexing may produce visual rivalry, a phenomenon where perception alternates between the augmented information and the background scene for seconds at a time. To investigate, we compared the effect of different peripheral vision multiplexing configurations (unilateral opaque, unilateral see-through and bilateral see-through) on the detection of augmented information, incorporating at the same time real-world characteristics (target eccentricity, depth condition, and gaze movement) for a more realistic assessment. Results showed a persistently lower target detection rate in unilateral configurations than the bilateral configuration, suggesting a larger effect of binocular rivalry on target visibility. Nevertheless, this effect does become attenuated when more naturalistic elements are incorporated, and we discuss recommendations for vision multiplexing design and possible avenues for further research.

Binocular see-through configuration and eye movement attenuate visual rivalry in peripheral wearable displays.

Visual confusion occurs when two dissimilar images are superimposed onto the same retinal location. In the context of wearable displays, it can be used to provide multiple sources of information to users on top of the real-world scene. While useful, visual confusion may cause visual rivalry that can suppress one of the sources. If two different images are projected to each eye (i.e., monocular displays), it provokes binocular rivalry wherein visual perception intermittently switches between the two images. When a semi-transparent image is superimposed (i.e., see-through displays), monocular rivalry results, causing perceptual alternations between the foreground and the background images. Here, we investigated how these rivalries influence the visibility of the peripheral target using three configurations of wearable displays (i.e., monocular opaque, monocular see-through, and binocular see-through) with three eye movement conditions (i.e., saccades, smooth pursuit, and central fixation). Using the HTC VIVE Eye Pro headset, subjects viewed a forward vection of a 3D corridor with a horizontally moving vertical grating at 10° above the center fixation. During each trial (~1 min), subjects followed a fixation cross that varied in location to induce eye movements and simultaneously reported whether the peripheral target was visible. Results showed that the binocular display had significantly higher target visibility than both monocular displays, and the monocular see-through display had the lowest target visibility. Target visibility was also higher when eye movements were executed, suggesting that the effects of rivalry are attenuated by eye movements and binocular see-through displays.

CFS-crafter: An open-source tool for creating and analyzing images for continuous flash suppression experiments.

Continuous flash suppression (CFS) is a popular masking technique used to manipulate visual awareness. By presenting a rapidly changing stimulus to one eye (the 'mask'), a static image viewed by the other (the 'target') may remain invisible for many seconds. This effectiveness affords a means to assess unconscious visual processing, leading to the widespread use of CFS in several basic and clinical sciences. However, the lack of principled stimulus selection has impeded generalization of conclusions across studies, as the strength of interocular suppression is dependent on the spatiotemporal properties of the CFS mask and target. To address this, we created CFS-crafter, a point-and-click, open-source tool for creating carefully controlled CFS stimuli. The CFS-crafter provides a streamlined workflow to create, modify, and analyze mask and target stimuli, requiring only a rudimentary understanding of image processing that is well supported by help files in the application. Users can create CFS masks ranging from classic Mondrian patterns to those comprising objects or faces, or they can create, upload, and analyze their own images. Mask and target images can be custom-designed using image-processing operations performed in the frequency domain, including phase-scrambling and spatial/temporal/orientation filtering. By providing the means for the customization and analysis of CFS stimuli, the CFS-crafter offers controlled creation, analysis, and cross-study comparison. Thus, the CFS-crafter-with its easy-to-use image processing functionality-should facilitate the creation of visual conditions that allow a principled assessment of hypotheses about visual processing outside of awareness.

The development of audio-visual temporal precision precedes its rapid recalibration.

Through development, multisensory systems reach a balance between stability and flexibility: the systems integrate optimally cross-modal signals from the same events, while remaining adaptive to environmental changes. Is continuous intersensory recalibration required to shape optimal integration mechanisms, or does multisensory integration develop prior to recalibration? Here, we examined the development of multisensory integration and rapid recalibration in the temporal domain by re-analyzing published datasets for audio-visual, audio-tactile, and visual-tactile combinations. Results showed that children reach an adult level of precision in audio-visual simultaneity perception and show the first sign of rapid recalibration at 9 years of age. In contrast, there was very weak rapid recalibration for other cross-modal combinations at all ages, even when adult levels of temporal precision had developed. Thus, the development of audio-visual rapid recalibration appears to require the maturation of temporal precision. It may serve to accommodate distance-dependent travel time differences between light and sound.

Dynamic face mask enhances continuous flash suppression.

In continuous flash suppression (CFS), an image presented to one eye is suppressed from awareness by a dynamic image masker presented to the other eye. Previous studies report that face stimuli break out of CFS more readily when they are oriented upright and contain ecologically relevant information such as facial expressions or direct eye gaze, potentially implicating face processing in the mechanisms of interocular competition. It is unknown, however, whether face content helps to drive interocular suppression when incorporated into the dynamic masker itself, either by engaging higher-level visual mechanisms that underlie face detection or due to lower-level image features that the faces happen to contain. To investigate this, we devised a dynamic mask composed of upright faces and tested how well it suppressed detection of face or grating targets presented to the other eye. Relative contributions of higher-level and lower-level features were compared by manipulating the image properties of the mask. Results show that the dynamic face mask is strikingly effective at suppressing sensory input presented to the opposing eye, but its effectiveness is largely attributable to image texture, which can be quantified in terms of image entropy and edge density. This is because strong suppression was still observed following phase-scrambling or spatial inversion of the face elements, and while a target-selective effect was observed for the face mask, inverting the face elements to interfere with configural processing did not significantly diminish this effect. Thus, visual properties of faces, such as their image entropy and complex phase structure, predominate in driving interocular suppression rather than face detection per se.

Brief localised monocular deprivation in adults alters binocular rivalry predominance retinotopically and reduces spatial inhibition.

Short-term deprivation (2.5 h) of an eye has been shown to boost its relative ocular dominance in young adults. Here, we show that a much shorter deprivation period (3-6 min) produces a similar paradoxical boost that is retinotopic and reduces spatial inhibition on neighbouring, non-deprived areas. Partial deprivation was conducted in the left hemifield, central vision or in an annular region, later assessed with a binocular rivalry tracking procedure. Post-deprivation, dominance of the deprived eye increased when rivalling images were within the deprived retinotopic region, but not within neighbouring, non-deprived areas where dominance was dependent on the correspondence between the orientation content of the stimuli presented in the deprived and that of the stimuli presented in non-deprived areas. Together, these results accord with other deprivation studies showing V1 activity changes and reduced GABAergic inhibition.

Continuous flash suppression operates in local spatial zones: Effects of mask size and contrast.

Continuous flash suppression (CFS) is a technique in which presenting one eye with a dynamic Mondrian sequence prevents a low-contrast target in the other eye from being perceived for many seconds. Frequently used to study unconscious visual processing, CFS bears many similarities with binocular rivalry (BR), another popular dichoptic stimulation technique. It is therefore puzzling that the effect of mask size and contrast seem to differ between CFS and BR. To resolve this discrepancy, we conducted a systematic investigation on the effects of mask size and contrast in CFS. Also, building on findings from BR, we asked if the collinearity of the contours in the Mondrian masker play a role in CFS suppression. Our results showed a robust effect of mask contrast on suppression durations, and an effect of mask size that depended on collinearity. Specifically, higher mask contrasts produced longer suppression regardless of collinearity and mask size. Mask size, on the other hand, had little effect on suppression when collinearity was low and it weakened suppression when collinearity is high. These observations parallel prior findings in BR, further substantiating the close link between the two paradigms and demonstrating the usefulness of a shared explanatory framework describing both phenomena.

Battle of the Mondrians: Investigating the Role of Unpredictability in Continuous Flash Suppression.

In continuous flash suppression (CFS), a dynamic sequence of Mondrian patterns presented to one eye suppresses a static target in the other eye for several seconds at a time. Its effectiveness has been linked to low-level properties such as spatial frequency and orientation, but the role of higher order influences remains unstudied. Here, using a tracking paradigm, we asked if the spatial and temporal predictability of the Mondrian sequence affects CFS dynamics. Predictable temporal sequences were regularly updated every 100 ms or modulated sinusoidally in pixel luminance at 2 Hz. Unpredictable temporal sequences were irregularly updated or had stochastic pixel luminance changes across time. To vary spatial predictability, sequences were either updated with different Mondrian patterns or had a fixed spatial pattern. We found a modest effect of spatial uncertainty when the target modulation was low (0.125 Hz) but not temporal uncertainty, which had no significant effects regardless of target modulation. Similar results were obtained when we pitted the standard Mondrian sequence against sequences with a fixed spatial pattern and temporally low-pass filtered sequences in a binocular rivalry paradigm. Thus, not only was the effect of information predictability was modest and spatial, but it was also dependent on the presence of higher temporal frequencies. Together, the results demonstrate the significance of low-level properties in affecting CFS dynamics and the possible involvement of pattern structure masking in CFS.

Word recognition: re-thinking prosthetic vision evaluation.

OBJECTIVE: Evaluations of vision prostheses and sensory substitution devices have frequently relied on repeated training and then testing with the same small set of items. These multiple forced-choice tasks produced above chance performance in blind users, but it is unclear if the observed performance represents restoration of vision that transfers to novel, untrained items. APPROACH: Here, we tested the generalizability of the forced-choice paradigm on discrimination of low-resolution word images. Extensive visual training was conducted with the same 10 words used in previous BrainPort tongue stimulation studies. The performance on these 10 words and an additional 50 words was measured before and after the training sessions. MAIN RESULTS: The results revealed minimal performance improvement with the untrained words, demonstrating instead pattern discrimination limited mostly to the trained words. SIGNIFICANCE: These findings highlight the need to reconsider current evaluation practices, in particular, the use of forced-choice paradigms with a few highly trained items. While appropriate for measuring the performance thresholds in acuity or contrast sensitivity of a functioning visual system, performance on such tasks cannot be taken to indicate restored spatial pattern vision.

Strength of continuous flash suppression is optimal when target and masker modulation rates are matched.

Continuous flash suppression (CFS) is a popular technique whereby a dynamic sequence of Mondrian patterns is presented to one eye in order to suppress a static target presented to the other eye. Although the effectiveness of CFS is generally assumed to increase with the flicker rate of the Mondrian masker, a recent study has shown that suppression is optimal at very low masker rates for sustained targets, but higher rates may be necessary for transient targets. Here we vary the modulation rates of the masker and target using temporally filtered dynamic noise, which allowed us to examine the relationship between target and masker frequency and its effect on suppression strength. Using these carefully controlled, temporally narrowband stimuli, we demonstrate a pattern of results showing that suppression is greatest when target and masker modulate at similar frequencies. This finding indicates the involvement of early temporal-frequency-tuned filters underlying CFS and is consistent with many existing findings in the CFS literature. We also find that these temporally selective processes are orientation selective, which points to an early cortical substrate such as neurons in primary visual cortex. Overall, our study reveals that CFS suppression can be maximized by carefully matching the masker and target in temporal frequency and orientation. More generally, we show the importance of using carefully controlled stimuli for elucidating the underlying mechanisms of CFS. This approach is important at a theoretical level, as it will enable comparison of CFS with existing models of binocular rivalry and interocular suppression and facilitate a unified explanatory framework.

Slow and steady, not fast and furious: Slow temporal modulation strengthens continuous flash suppression.

Continuous flash suppression (CFS) involves the presentation of a rapidly changing Mondrian sequence to one eye and a static target in the other eye. Targets presented in this manner remain suppressed for several seconds at a time, and this has seen the prevalent use of CFS in studies of unconscious visual processes. However, the mechanisms behind CFS remain unclear, complicating its use and the comprehension of results obtained with the paradigm. For example, some studies report observations indicative of faster, visual masking processes whereas others suggest slower, rivalry processes. To reconcile this discrepancy, this study investigates the effect of temporal frequency content and Mondrian pattern structure on CFS suppression. Our results show predominant influences of spatial edges and low temporal-frequency content, which are similar to binocular rivalry, affording a parsimonious alternative in unifying the two paradigms.

A Matched Comparison Across Three Different Sensory Pairs of Cross-Modal Temporal Recalibration From Sustained and Transient Adaptation.

Sustained exposure to an asynchronous multisensory signal causes perceived simultaneity to shift in the direction of the leading component of the adapting stimulus. This is known as temporal recalibration, and recent evidence suggests that it can occur very rapidly, even after a single asynchronous audiovisual (AV) stimulus. However, this form of rapid recalibration appears to be unique to AV stimuli, in contrast to recalibration following sustained asynchronies which occurs with audiotactile (AT) and visuotactile (VT) stimuli. This study examines temporal recalibration to AV, VT and AT asynchrony with spatially collocated stimuli using a design that produces both sustained and inter-trial recalibration by combining the traditional sustained adaptation approach with an inter-trial analysis of sequential dependencies in an extended test period. Thus, we compare temporal recalibration to both sustained and transient asynchrony in three crossmodal combinations using the same design, stimuli and observers. The results reveal that prolonged exposure to asynchrony produced equivalent temporal recalibration for all combinations: AV, AT and VT. The pattern for rapid, inter-trial recalibration was very different. Rapid recalibration occurred strongly for AV stimuli, weakly for AT and did not occur at all for VT. For all sensory pairings, recalibration from sustained asynchrony decayed to baseline during the test phase while inter-trial recalibration was present and stable throughout testing, suggesting different mechanisms may underlie adaptation at long and short timescales.

The temporal frequency tuning of continuous flash suppression reveals peak suppression at very low frequencies.

Continuous flash suppression (CFS) is a psychophysical technique where a rapidly changing Mondrian pattern viewed by one eye suppresses the target in the other eye for several seconds. Despite the widespread use of CFS to study unconscious visual processes, the temporal tuning of CFS suppression is currently unknown. In the present study we used spatiotemporally filtered dynamic noise as masking stimuli to probe the temporal characteristics of CFS. Surprisingly, we find that suppression in CFS peaks very prominently at approximately 1 Hz, well below the rates typically used in CFS studies (10 Hz or more). As well as a strong bias to low temporal frequencies, CFS suppression is greater for high spatial frequencies and increases with increasing masker contrast, indicating involvement of parvocellular/ventral mechanisms in the suppression process. These results are reminiscent of binocular rivalry, and unifies two phenomenon previously thought to require different explanations.

Are auditory percepts determined by experience?

Audition--what listeners hear--is generally studied in terms of the physical properties of sound stimuli and physiological properties of the auditory system. Based on recent work in vision, we here consider an alternative perspective that sensory percepts are based on past experience. In this framework, basic auditory qualities (e.g., loudness and pitch) are based on the frequency of occurrence of stimulus patterns in natural acoustic stimuli. To explore this concept of audition, we examined five well-documented psychophysical functions. The frequency of occurrence of acoustic patterns in a database of natural sound stimuli (speech) predicts some qualitative aspects of these functions, but with substantial quantitative discrepancies. This approach may offer a rationale for auditory phenomena that are difficult to explain in terms of the physical attributes of the stimuli as such.

Expression of emotion in Eastern and Western music mirrors vocalization.

In Western music, the major mode is typically used to convey excited, happy, bright or martial emotions, whereas the minor mode typically conveys subdued, sad or dark emotions. Recent studies indicate that the differences between these modes parallel differences between the prosodic and spectral characteristics of voiced speech sounds uttered in corresponding emotional states. Here we ask whether tonality and emotion are similarly linked in an Eastern musical tradition. The results show that the tonal relationships used to express positive/excited and negative/subdued emotions in classical South Indian music are much the same as those used in Western music. Moreover, tonal variations in the prosody of English and Tamil speech uttered in different emotional states are parallel to the tonal trends in music. These results are consistent with the hypothesis that the association between musical tonality and emotion is based on universal vocal characteristics of different affective states.

Co-variation of tonality in the music and speech of different cultures.

Whereas the use of discrete pitch intervals is characteristic of most musical traditions, the size of the intervals and the way in which they are used is culturally specific. Here we examine the hypothesis that these differences arise because of a link between the tonal characteristics of a culture's music and its speech. We tested this idea by comparing pitch intervals in the traditional music of three tone language cultures (Chinese, Thai and Vietnamese) and three non-tone language cultures (American, French and German) with pitch intervals between voiced speech segments. Changes in pitch direction occur more frequently and pitch intervals are larger in the music of tone compared to non-tone language cultures. More frequent changes in pitch direction and larger pitch intervals are also apparent in the speech of tone compared to non-tone language cultures. These observations suggest that the different tonal preferences apparent in music across cultures are closely related to the differences in the tonal characteristics of voiced speech.