Fixation-related visual mismatch negativity.

Vision under natural conditions could be studied by combining electroencephalogram (EEG) and eye tracking as well as using saccades as triggers for the onset of the fixation-related potentials (FRPs) and for the oculomotor inhibition (OMI) that follows every saccade. The result of this analysis is thought to be equivalent to the event-related response following a peripheral preview. Previous studies that measured responses to visual deviants in a sequence of flashed stimuli found an increased negativity in the occipital N1 component (visual mismatch negativity [vMMN]), and prolonged saccadic inhibition for unexpected events. The aim of the current study was to develop an oddball paradigm in constrained natural-viewing and determine whether a similar mismatched FRP and prolonged OMI for deviance could be found. To this end, we developed a visual oddball paradigm on a static display to generate expectancy and surprise across successive saccades. Observers (n = 26) inspected, one after the other, seven small patterns of E and an inverted E arranged on the screen along a horizontal path, with one frequent (standard) and one rare (deviant), looking for a superimposed tiny dot target in each 5-second trial. Our results show a significantly larger FRP-N1 negativity for the deviant, compared with the standard and prolonged OMI of the following saccade, as previously found for transient oddballs. Our results show, for the first time, prolonged OMI and stronger fixation-related N1 to a task-irrelevant visual mismatch (vMMN) in natural, but task-guided viewing. These two signals combined could serve as markers of prediction error in free viewing.

Involuntary oculomotor inhibition markers of saliency and deviance in response to auditory sequences.

Our eyes move constantly but are often inhibited momentarily in response to external stimuli (oculomotor inhibition [OMI]), depending on the stimulus saliency, anticipation, and attention. Previous studies have shown prolonged OMI for auditory oddballs; however, they required counting the oddballs, possibly reflecting voluntary attention. Here, we investigated whether the "passive" OMI response to auditory deviants can provide a quantitative measure of deviance strength (pitch difference) and studied its dependence on the inter-trial interval (ITI). Participants fixated centrally and passively listened to repeated short sequences of pure tones that contained a deviant tone either regularly or with 20% probability (oddballs). In an "active" control experiment, participants counted the deviant or the standard. As in previous studies, the results showed prolonged microsaccade inhibition and increased pupil dilation following the rare deviant tone. Earlier inhibition onset was found in proportion to the pitch deviance (the saliency effect), and a later release was found for oddballs, but only for ITI <2.5 seconds. The active control experiment showed similar results when counting the deviant but longer OMI for the standard when counting it. Taken together, these results suggest that OMI provides involuntary markers of saliency and deviance, which can be obtained without the participant's response.

Oculomotor inhibition markers of working memory load.

Involuntary eye movements occur constantly even during fixation and were shown to convey information about cognitive processes. They are inhibited momentarily in response to external stimuli (oculomotor inhibition, OMI), with a time and magnitude that depend on stimulus saliency, attention, and expectations. It was recently shown that the working memory load for numbers modulates the microsaccade rate; however, the generality of the effect and its temporal properties remain unclear. Our goal was to investigate the relationship between OMI and the working memory load for simple colored shapes. Participants (N = 26) maintained their fixation while their eyes were tracked; they viewed briefly flashed colored shapes accompanied by small arrows indicating the shapes to be memorized (1/2/3). After a retention period, a probe shape appeared for matching. The microsaccade rate modulation and temporal properties were analyzed for the memory encoding, maintenance, and retrieval phases. Microsaccade inhibition was stronger when more shapes were memorized, and performance improved when microsaccades were suppressed during maintenance and retrieval. This occurred even though the physical stimuli were identical in number under all conditions. Thus, oculomotor inhibition may play a role in silencing the visual input while processing current stimuli and is generally related to processing time and load.

Face familiarity revealed by fixational eye movements and fixation-related potentials in free viewing.

Event-related potentials (ERPs) and the oculomotor inhibition (OMI) in response to visual transients are known to be sensitive to stimulus properties, attention, and expectation. We have recently found that the OMI is also sensitive to face familiarity. In natural vision, stimulation of the visual cortex is generated primarily by saccades, and it has been recently suggested that fixation-related potentials (FRPs) share similar components with the ERPs. Here, we investigated whether FRPs and microsaccade inhibition (OMI) in free viewing are sensitive to face familiarity. Observers freely watched a slideshow of seven unfamiliar and one familiar facial images presented randomly for 4-s periods, with multiple images per identity. We measured the occipital fixation-related N1 relative to the P1 magnitude as well as the associated fixation-triggered OMI. We found that the average N1-P1 was significantly smaller and the OMI was shorter for the familiar face, compared with any of the seven unfamiliar faces. Moreover, the P1 was suppressed across saccades for the familiar but not for the unfamiliar faces. Our results highlight the sensitivity of the occipital FRPs to stimulus properties such as face familiarity and advance our understanding of the integration process across successive saccades in natural vision.

Fixation-related saccadic inhibition in free viewing in response to stimulus saliency.

Microsaccades that occur during fixation were studied extensively in response to transient stimuli, showing a typical inhibition (Oculomotor Inhibition, OMI), and a later release with a latency that depends on stimulus saliency, attention, and expectations. Here, we investigated the hypothesis that in free viewing every saccade provides a new transient stimulation that should result in a stimulus-dependent OMI like a flashed presentation during fixation. Participants (N = 16) freely inspected static displays of randomly oriented Gabor texture images, with varied contrast and spatial frequency (SF) for periods of 10 s each. Eye tracking recordings were divided into epochs triggered by saccade landing (> 1 dva), and microsaccade latency relative to fixation onset was computed (msRT). We found that the msRT in free viewing was shorter for more salient stimuli (higher contrast or lower SF), as previously found for flashed stimuli. It increased with saccade size and decreased across successive saccades, but only for higher contrast, suggesting contrast-dependent repetition enhancement in free viewing. Our results indicate that visual stimulus-dependent inhibition of microsaccades also applies to free viewing. These findings are in agreement with the similarity found between event-related and fixation-related potentials and open the way for studies combining both approaches to study natural vision.

What Makes an Image Interesting and How Can We Explain It.

Here, we explore the question: What makes a photograph interesting? Answering this question deepens our understanding of human visual cognition and knowledge gained can be leveraged to reliably and widely disseminate information. Observers viewed images belonging to different categories, which covered a wide, representative spectrum of real-world scenes, in a self-paced manner and, at trial's end, rated each image's interestingness. Our studies revealed the following: landscapes were the most interesting of all categories tested, followed by scenes with people and cityscapes, followed still by aerial scenes, with indoor scenes of homes and offices being least interesting. Judgments of relative interestingness of pairs of images, setting a fixed viewing duration, or changing viewing history - all of the above manipulations failed to alter the hierarchy of image category interestingness, indicating that interestingness is an intrinsic property of an image unaffected by external manipulation or agent. Contrary to popular belief, low-level accounts based on computational image complexity, color, or viewing time failed to explain image interestingness: more interesting images were not viewed for longer and were not more complex or colorful. On the other hand, a single higher-order variable, namely image uprightness, significantly improved models of average interest. Observers' eye movements partially predicted overall average interest: a regression model with number of fixations, mean fixation duration, and a custom measure of novel fixations explained >40% of variance. Our research revealed a clear category-based hierarchy of image interestingness, which appears to be a different dimension altogether from memorability or awe and is as yet unexplained by the dual appraisal hypothesis.