Evidence of Serial Dependence from Decoding of Visual Evoked Potentials.

It is well known that recent sensory experience influences perception, recently demonstrated by a phenomenon termed "serial dependence." However, its underlying neural mechanisms are poorly understood. We measured ERP responses to pairs of stimuli presented randomly to the left or right hemifield. Seventeen male and female adults judged whether the upper or lower half of the grating had higher spatial frequency, independent of the horizontal position of the grating. This design allowed us to trace the memory signal modulating task performance and also the implicit memory signal associated with hemispheric position. Using classification techniques, we decoded the position of the current and previous stimuli and the response from voltage scalp distributions of the current trial. Classification of previous responses reached full significance only 700 ms after presentation of the current stimulus, consistent with retrieval of an activity-silent memory trace. Cross-condition classification accuracy of past responses (trained on current responses) correlated with the strength of serial dependence effects of individual participants. Overall, our data provide evidence for a silent memory signal that can be decoded from the EEG potential, which interacts with the neural processing of the current stimulus. This silent memory signal could be the physiological substrate subserving at least one type of serial dependence.SIGNIFICANCE STATEMENT The neurophysiological underpinnings of how past perceptual experience affects current perception are poorly understood. Here, we show that recent experience is reactivated when a new stimulus is presented and that the strength of this reactivation correlates with serial biases in individual participants, suggesting that serial dependence is established on the basis of a silent memory signal.

Serial dependence in orientation judgments at the time of saccades.

We actively seek information from the environment through saccadic eye movements, necessitating continual integration of presaccadic and postsaccadic signals, which are displaced on the retina by each saccade. We tested whether trans-saccadic integration may be related to serial dependence (a measure of how perceptual history influences current perception) by measuring how viewing a presaccadic stimulus affects the perceived orientation of a subsequent test stimulus presented around the time of a saccade. Participants reproduced the position, and orientation of a test stimulus presented around a 16° saccade. The reproduced position was mislocalized toward the saccadic target, agreeing with previous work. The reproduced orientation was attracted toward the prior stimulus and regressed to the mean orientation. These results suggest that both short- and long-term past information affects trans-saccadic perception, most strongly when the test stimulus is presented perisaccadically. This study unites the fields of serial dependence and trans-saccadic perception, leading to potential new insights of how information is transferred and accumulated across saccades.

Serial dependence improves performance and biases confidence-based decisions.

Perception depends on both the current sensory input and on the preceding stimuli history, a mechanism referred to as serial dependence (SD). One interesting, and somewhat controversial, question is whether serial dependence originates at the perceptual stage, which should lead to a sensory improvement, or at a subsequent decisional stage, causing solely a bias. Here, we studied the effects of SD in a novel manner by leveraging on the human capacity to spontaneously assess the quality of sensory information. Two noisy-oriented Gabor stimuli were simultaneously presented along with two bars of the same orientation as the Gabor stimuli. Participants were asked to choose which Gabor stimulus to judge and then make a forced-choice judgment of its orientation by selecting the appropriate response bar. On all trials, one of the Gabor stimuli had the same orientation as the Gabor in the same position on the previous trial. We explored whether continuity in orientation and position affected choice and accuracy. Results show that continuity of orientation leads to a persistent (up to four back) accuracy advantage and a higher preference in the selection of stimuli with the same orientation, and this advantage accumulates over trials. In contrast, analysis of the continuity of the selected position indicated that participants had a strong tendency to choose stimuli in the same position, but this behavior did not lead to an improvement in accuracy.

Propagation and update of auditory perceptual priors through alpha and theta rhythms.

To maintain a continuous and coherent percept over time, the brain makes use of past sensory information to anticipate forthcoming stimuli. We recently showed that auditory experience of the immediate past is propagated through ear-specific reverberations, manifested as rhythmic fluctuations of decision bias at alpha frequencies. Here, we apply the same time-resolved behavioural method to investigate how perceptual performance changes over time under conditions of stimulus expectation and to examine the effect of unexpected events on behaviour. As in our previous study, participants were required to discriminate the ear-of-origin of a brief monaural pure tone embedded in uncorrelated dichotic white noise. We manipulated stimulus expectation by increasing the target probability in one ear to 80%. Consistent with our earlier findings, performance did not remain constant across trials, but varied rhythmically with delay from noise onset. Specifically, decision bias showed a similar oscillation at ~9 Hz, which depended on ear congruency between successive targets. This suggests rhythmic communication of auditory perceptual history occurs early and is not readily influenced by top-down expectations. In addition, we report a novel observation specific to infrequent, unexpected stimuli that gave rise to oscillations in accuracy at ~7.6 Hz one trial after the target occurred in the non-anticipated ear. This new behavioural oscillation may reflect a mechanism for updating the sensory representation once a prediction error has been detected.

Groupitizing modifies neural coding of numerosity.

Numerical estimation of arrays of objects is faster and more accurate when items can be clustered into groups, a phenomenon termed "groupitizing." Grouping can facilitate segregation into subitizable "chunks," each easily estimated, then summed. The current study investigates whether spatial grouping of arrays drives specific neural responses during numerical estimation, reflecting strategies such as exact calculation and fact retrieval. Fourteen adults were scanned with fMRI while estimating either the numerosity or shape of arrays of items, either randomly distributed or spatially grouped. Numerosity estimation of both classes of stimuli elicited common activation of a right lateralized frontoparietal network. Grouped stimuli additionally recruited regions in the left hemisphere and bilaterally in the angular gyrus. Multivariate pattern analysis showed that classifiers trained with the pattern of neural activations read out from parietal regions, but not from the primary visual areas, can decode different numerosities both within and across spatial arrangements. The behavioral numerical acuity correlated with the decoding performance of the parietal but not with occipital regions. Overall, this experiment suggests that the estimation of grouped stimuli relies on the approximate number system for numerosity estimation, but additionally recruits regions involved in calculation.

Uncertainty and Prior Assumptions, Rather Than Innate Logarithmic Encoding, Explain Nonlinear Number-to-Space Mapping.

Mapping number to space is natural and spontaneous but often nonveridical, showing a clear compressive nonlinearity that is thought to reflect intrinsic logarithmic encoding of numerical values. We asked 78 adult participants to map dot arrays onto a number line across nine trials. Combining participant data, we confirmed that on the first trial, mapping was heavily compressed along the number line, but it became more linear across trials. Responses were well described by logarithmic compression but also by a parameter-free Bayesian model of central tendency, which quantitatively predicted the relationship between nonlinearity and number acuity. To experimentally test the Bayesian hypothesis, we asked 90 new participants to complete a color-line task in which they mapped noise-perturbed color patches to a "color line." When there was more noise at the high end of the color line, the mapping was logarithmic, but it became exponential with noise at the low end. We conclude that the nonlinearity of both number and color mapping reflects contextual Bayesian inference processes rather than intrinsic logarithmic encoding.

Visual priming and serial dependence are mediated by separate mechanisms.

Perceptual history influences current perception, readily revealed by visual priming (the facilitation of responses on repeated presentations of similar stimuli) and by serial dependence (systematic biases toward the previous stimuli). We asked whether the two phenomena shared perceptual mechanisms. We modified the standard "priming of pop-out" paradigm to measure both priming and serial dependence concurrently. The stimulus comprised three grating patches, one or two red, and the other green. Participants identified the color singleton (either red or green), and reproduced its orientation. Trial sequences were designed to maximize serial dependence, and long runs of priming color and position. The results showed strong effects of priming, both on reaction times and accuracy, which accumulated steadily over time, as generally reported in the literature. The serial dependence effects were also strong, but did not depend on previous color, nor on the run length. Reaction times measured under various conditions of repetition or change of priming color or position were reliably correlated with imprecision in orientation reproduction, but reliably uncorrelated with magnitude of serial dependence. The results suggest that visual priming and serial dependence are mediated by different neural mechanisms. We propose that priming affects sensitivity, possibly via attention-like mechanisms, whereas serial dependence affects criteria, two orthogonal dimensions in the signal detection theory.

Numbers in action.

To understand the number sense, we need to understand its function. We argue that numerosity estimation is fundamental not only for perception, but also preparation and control of action. We outline experiments that link numerosity estimation with action, pointing to a generalized numerosity system that serves both perception and action preparation.

Adaptation to hand-tapping affects sensory processing of numerosity directly: evidence from reaction times and confidence.

Like most perceptual attributes, the perception of numerosity is susceptible to adaptation, both to prolonged viewing of spatial arrays and to repeated motor actions such as hand-tapping. However, the possibility has been raised that adaptation may reflect response biases rather than modification of sensory processing. To disentangle these two possibilities, we studied visual and motor adaptation of numerosity perception while measuring confidence and reaction times. Both sensory and motor adaptation robustly distorted numerosity estimates, and these shifts in perceived numerosity were accompanied by similar shifts in confidence and reaction-time distributions. After adaptation, maximum uncertainty and slowest response-times occurred at the point of subjective (rather than physical) equality of the matching task, suggesting that adaptation acts directly on the sensory representation of numerosity, before the decisional processes. On the other hand, making reward response-contingent, which also caused robust shifts in the psychometric function, caused no significant shifts in confidence or reaction-time distributions. These results reinforce evidence for shared mechanisms that encode the quantity of both internally and externally generated events, and advance a useful general technique to test whether contextual effects like adaptation and serial dependence really affect sensory processing.

Pupillometry correlates of visual priming, and their dependency on autistic traits.

In paradigms of visual search where the search feature (say color) can change from trial to trials, responses are faster for trials where the search color is repeated than when it changes. This is a clear example of "priming" of attention. Here we test whether the priming effects can be revealed by pupillometry, and also whether they are related to autistic-like personality traits, as measured by the Autism-Spectrum Quotient (AQ). We repeated Maljkovic and Nakayama's (1994) classic priming experiment, asking subjects to identify rapidly the shape of a singleton target defined by color. As expected, reaction times were faster when target color repeated, and the effect accumulated over several trials; but the magnitude of the effect did not correlate with AQ. Reaction times were also faster when target position was repeated, again independent of AQ. Presentation of stimuli caused the pupil to dilate, and the magnitude of dilation was greater for switched than repeated trials. This effect did not accumulate over trials, and did not correlate with the reaction times difference, suggesting that the two indexes measure independent aspects of the priming phenomenon. Importantly, the amplitude of pupil modulation correlated negatively with AQ, and was significant only for those participants with low AQ. The results confirm that pupillometry can track perceptual and attentional processes, and furnish useful information unobtainable from standard psychophysics, including interesting dependencies on personality traits.

Simultaneous and sequential subitizing are separate systems, and neither predicts math abilities.

Small quantities of visual objects can be rapidly estimated without error, a phenomenon known as subitizing. Larger quantities can also be rapidly estimated, but with error, and the error rate predicts math abilities. This study addressed two issues: (a) whether subitizing generalizes over modalities and stimulus formats and (b) whether subitizing correlates with math abilities. We measured subitizing limits in primary school children and adults for visual and auditory stimuli presented either sequentially (sequences of flashes or sounds) or simultaneously (visual presentations, dot arrays). The results show that (a) subitizing limits for adults were one item larger than those for primary school children across all conditions; (b) subitizing for simultaneous visual stimuli (dots) was better than that for sequential stimuli; (c) subitizing limits for dots do not correlate with subitizing limits for either flashes or sounds; (d) subitizing of sequences of flashes and subitizing of sequences of sounds are strongly correlated with each other in children; and (e) regardless of stimuli sensory modality and format, subitizing limits do not correlate with mental calculation or digit magnitude knowledge proficiency. These results suggest that although children can subitize sequential numerosity, simultaneous and temporal subitizing may be subserved by separate systems. Furthermore, subitizing does not seem to be related to numerical abilities.

The functional role of serial dependence.

The world tends to be stable from moment to moment, leading to strong serial correlations in natural scenes. As similar stimuli usually require similar behavioural responses, it is highly likely that the brain has developed strategies to leverage these regularities. A good deal of recent psychophysical evidence is beginning to show that the brain is sensitive to serial correlations, causing strong drifts in observer responses towards previously seen stimuli. However, it is still not clear that this tendency leads to a functional advantage. Here, we test a formal model of optimal serial dependence and show that as predicted, serial dependence in an orientation reproduction task is dependent on current stimulus reliability, with less precise stimuli, such as low spatial frequency oblique Gabors, exhibiting the strongest effects. We also show that serial dependence depends on the similarity between two successive stimuli, again consistent with the behaviour of an ideal observer aiming at minimizing reproduction errors. Lastly, we show that serial dependence leads to faster response times, indicating that the benefits of serial integration go beyond reproduction error. Overall our data show that serial dependence has a beneficial role at various levels of perception, consistent with the idea that the brain exploits the temporal redundancy of the visual scene as an optimization strategy.

Children with autism spectrum disorder show reduced adaptation to number.

Autism is known to be associated with major perceptual atypicalities. We have recently proposed a general model to account for these atypicalities in Bayesian terms, suggesting that autistic individuals underuse predictive information or priors. We tested this idea by measuring adaptation to numerosity stimuli in children diagnosed with autism spectrum disorder (ASD). After exposure to large numbers of items, stimuli with fewer items appear to be less numerous (and vice versa). We found that children with ASD adapted much less to numerosity than typically developing children, although their precision for numerosity discrimination was similar to that of the typical group. This result reinforces recent findings showing reduced adaptation to facial identity in ASD and goes on to show that reduced adaptation is not unique to faces (social stimuli with special significance in autism), but occurs more generally, for both parietal and temporal functions, probably reflecting inefficiencies in the adaptive interpretation of sensory signals. These results provide strong support for the Bayesian theories of autism.

Inhibitory surrounds of motion mechanisms revealed by continuous tracking.

Continuous psychophysics is a newly developed technique that allows rapid estimation of visual thresholds by asking subjects to track a moving object, then deriving the integration window underlying tracking behavior (Bonnen, Burge, Yates, Pillow, & Cormack, 2015). Leveraging the continuous flow of stimuli and responses, continuous psychophysics allows for estimation of psychophysical thresholds in as little as 1 min. To date this technique has been applied only to tracking visual objects, where it has been used to measure localization thresholds. Here we adapt the technique to visual motion discrimination, by displaying a drifting grating that changes direction on a binary random walk and asking participants to continuously report drift direction by alternate key press. This technique replicates and confirms well-known findings of the motion-perception system. It also proves particularly valuable in demonstrating induced motion, reinforcing evidence for the existence of antagonistic surround fields. At low contrasts, the surround summates with the center, rather than opposing it, again consistent with existing evidence on classical techniques. The user-friendliness and efficiency of the method may lend it to clinical and developmental work.

Serial effects are optimal.

In the target article, Rahnev & Denison (R&D) use serial effects as an example of suboptimality. We show here that serial effects can be beneficial to perception, serving to reduce both error and response times in a near-optimal fashion. Furthermore, serial effects for stable attributes are positive, whereas those for changeable attributes are negative, demonstrating that they are engaged flexibly to optimize performance.

Spatiotopic coding during dynamic head tilt.

Humans maintain a stable representation of the visual world effortlessly, despite constant movements of the eyes, head, and body, across multiple planes. Whereas visual stability in the face of saccadic eye movements has been intensely researched, fewer studies have investigated retinal image transformations induced by head movements, especially in the frontal plane. Unlike head rotations in the horizontal and sagittal planes, tilting the head in the frontal plane is only partially counteracted by torsional eye movements and consequently induces a distortion of the retinal image to which we seem to be completely oblivious. One possible mechanism aiding perceptual stability is an active reconstruction of a spatiotopic map of the visual world, anchored in allocentric coordinates. To explore this possibility, we measured the positional motion aftereffect (PMAE; the apparent change in position after adaptation to motion) with head tilts of ∼42° between adaptation and test (to dissociate retinal from allocentric coordinates). The aftereffect was shown to have both a retinotopic and spatiotopic component. When tested with unpatterned Gaussian blobs rather than sinusoidal grating stimuli, the retinotopic component was greatly reduced, whereas the spatiotopic component remained. The results suggest that perceptual stability may be maintained at least partially through mechanisms involving spatiotopic coding.NEW & NOTEWORTHY Given that spatiotopic coding could play a key role in maintaining visual stability, we look for evidence of spatiotopic coding after retinal image transformations caused by head tilt. To this end, we measure the strength of the positional motion aftereffect (PMAE; previously shown to be largely spatiotopic after saccades) after large head tilts. We find that, as with eye movements, the spatial selectivity of the PMAE has a large spatiotopic component after head rotation.

Compressive mapping of number to space reflects dynamic encoding mechanisms, not static logarithmic transform.

The mapping of number onto space is fundamental to measurement and mathematics. However, the mapping of young children, unschooled adults, and adults under attentional load shows strong compressive nonlinearities, thought to reflect intrinsic logarithmic encoding mechanisms, which are later "linearized" by education. Here we advance and test an alternative explanation: that the nonlinearity results from adaptive mechanisms incorporating the statistics of recent stimuli. This theory predicts that the response to the current trial should depend on the magnitude of the previous trial, whereas a static logarithmic nonlinearity predicts trialwise independence. We found a strong and highly significant relationship between numberline mapping of the current trial and the magnitude of the previous trial, in both adults and school children, with the current response influenced by up to 15% of the previous trial value. The dependency is sufficient to account for the shape of the numberline, without requiring logarithmic transform. We show that this dynamic strategy results in a reduction of reproduction error, and hence improvement in accuracy.

Evidence for a number sense.

Numerosity is inherently confounded by related stimulus attributes such as density and area, and many studies have reported interactions of various strengths between area, density, and numerosity. However, direct measurements of sensitivity within the area-density-numerosity space show that numerosity emerges as the most spontaneous and sensitive dimension, strongly supporting the existence of a dedicated number sense.

Impairment of auditory spatial localization in congenitally blind human subjects.

Several studies have demonstrated enhanced auditory processing in the blind, suggesting that they compensate their visual impairment in part with greater sensitivity of the other senses. However, several physiological studies show that early visual deprivation can impact negatively on auditory spatial localization. Here we report for the first time severely impaired auditory localization in the congenitally blind: thresholds for spatially bisecting three consecutive, spatially-distributed sound sources were seriously compromised, on average 4.2-fold typical thresholds, and half performing at random. In agreement with previous studies, these subjects showed no deficits on simpler auditory spatial tasks or with auditory temporal bisection, suggesting that the encoding of Euclidean auditory relationships is specifically compromised in the congenitally blind. It points to the importance of visual experience in the construction and calibration of auditory spatial maps, with implications for rehabilitation strategies for the congenitally blind.

Mechanisms for perception of numerosity or texture-density are governed by crowding-like effects.

We have recently provided evidence that the perception of number and texture density is mediated by two independent mechanisms: numerosity mechanisms at relatively low numbers, obeying Weber's law, and texture-density mechanisms at higher numerosities, following a square root law. In this study we investigated whether the switch between the two mechanisms depends on the capacity to segregate individual dots, and therefore follows similar laws to those governing visual crowding. We measured numerosity discrimination for a wide range of numerosities at three eccentricities. We found that the point where the numerosity regime (Weber's law) gave way to the density regime (square root law) depended on eccentricity. In central vision, the regime changed at 2.3 dots/°2, while at 15° eccentricity, it changed at 0.5 dots/°2, three times less dense. As a consequence, thresholds for low numerosities increased with eccentricity, while at higher numerosities thresholds remained constant. We further showed that like crowding, the regime change was independent of dot size, depending on distance between dot centers, not distance between dot edges or ink coverage. Performance was not affected by stimulus contrast or blur, indicating that the transition does not depend on low-level stimulus properties. Our results reinforce the notion that numerosity and texture are mediated by two distinct processes, depending on whether the individual elements are perceptually segregable. Which mechanism is engaged follows laws that determine crowding.