Enabling identification of component processes in perceptual learning with nonparametric hierarchical Bayesian modeling.

Perceptual learning is a multifaceted process, encompassing general learning, between-session forgetting or consolidation, and within-session fast relearning and deterioration. The learning curve constructed from threshold estimates in blocks or sessions, based on tens or hundreds of trials, may obscure component processes; high temporal resolution is necessary. We developed two nonparametric inference procedures: a Bayesian inference procedure (BIP) to estimate the posterior distribution of contrast threshold in each learning block for each learner independently and a hierarchical Bayesian model (HBM) that computes the joint posterior distribution of contrast threshold across all learning blocks at the population, subject, and test levels via the covariance of contrast thresholds across blocks. We applied the procedures to the data from two studies that investigated the interaction between feedback and training accuracy in Gabor orientation identification over 1920 trials across six sessions and estimated learning curve with block sizes L = 10, 20, 40, 80, 160, and 320 trials. The HBM generated significantly better fits to the data, smaller standard deviations, and more precise estimates, compared to the BIP across all block sizes. In addition, the HBM generated unbiased estimates, whereas the BIP only generated unbiased estimates with large block sizes but exhibited increased bias with small block sizes. With L = 10, 20, and 40, we were able to consistently identify general learning, between-session forgetting, and rapid relearning and adaptation within sessions. The nonparametric HBM provides a general framework for fine-grained assessment of the learning curve and enables identification of component processes in perceptual learning.

Estimating the Trial-by-Trial Learning Curve in Perceptual Learning with Hierarchical Bayesian Modeling.

The learning curve serves as a crucial metric for assessing human performance in perceptual learning. It may encompass various component processes, including general learning, between-session forgetting or consolidation, and within-session rapid relearning and adaptation or deterioration. Typically, empirical learning curves are constructed by aggregating tens or hundreds of trials of data in blocks or sessions. Here, we devised three inference procedures for estimating the trial-by-trial learning curve based on the multi-component functional form identified in Zhao et al. (submitted): general learning, between-session forgetting, and within-session rapid relearning and adaptation. These procedures include a Bayesian inference procedure (BIP) estimating the posterior distribution of parameters for each learner independently, and two hierarchical Bayesian models (HBMv and HBMc) computing the joint posterior distribution of parameters and hyperparameters at the population, subject, and test levels. The HBMv and HBMc incorporate variance and covariance hyperparameters, respectively, between and within subjects. We applied these procedures to data from two studies investigating the interaction between feedback and training accuracy in Gabor orientation identification across about 2000 trials spanning six sessions (Liu et al., 2010, 2012) and estimated the trial-by-trial learning curves at both the subject and population levels. The HBMc generated best fits to the data and the smallest half width of 68.2% credible interval of the learning curves compared to the BIP and HBMv. The parametric HBMc with the multi-component functional form provides a general framework for trial-by-trial analysis of the component processes in perceptual learning and for predicting the learning curve in unmeasured time points.

Hierarchical Bayesian perceptual template modeling of mechanisms of spatial attention in central and peripheral cuing.

The external noise paradigm and perceptual template model (PTM) have successfully been applied to characterize observer properties and mechanisms of observer state changes (e.g. attention and perceptual learning) in several research domains, focusing on individual level analysis. In this study, we developed a new hierarchical Bayesian perceptual template model (HBPTM) to model the trial-by-trial data from all individuals and conditions in a published spatial cuing study within a single structure and compared its performance to that of a Bayesian Inference Procedure (BIP), which separately infers the posterior distributions of the model parameters for each individual subject without the hierarchical structure. The HBPTM allowed us to compute the joint posterior distribution of the hyperparameters and parameters at the population, observer, and experiment levels and make statistical inferences at all these levels. In addition, we ran a large simulation study that varied the number of observers and number of trials in each condition and demonstrated the advantage of the HBPTM over the BIP across all the simulated datasets. Although it is developed in the context of spatial attention, the HBPTM and its extensions can be used to model data from the external noise paradigm in other domains and enable predictions of human performance at both the population and individual levels.

Current directions in visual perceptual learning.

The visual expertise of adult humans is jointly determined by evolution, visual development, and visual perceptual learning. Perceptual learning refers to performance improvements in perceptual tasks after practice or training in the task. It occurs in almost all visual tasks, ranging from simple feature detection to complex scene analysis. In this Review, we focus on key behavioral aspects of visual perceptual learning. We begin by describing visual perceptual learning tasks and manipulations that influence the magnitude of learning, and then discuss specificity of learning. Next, we present theories and computational models of learning and specificity. We then review applications of visual perceptual learning in visual rehabilitation. Finally, we summarize the general principles of visual perceptual learning, discuss the tension between plasticity and stability, and conclude with new research directions.

Visual attention in spatial cueing and visual search.

To characterize internal processes of an observer conducting perceptual tasks, we developed an observer model that combines the perceptual template model (PTM), the attention mechanisms in the PTM framework (Lu & Dosher, 1998), and uncertainty of signal detection theory (Green & Swets, 1966). The model was evaluated with a visual search experiment conducted in a range of external noise, signal contrast, and target-distractor similarity conditions. In each trial, eight Gabor patches were shown in each of two brief intervals, with one target at a different orientation from the distractors in one of the presentations. Subjects were precued to a subset of the stimuli (1, 2, 4, or 8) and asked to report (a) which interval contained the target and (b) where the target was. Individual roles of uncertainty and of attention in visual search were investigated by comparing models with and without an attention component. The results showed that decision uncertainty alone was sufficient to account for the set-size effect, even in conditions with high target-distractor similarity. Our theoretical model and empirical results provide a coherent picture regarding how visual information is selected and processed during feature search.

Roving: The causes of interference and re-enabled learning in multi-task visual training.

People routinely perform multiple visual judgments in the real world, yet, intermixing tasks or task variants during training can damage or even prevent learning. This paper explores why. We challenged theories of visual perceptual learning focused on plastic retuning of low-level retinotopic cortical representations by placing different task variants in different retinal locations, and tested theories of perceptual learning through reweighting (changes in readout) by varying task similarity. Discriminating different (but equivalent) and similar orientations in separate retinal locations interfered with learning, whereas training either with identical orientations or sufficiently different ones in different locations released rapid learning. This location crosstalk during learning renders it unlikely that the primary substrate of learning is retuning in early retinotopic visual areas; instead, learning likely involves reweighting from location-independent representations to a decision. We developed an Integrated Reweighting Theory (IRT), which has both V1-like location-specific representations and higher level (V4/IT or higher) location-invariant representations, and learns via reweighting the readout to decision, to predict the order of learning rates in different conditions. This model with suitable parameters successfully fit the behavioral data, as well as some microstructure of learning performance in a new trial-by-trial analysis.

Evaluating the performance of the staircase and quick Change Detection methods in measuring perceptual learning.

The staircase method has been widely used in measuring perceptual learning. Recently, Zhao, Lesmes, and Lu (2017, 2019) developed the quick Change Detection (qCD) method and applied it to measure the trial-by-trial time course of dark adaptation. In the current study, we conducted two simulations to evaluate the performance of the 3-down/1-up staircase and qCD methods in measuring perceptual learning in a two-alternative forced-choice task. In Study 1, three observers with different time constants (40, 80, and 160 trials) of an exponential learning curve were simulated. Each simulated observer completed staircases with six step sizes (1%, 5%, 10%, 20%, 30%, and 60%) and a qCD procedure, each starting at five levels (+50%, +25%, 0, -25%, and -50% different from the true threshold in the first trial). We found the following results: Staircases with 1% and 5% step sizes failed to generate more than five reversals half of the time; and the bias and standard deviations of thresholds estimated from the post hoc segment-by-segment qCD analysis were much smaller than those from the staircase method with the other four step sizes. In Study 2, we simulated thresholds in the transfer phases with the same time constants and 50% transfer for each observer in Study 1. We found that the estimated transfer indexes from qCD showed smaller biases and standard deviations than those from the staircase method. In addition, rescoring the simulated data from the staircase method using the Bayesian estimation component of the qCD method resulted in much-improved estimates. We conclude that the qCD method characterizes the time course of perceptual learning and transfer more accurately, precisely, and efficiently than the staircase method, even with the optimal 10% step size.

Assessing the detailed time course of perceptual sensitivity change in perceptual learning.

The learning curve in perceptual learning is typically sampled in blocks of trials, which could result in imprecise and possibly biased estimates, especially when learning is rapid. Recently, Zhao, Lesmes, and Lu (2017, 2019) developed a Bayesian adaptive quick Change Detection (qCD) method to accurately, precisely, and efficiently assess the time course of perceptual sensitivity change. In this study, we implemented and tested the qCD method in assessing the learning curve in a four-alternative forced-choice global motion direction identification task in both simulations and a psychophysical experiment. The stimulus intensity in each trial was determined by the qCD, staircase or random stimulus selection (RSS) methods. Simulations showed that the accuracy (bias) and precision (standard deviation or confidence bounds) of the estimated learning curves from the qCD were much better than those obtained by the staircase and RSS method; this is true for both trial-by-trial and post hoc segment-by-segment qCD analyses. In the psychophysical experiment, the average half widths of the 68.2% credible interval of the estimated thresholds from the trial-by-trial and post hoc segment-by-segment qCD analyses were both quite small. Additionally, the overall estimates from the qCD and staircase methods matched extremely well in this task where the behavioral rate of learning is relatively slow. Our results suggest that the qCD method can precisely and accurately assess the trial-by-trial time course of perceptual learning.

High reward enhances perceptual learning.

Studies of perceptual learning have revealed a great deal of plasticity in adult humans. In this study, we systematically investigated the effects and mechanisms of several forms (trial-by-trial, block, and session rewards) and levels (no, low, high, subliminal) of monetary reward on the rate, magnitude, and generalizability of perceptual learning. We found that high monetary reward can greatly promote the rate and boost the magnitude of learning and enhance performance in untrained spatial frequencies and eye without changing interocular, interlocation, and interdirection transfer indices. High reward per se made unique contributions to the enhanced learning through improved internal noise reduction. Furthermore, the effects of high reward on perceptual learning occurred in a range of perceptual tasks. The results may have major implications for the understanding of the nature of the learning rule in perceptual learning and for the use of reward to enhance perceptual learning in practical applications.

Mixture of easy trials enables transient and sustained perceptual improvements through priming and perceptual learning.

The sense of vision allows us to discriminate fine details across a wide range of tasks. How to improve this perceptual skill, particularly within a short training session, is of substantial interest. Emerging evidence suggests that mixing easy trials can quickly improve performance in hard trials, but it is equivocal whether the improvement is short-lived or long-lasting, and additionally what accounts for this improvement. Here, by tracking objective performance (accuracy) and subjective experience (ratings of target visibility and choice confidence) over time and in a large sample of participants, we demonstrate the coexistence of transient and sustained effects of mixing easy trials, which differ markedly in their timescales, in their effects on subjective awareness, and in individual differences. In particular, whereas the transient effect was found to be ubiquitous and manifested similarly across objective and subjective measures, the sustained effect was limited to a subset of participants with weak convergence from objective and subjective measures. These results indicate that mixture of easy trials enables two distinct, co-existing forms of rapid perceptual improvements in hard trials, as mediated by robust priming and fragile learning. Placing constraints on theory of brain plasticity, this finding may also have implications for alleviating visual deficits.

Translating Perceptual Learning from the Laboratory to Applications.

Human training studies in the laboratory have demonstrated plasticity in brain systems and, in some cases, large improvements in perceptual performance, inspiring a quest to develop training apps and systems. We consider the next steps in the translation from the laboratory to the clinic and commerce.

Augmented Hebbian reweighting accounts for accuracy and induced bias in perceptual learning with reverse feedback.

Using an asymmetrical set of vernier stimuli (-15″, -10″, -5″, +10″, +15″) together with reverse feedback on the small subthreshold offset stimulus (-5″) induces response bias in performance (Aberg & Herzog, 2012; Herzog, Eward, Hermens, & Fahle, 2006; Herzog & Fahle, 1999). These conditions are of interest for testing models of perceptual learning because the world does not always present balanced stimulus frequencies or accurate feedback. Here we provide a comprehensive model for the complex set of asymmetric training results using the augmented Hebbian reweighting model (Liu, Dosher, & Lu, 2014; Petrov, Dosher, & Lu, 2005, 2006) and the multilocation integrated reweighting theory (Dosher, Jeter, Liu, & Lu, 2013). The augmented Hebbian learning algorithm incorporates trial-by-trial feedback, when present, as another input to the decision unit and uses the observer's internal response to update the weights otherwise; block feedback alters the weights on bias correction (Liu et al., 2014). Asymmetric training with reversed feedback incorporates biases into the weights between representation and decision. The model correctly predicts the basic induction effect, its dependence on trial-by-trial feedback, and the specificity of bias to stimulus orientation and spatial location, extending the range of augmented Hebbian reweighting accounts of perceptual learning.

Separating decision and encoding noise in signal detection tasks.

In this article we develop an extension to the signal detection theory framework to separately estimate internal noise arising from representational and decision processes. Our approach constrains signal detection theory models with decision noise by combining a multipass external noise paradigm with confidence rating responses. In a simulation study we present evidence that representation and decision noise can be separately estimated over a range of representative underlying representational and decision noise level configurations. These results also hold across a number of decision rules and show resilience to rule miss-specification. The new theoretical framework is applied to a visual detection confidence-rating task with 3 and 5 response categories. This study compliments and extends the recent efforts of researchers (Benjamin, Diaz, & Wee, 2009; Mueller & Weidemann, 2008; Rosner & Kochanski, 2009; Kellen, Klauer, & Singmann, 2012) to separate and quantify underlying sources of response variability in signal detection tasks.

Construction and evaluation of an integrated dynamical model of visual motion perception.

Although numerous models describe the individual neural mechanisms that may be involved in the perception of visual motion, few of them have been constructed to take arbitrary stimuli and map them to a motion percept. Here, we propose an integrated dynamical motion model (IDM), which is sufficiently general to handle diverse moving stimuli, yet sufficiently precise to account for a wide-ranging set of empirical observations made on a family of random dot kinematograms. In particular, we constructed models of the cortical areas involved in motion detection, motion integration and perceptual decision. We analyzed their parameters through dynamical simulations and numerical continuation to constrain their proper ranges. Then, empirical data from a family of random dot kinematograms experiments with systematically varying direction distribution, presentation duration and stimulus size, were used to evaluate our model and estimate corresponding model parameters. The resulting model provides an excellent account of a demanding set of parametrically varied behavioral effects on motion perception, providing both quantitative and qualitative elements of evaluation.

Generating a taxonomy of spatially cued attention for visual discrimination: effects of judgment precision and set size on attention.

Attention precues improve the performance of perceptual tasks in many but not all circumstances. These spatial attention effects may depend upon display set size or workload, and have been variously attributed to external noise filtering, stimulus enhancement, contrast gain, or response gain, or to uncertainty or other decision effects. In this study, we document systematically different effects of spatial attention in low- and high-precision judgments, with and without external noise, and in different set sizes in order to contribute to the development of a taxonomy of spatial attention. An elaborated perceptual template model (ePTM) provides an integrated account of a complex set of effects of spatial attention with just two attention factors: a set-size dependent exclusion or filtering of external noise and a narrowing of the perceptual template to focus on the signal stimulus. These results are related to the previous literature by classifying the judgment precision and presence of external noise masks in those experiments, suggesting a taxonomy of spatially cued attention in discrimination accuracy.

An integrated reweighting theory of perceptual learning.

Improvements in performance on visual tasks due to practice are often specific to a retinal position or stimulus feature. Many researchers suggest that specific perceptual learning alters selective retinotopic representations in early visual analysis. However, transfer is almost always practically advantageous, and it does occur. If perceptual learning alters location-specific representations, how does it transfer to new locations? An integrated reweighting theory explains transfer over retinal locations by incorporating higher level location-independent representations into a multilevel learning system. Location transfer is mediated through location-independent representations, whereas stimulus feature transfer is determined by stimulus similarity at both location-specific and location-independent levels. Transfer to new locations/positions differs fundamentally from transfer to new stimuli. After substantial initial training on an orientation discrimination task, switches to a new location or position are compared with switches to new orientations in the same position, or switches of both. Position switches led to the highest degree of transfer, whereas orientation switches led to the highest levels of specificity. A computational model of integrated reweighting is developed and tested that incorporates the details of the stimuli and the experiment. Transfer to an identical orientation task in a new position is mediated via more broadly tuned location-invariant representations, whereas changing orientation in the same position invokes interference or independent learning of the new orientations at both levels, reflecting stimulus dissimilarity. Consistent with single-cell recording studies, perceptual learning alters the weighting of both early and midlevel representations of the visual system.

Mixed training at high and low accuracy levels leads to perceptual learning without feedback.

In this study, we investigated whether mixing easy and difficult trials can lead to learning in the difficult conditions. We hypothesized that while feedback is necessary for significant learning in training regimes consisting solely of low training accuracy trials, training mixtures with sufficient proportions of high accuracy training trials would lead to significant learning without feedback. Thirty-six subjects were divided into one experimental group in which trials with high training accuracy were mixed with those with low training accuracy and no feedback, and five control groups in which high and low accuracy training were mixed in the presence of feedback; high and high training accuracy were mixed or low and low training accuracy were mixed with and without feedback trials. Contrast threshold improved significantly in the low accuracy condition in the presence of high training accuracy trials (the high-low mixture group) in the absence of feedback, although no significant learning was found in the low accuracy condition in the group with the low-low mixture without feedback. Moreover, the magnitude of improvement in low accuracy trials without feedback in the high-low training mixture is comparable to that in the high accuracy training without feedback condition and those obtained in the presence of trial-by-trial external feedback. The results are both qualitatively and quantitatively consistent with the predictions of the Augmented Hebbian Re-Weighting model. We conclude that mixed training at high and low accuracy levels can lead to perceptual learning at low training accuracy levels without feedback.

Visual perceptual learning.

Perceptual learning refers to the phenomenon that practice or training in perceptual tasks often substantially improves perceptual performance. Often exhibiting stimulus or task specificities, perceptual learning differs from learning in the cognitive or motor domains. Research on perceptual learning reveals important plasticity in adult perceptual systems, and as well as the limitations in the information processing of the human observer. In this article, we review the behavioral results, mechanisms, physiological basis, computational models, and applications of visual perceptual learning.

Information-limited parallel processing in difficult heterogeneous covert visual search.

Difficult visual search is often attributed to time-limited serial attention operations, although neural computations in the early visual system are parallel. Using probabilistic search models (Dosher, Han, & Lu, 2004) and a full time-course analysis of the dynamics of covert visual search, we distinguish unlimited capacity parallel versus serial search mechanisms. Performance is measured for difficult and error-prone searches among heterogeneous background elements and for easy and accurate searches among homogeneous background elements. Contrary to the claims of time-limited serial attention, searches in heterogeneous backgrounds instead exhibited nearly identical search dynamics for display sizes up to 12 items. A review and new analyses indicate that most difficult as well as easy visual searches operate as an unlimited-capacity parallel analysis over the visual field within a single eye fixation, which suggests limitations in the availability of information, not temporal bottlenecks in analysis or comparison. Serial properties likely reflect overt attention expressed in eye movements.

Specificity of perceptual learning increases with increased training.

Perceptual learning often shows substantial and long-lasting changes in the ability to classify relevant perceptual stimuli due to practice. Specificity to trained stimuli and tasks is a key characteristic of visual perceptual learning, but little is known about whether specificity depends upon the extent of initial training. Using an orientation discrimination task, we demonstrate that specificity follows after extensive training, while the earliest stages of perceptual learning exhibit substantial transfer to a new location and an opposite orientation. Brief training shows the best performance at the point of transfer. These results for orientation-location transfer have both theoretical and practical implications for understanding perceptual expertise.