Transfer of visual perceptual learning over a task-irrelevant feature through feature-invariant representations: Behavioral experiments and model simulations.

A large body of literature has examined specificity and transfer of perceptual learning, suggesting a complex picture. Here, we distinguish between transfer over variations in a "task-relevant" feature (e.g., transfer of a learned orientation task to a different reference orientation) and transfer over a "task-irrelevant" feature (e.g., transfer of a learned orientation task to a different retinal location or different spatial frequency), and we focus on the mechanism for the latter. Experimentally, we assessed whether learning a judgment of one feature (such as orientation) using one value of an irrelevant feature (e.g., spatial frequency) transfers to another value of the irrelevant feature. Experiment 1 examined whether learning in eight-alternative orientation identification with one or multiple spatial frequencies transfers to stimuli at five different spatial frequencies. Experiment 2 paralleled Experiment 1, examining whether learning in eight-alternative spatial-frequency identification at one or multiple orientations transfers to stimuli with five different orientations. Training the orientation task with a single spatial frequency transferred widely to all other spatial frequencies, with a tendency to specificity when training with the highest spatial frequency. Training the spatial frequency task fully transferred across all orientations. Computationally, we extended the identification integrated reweighting theory (I-IRT) to account for the transfer data (Dosher, Liu, & Lu, 2023; Liu, Dosher, & Lu, 2023). Just as location-invariant representations in the original IRT explain transfer over retinal locations, incorporating feature-invariant representations effectively accounted for the observed transfer. Taken together, we suggest that feature-invariant representations can account for transfer of learning over a "task-irrelevant" feature.

Enabling identification of component processes in perceptual learning with nonparametric hierarchical Bayesian modeling.

Perceptual learning is a multifaceted process, encompassing general learning, between-session forgetting or consolidation, and within-session fast relearning and deterioration. The learning curve constructed from threshold estimates in blocks or sessions, based on tens or hundreds of trials, may obscure component processes; high temporal resolution is necessary. We developed two nonparametric inference procedures: a Bayesian inference procedure (BIP) to estimate the posterior distribution of contrast threshold in each learning block for each learner independently and a hierarchical Bayesian model (HBM) that computes the joint posterior distribution of contrast threshold across all learning blocks at the population, subject, and test levels via the covariance of contrast thresholds across blocks. We applied the procedures to the data from two studies that investigated the interaction between feedback and training accuracy in Gabor orientation identification over 1920 trials across six sessions and estimated learning curve with block sizes L = 10, 20, 40, 80, 160, and 320 trials. The HBM generated significantly better fits to the data, smaller standard deviations, and more precise estimates, compared to the BIP across all block sizes. In addition, the HBM generated unbiased estimates, whereas the BIP only generated unbiased estimates with large block sizes but exhibited increased bias with small block sizes. With L = 10, 20, and 40, we were able to consistently identify general learning, between-session forgetting, and rapid relearning and adaptation within sessions. The nonparametric HBM provides a general framework for fine-grained assessment of the learning curve and enables identification of component processes in perceptual learning.

Hierarchical Bayesian perceptual template modeling of mechanisms of spatial attention in central and peripheral cuing.

The external noise paradigm and perceptual template model (PTM) have successfully been applied to characterize observer properties and mechanisms of observer state changes (e.g. attention and perceptual learning) in several research domains, focusing on individual level analysis. In this study, we developed a new hierarchical Bayesian perceptual template model (HBPTM) to model the trial-by-trial data from all individuals and conditions in a published spatial cuing study within a single structure and compared its performance to that of a Bayesian Inference Procedure (BIP), which separately infers the posterior distributions of the model parameters for each individual subject without the hierarchical structure. The HBPTM allowed us to compute the joint posterior distribution of the hyperparameters and parameters at the population, observer, and experiment levels and make statistical inferences at all these levels. In addition, we ran a large simulation study that varied the number of observers and number of trials in each condition and demonstrated the advantage of the HBPTM over the BIP across all the simulated datasets. Although it is developed in the context of spatial attention, the HBPTM and its extensions can be used to model data from the external noise paradigm in other domains and enable predictions of human performance at both the population and individual levels.

Learning spatial frequency identification through reweighted decoding.

Perceptual learning, the improvement of perceptual judgments with practice, occurs in many visual tasks. There are, however, relatively fewer studies examining perceptual learning in spatial frequency judgments. In addition, perceptual learning has generally been studied in two-alternative tasks, occasionally in n-alternative tasks, and infrequently in identification. Recently, perceptual learning was found in an orientation identification task (eight-alternatives) and was well accounted for by a new identification integrated reweighting theory (I-IRT) (Liu et al., submitted). Here, we examined perceptual learning in a similar eight-alternative spatial frequency absolute identification task in two different training protocols, finding learning in the majority but not all observers. We fit the I-IRT to the spatial frequency learning data and discuss possible model explanations for variations in learning.

Visual attention in spatial cueing and visual search.

To characterize internal processes of an observer conducting perceptual tasks, we developed an observer model that combines the perceptual template model (PTM), the attention mechanisms in the PTM framework (Lu & Dosher, 1998), and uncertainty of signal detection theory (Green & Swets, 1966). The model was evaluated with a visual search experiment conducted in a range of external noise, signal contrast, and target-distractor similarity conditions. In each trial, eight Gabor patches were shown in each of two brief intervals, with one target at a different orientation from the distractors in one of the presentations. Subjects were precued to a subset of the stimuli (1, 2, 4, or 8) and asked to report (a) which interval contained the target and (b) where the target was. Individual roles of uncertainty and of attention in visual search were investigated by comparing models with and without an attention component. The results showed that decision uncertainty alone was sufficient to account for the set-size effect, even in conditions with high target-distractor similarity. Our theoretical model and empirical results provide a coherent picture regarding how visual information is selected and processed during feature search.

Roving: The causes of interference and re-enabled learning in multi-task visual training.

People routinely perform multiple visual judgments in the real world, yet, intermixing tasks or task variants during training can damage or even prevent learning. This paper explores why. We challenged theories of visual perceptual learning focused on plastic retuning of low-level retinotopic cortical representations by placing different task variants in different retinal locations, and tested theories of perceptual learning through reweighting (changes in readout) by varying task similarity. Discriminating different (but equivalent) and similar orientations in separate retinal locations interfered with learning, whereas training either with identical orientations or sufficiently different ones in different locations released rapid learning. This location crosstalk during learning renders it unlikely that the primary substrate of learning is retuning in early retinotopic visual areas; instead, learning likely involves reweighting from location-independent representations to a decision. We developed an Integrated Reweighting Theory (IRT), which has both V1-like location-specific representations and higher level (V4/IT or higher) location-invariant representations, and learns via reweighting the readout to decision, to predict the order of learning rates in different conditions. This model with suitable parameters successfully fit the behavioral data, as well as some microstructure of learning performance in a new trial-by-trial analysis.

Evaluating the performance of the staircase and quick Change Detection methods in measuring perceptual learning.

The staircase method has been widely used in measuring perceptual learning. Recently, Zhao, Lesmes, and Lu (2017, 2019) developed the quick Change Detection (qCD) method and applied it to measure the trial-by-trial time course of dark adaptation. In the current study, we conducted two simulations to evaluate the performance of the 3-down/1-up staircase and qCD methods in measuring perceptual learning in a two-alternative forced-choice task. In Study 1, three observers with different time constants (40, 80, and 160 trials) of an exponential learning curve were simulated. Each simulated observer completed staircases with six step sizes (1%, 5%, 10%, 20%, 30%, and 60%) and a qCD procedure, each starting at five levels (+50%, +25%, 0, -25%, and -50% different from the true threshold in the first trial). We found the following results: Staircases with 1% and 5% step sizes failed to generate more than five reversals half of the time; and the bias and standard deviations of thresholds estimated from the post hoc segment-by-segment qCD analysis were much smaller than those from the staircase method with the other four step sizes. In Study 2, we simulated thresholds in the transfer phases with the same time constants and 50% transfer for each observer in Study 1. We found that the estimated transfer indexes from qCD showed smaller biases and standard deviations than those from the staircase method. In addition, rescoring the simulated data from the staircase method using the Bayesian estimation component of the qCD method resulted in much-improved estimates. We conclude that the qCD method characterizes the time course of perceptual learning and transfer more accurately, precisely, and efficiently than the staircase method, even with the optimal 10% step size.

Assessing the detailed time course of perceptual sensitivity change in perceptual learning.

The learning curve in perceptual learning is typically sampled in blocks of trials, which could result in imprecise and possibly biased estimates, especially when learning is rapid. Recently, Zhao, Lesmes, and Lu (2017, 2019) developed a Bayesian adaptive quick Change Detection (qCD) method to accurately, precisely, and efficiently assess the time course of perceptual sensitivity change. In this study, we implemented and tested the qCD method in assessing the learning curve in a four-alternative forced-choice global motion direction identification task in both simulations and a psychophysical experiment. The stimulus intensity in each trial was determined by the qCD, staircase or random stimulus selection (RSS) methods. Simulations showed that the accuracy (bias) and precision (standard deviation or confidence bounds) of the estimated learning curves from the qCD were much better than those obtained by the staircase and RSS method; this is true for both trial-by-trial and post hoc segment-by-segment qCD analyses. In the psychophysical experiment, the average half widths of the 68.2% credible interval of the estimated thresholds from the trial-by-trial and post hoc segment-by-segment qCD analyses were both quite small. Additionally, the overall estimates from the qCD and staircase methods matched extremely well in this task where the behavioral rate of learning is relatively slow. Our results suggest that the qCD method can precisely and accurately assess the trial-by-trial time course of perceptual learning.

High reward enhances perceptual learning.

Studies of perceptual learning have revealed a great deal of plasticity in adult humans. In this study, we systematically investigated the effects and mechanisms of several forms (trial-by-trial, block, and session rewards) and levels (no, low, high, subliminal) of monetary reward on the rate, magnitude, and generalizability of perceptual learning. We found that high monetary reward can greatly promote the rate and boost the magnitude of learning and enhance performance in untrained spatial frequencies and eye without changing interocular, interlocation, and interdirection transfer indices. High reward per se made unique contributions to the enhanced learning through improved internal noise reduction. Furthermore, the effects of high reward on perceptual learning occurred in a range of perceptual tasks. The results may have major implications for the understanding of the nature of the learning rule in perceptual learning and for the use of reward to enhance perceptual learning in practical applications.

An integrated reweighting theory of perceptual learning.

Improvements in performance on visual tasks due to practice are often specific to a retinal position or stimulus feature. Many researchers suggest that specific perceptual learning alters selective retinotopic representations in early visual analysis. However, transfer is almost always practically advantageous, and it does occur. If perceptual learning alters location-specific representations, how does it transfer to new locations? An integrated reweighting theory explains transfer over retinal locations by incorporating higher level location-independent representations into a multilevel learning system. Location transfer is mediated through location-independent representations, whereas stimulus feature transfer is determined by stimulus similarity at both location-specific and location-independent levels. Transfer to new locations/positions differs fundamentally from transfer to new stimuli. After substantial initial training on an orientation discrimination task, switches to a new location or position are compared with switches to new orientations in the same position, or switches of both. Position switches led to the highest degree of transfer, whereas orientation switches led to the highest levels of specificity. A computational model of integrated reweighting is developed and tested that incorporates the details of the stimuli and the experiment. Transfer to an identical orientation task in a new position is mediated via more broadly tuned location-invariant representations, whereas changing orientation in the same position invokes interference or independent learning of the new orientations at both levels, reflecting stimulus dissimilarity. Consistent with single-cell recording studies, perceptual learning alters the weighting of both early and midlevel representations of the visual system.

Augmented Hebbian reweighting accounts for accuracy and induced bias in perceptual learning with reverse feedback.

Using an asymmetrical set of vernier stimuli (-15″, -10″, -5″, +10″, +15″) together with reverse feedback on the small subthreshold offset stimulus (-5″) induces response bias in performance (Aberg & Herzog, 2012; Herzog, Eward, Hermens, & Fahle, 2006; Herzog & Fahle, 1999). These conditions are of interest for testing models of perceptual learning because the world does not always present balanced stimulus frequencies or accurate feedback. Here we provide a comprehensive model for the complex set of asymmetric training results using the augmented Hebbian reweighting model (Liu, Dosher, & Lu, 2014; Petrov, Dosher, & Lu, 2005, 2006) and the multilocation integrated reweighting theory (Dosher, Jeter, Liu, & Lu, 2013). The augmented Hebbian learning algorithm incorporates trial-by-trial feedback, when present, as another input to the decision unit and uses the observer's internal response to update the weights otherwise; block feedback alters the weights on bias correction (Liu et al., 2014). Asymmetric training with reversed feedback incorporates biases into the weights between representation and decision. The model correctly predicts the basic induction effect, its dependence on trial-by-trial feedback, and the specificity of bias to stimulus orientation and spatial location, extending the range of augmented Hebbian reweighting accounts of perceptual learning.

Hierarchical Bayesian Augmented Hebbian Reweighting Model of Perceptual Learning.

The Augmented Hebbian Reweighting Model (AHRM) has been effectively utilized to model the collective performance of observers in various perceptual learning studies. In this work, we have introduced a novel hierarchical Bayesian Augmented Hebbian Reweighting Model (HB-AHRM) to simultaneously model the learning curves of individual participants and the entire population within a single framework. We have compared its performance to that of a Bayesian Inference Procedure (BIP), which independently estimates the posterior distributions of model parameters for each individual subject without employing a hierarchical structure. To cope with the substantial computational demands, we developed an approach to approximate the likelihood function in the AHRM with feature engineering and linear regression, increasing the speed of the estimation procedure by 20,000 times. The HB-AHRM has enabled us to compute the joint posterior distribution of hyperparameters and parameters at the population, observer, and test levels, facilitating statistical inferences across these levels. While we have developed this methodology within the context of a single experiment, the HB-AHRM and the associated modeling techniques can be readily applied to analyze data from various perceptual learning experiments and provide predictions of human performance at both the population and individual levels. The likelihood approximation concept introduced in this study may have broader utility in fitting other stochastic models lacking analytic forms.

Informational feedback accelerates learning in multi-alternative perceptual judgements of orientation.

Experience or training can substantially improve perceptual performance through perceptual learning, and the extent and rate of these improvements may be affected by feedback. In this paper, we first developed a neural network model based on the integrated reweighting theory (Dosher et al., 2013) to account for perceptual learning and performance in n-alternative identification tasks and the dependence of learning on different forms of feedback. We then report an experiment comparing the effectiveness of response feedback (RF) versus accuracy feedback (AF) or no feedback (NF) (full versus partial versus no supervision) in learning a challenging eight-alternative visual orientation identification (8AFC) task. Although learning sometimes occurred in the absence of feedback (NF), RF had a clear advantage above AF or NF in this task. Using hybrid supervision learning rules, a new n-alternative identification integrated reweighting theory (I-IRT) explained both the differences in learning curves given different feedback and the dynamic changes in identification confusion data. This study shows that training with more informational feedback (RF) is more effective, though not necessary, in these challenging n-alternative tasks, a result that has implications for developing training paradigms in realistic tasks.

Estimating the Trial-by-Trial Learning Curve in Perceptual Learning with Hierarchical Bayesian Modeling.

The learning curve serves as a crucial metric for assessing human performance in perceptual learning. It may encompass various component processes, including general learning, between-session forgetting or consolidation, and within-session rapid relearning and adaptation or deterioration. Typically, empirical learning curves are constructed by aggregating tens or hundreds of trials of data in blocks or sessions. Here, we devised three inference procedures for estimating the trial-by-trial learning curve based on the multi-component functional form identified in Zhao et al. (submitted): general learning, between-session forgetting, and within-session rapid relearning and adaptation. These procedures include a Bayesian inference procedure (BIP) estimating the posterior distribution of parameters for each learner independently, and two hierarchical Bayesian models (HBMv and HBMc) computing the joint posterior distribution of parameters and hyperparameters at the population, subject, and test levels. The HBMv and HBMc incorporate variance and covariance hyperparameters, respectively, between and within subjects. We applied these procedures to data from two studies investigating the interaction between feedback and training accuracy in Gabor orientation identification across about 2000 trials spanning six sessions (Liu et al., 2010, 2012) and estimated the trial-by-trial learning curves at both the subject and population levels. The HBMc generated best fits to the data and the smallest half width of 68.2% credible interval of the learning curves compared to the BIP and HBMv. The parametric HBMc with the multi-component functional form provides a general framework for trial-by-trial analysis of the component processes in perceptual learning and for predicting the learning curve in unmeasured time points.

Current directions in visual perceptual learning.

The visual expertise of adult humans is jointly determined by evolution, visual development, and visual perceptual learning. Perceptual learning refers to performance improvements in perceptual tasks after practice or training in the task. It occurs in almost all visual tasks, ranging from simple feature detection to complex scene analysis. In this Review, we focus on key behavioral aspects of visual perceptual learning. We begin by describing visual perceptual learning tasks and manipulations that influence the magnitude of learning, and then discuss specificity of learning. Next, we present theories and computational models of learning and specificity. We then review applications of visual perceptual learning in visual rehabilitation. Finally, we summarize the general principles of visual perceptual learning, discuss the tension between plasticity and stability, and conclude with new research directions.

Attention extracts signal in external noise: a BOLD fMRI study.

On the basis of results from behavioral studies that spatial attention improves the exclusion of external noise in the target region, we predicted that attending to a spatial region would reduce the impact of external noise on the BOLD response in corresponding cortical areas, seen as reduced BOLD responses in conditions with large amounts of external noise but relatively low signal, and increased dynamic range of the BOLD response to variations in signal contrast. We found that, in the presence of external noise, covert attention reduced the trial-by-trial BOLD response by 15.5-18.9% in low signal contrast conditions in V1. It also increased the BOLD dynamic range in V1, V2, V3, V3A/B, and V4 by a factor of at least three. Overall, covert attention reduced the impact of external noise by about 73-85% in these early visual areas. It also increased the contrast gain by a factor of 2.6-3.8.

Visual perceptual learning.

Perceptual learning refers to the phenomenon that practice or training in perceptual tasks often substantially improves perceptual performance. Often exhibiting stimulus or task specificities, perceptual learning differs from learning in the cognitive or motor domains. Research on perceptual learning reveals important plasticity in adult perceptual systems, and as well as the limitations in the information processing of the human observer. In this article, we review the behavioral results, mechanisms, physiological basis, computational models, and applications of visual perceptual learning.

Blood oxygenation level-dependent contrast response functions identify mechanisms of covert attention in early visual areas.

Covert attention can lead to improved performance in perceptual tasks. The neural and functional mechanisms of covert attention are still under investigation. Using both rapid event-related and mixed designs, we measured the blood oxygenation level-dependent functional MRI contrast response functions over the full range of contrast (0-100%) in the retinotopically defined early visual areas (V1, V2, V3, V3A, and V4) in humans. Covert attention increased both the baseline activities and contrast gains in the five cortical areas. The effect on baseline can be decomposed into a transient trial-by-trial component and a component across an entire attention block. On average, increase in contrast gain accounted for approximately 88.0%, 28.5%, 12.7%, 35.9%, and 25.2% of the trial-by-trial effects of attention in the five areas, respectively, and 22.2%, 12.8%, 7.4%, 19.7%, and 17.3% of the total effects of attention in those areas, consistent with single-unit findings in V4 and MT. The results provide strong evidence for a stimulus enhancement mechanism of attention as demonstrated in various behavioral studies.

Augmented Hebbian reweighting: interactions between feedback and training accuracy in perceptual learning.

Feedback plays an interesting role in perceptual learning. The complex pattern of empirical results concerning the role of feedback in perceptual learning rules out both a pure supervised mode and a pure unsupervised mode of learning and leads some researchers to the proposal that feedback may change the learning rate through top-down control but does not act as a teaching signal in perceptual learning (M. H. Herzog & M. Fahle, 1998). In this study, we tested the predictions of an augmented Hebbian reweighting model (AHRM) of perceptual learning (A. Petrov, B. A. Dosher, & Z.-L. Lu, 2005), in which feedback influences the effective rate of learning by serving as an additional input and not as a direct teaching signal. We investigated the interactions between feedback and training accuracy in a Gabor orientation identification task over six training days. The accelerated stochastic approximation method was used to track threshold contrasts at particular performance accuracy levels throughout training. Subjects were divided into 4 groups: high training accuracy (85% correct) with and without feedback, and low training accuracy (65%) with and without feedback. Contrast thresholds improved in the high training accuracy condition, independent of the feedback condition. However, thresholds improved in the low training accuracy condition only in the presence of feedback but not in the absence of feedback. The results are both qualitatively and quantitatively consistent with the predictions of the augmented Hebbian learning model and are not consistent with pure supervised error correction or pure Hebbian learning models.

Characterizing perceptual performance at multiple discrimination precisions in external noise.

Existing observer models developed for studies with the external noise paradigm are strictly applicable only to target detection or identification/discrimination of orthogonal target(s). We elaborated the perceptual template model (PTM) to account for contrast thresholds in identifying nonorthogonal targets. Full contrast psychometric functions were measured in an orientation identification task with four orientation differences across a wide range of external noise levels. We showed that observer performance can be modeled by the elaborated PTM with two templates that correspond to the two stimulus categories. Sampling efficiencies of the human observers were also estimated. The elaborated PTM provides a theoretical framework for characterizing joint feature and contrast sensitivity of human observers.