Perceptual Learning beyond Perception: Mnemonic Representation in Early Visual Cortex and Intraparietal Sulcus.

The ability to discriminate between stimuli relies on a chain of neural operations associated with perception, memory and decision-making. Accumulating studies show learning-dependent plasticity in perception or decision-making, yet whether perceptual learning modifies mnemonic processing remains unclear. Here, we trained human participants of both sexes in an orientation discrimination task, while using functional magnetic resonance imaging (fMRI) and transcranial magnetic stimulation (TMS) to separately examine training-induced changes in working memory (WM) representation. fMRI decoding revealed orientation-specific neural patterns during the delay period in primary visual cortex (V1) before, but not after, training, whereas neurodisruption of V1 during the delay period led to behavioral deficits in both phases. In contrast, both fMRI decoding and disruptive effect of TMS showed that intraparietal sulcus (IPS) represented WM content after, but not before, training. These results suggest that training does not affect the necessity of sensory area in representing WM information, consistent with the sensory recruitment hypothesis in WM, but likely alters the coding format of the stored stimulus in this region. On the other hand, training can render WM content to be maintained in higher-order parietal areas, complementing sensory area to support more robust maintenance of information.SIGNIFICANCE STATEMENT There has been accumulating progresses regarding experience-dependent plasticity in perception or decision-making, yet how perceptual experience moulds mnemonic processing of visual information remains less explored. Here, we provide novel findings that learning-dependent improvement of discriminability accompanies altered WM representation at different cortical levels. Critically, we suggest a role of training in modulating cortical locus of WM representation, providing a plausible explanation to reconcile the discrepant findings between human and animal studies regarding the recruitment of sensory or higher-order areas in WM.

Inverse transformed encoding models - a solution to the problem of correlated trial-by-trial parameter estimates in fMRI decoding.

Techniques of multivariate pattern analysis (MVPA) can be used to decode the discrete experimental condition or a continuous modulator variable from measured brain activity during a particular trial. In functional magnetic resonance imaging (fMRI), trial-wise response amplitudes are sometimes estimated from the measured signal using a general linear model (GLM) with one onset regressor for each trial. When using rapid event-related designs with trials closely spaced in time, those estimates are highly variable and serially correlated due to the temporally extended shape of the hemodynamic response function (HRF). Here, we describe inverse transformed encoding modelling (ITEM), a principled approach of accounting for those serial correlations and decoding from the resulting estimates, at low computational cost and with no loss in statistical power. We use simulated data to show that ITEM outperforms the current standard approach in terms of decoding accuracy and analyze empirical data to demonstrate that ITEM is capable of visual reconstruction from fMRI signals.

Edge-Related Activity Is Not Necessary to Explain Orientation Decoding in Human Visual Cortex.

Multivariate pattern analysis is a powerful technique; however, a significant theoretical limitation in neuroscience is the ambiguity in interpreting the source of decodable information used by classifiers. This is exemplified by the continued controversy over the source of orientation decoding from fMRI responses in human V1. Recently Carlson (2014) identified a potential source of decodable information by modeling voxel responses based on the Hubel and Wiesel (1972) ice-cube model of visual cortex. The model revealed that activity associated with the edges of gratings covaries with orientation and could potentially be used to discriminate orientation. Here we empirically evaluate whether "edge-related activity" underlies orientation decoding from patterns of BOLD response in human V1. First, we systematically mapped classifier performance as a function of stimulus location using population receptive field modeling to isolate each voxel's overlap with a large annular grating stimulus. Orientation was decodable across the stimulus; however, peak decoding performance occurred for voxels with receptive fields closer to the fovea and overlapping with the inner edge. Critically, we did not observe the expected second peak in decoding performance at the outer stimulus edge as predicted by the edge account. Second, we evaluated whether voxels that contribute most to classifier performance have receptive fields that cluster in cortical regions corresponding to the retinotopic location of the stimulus edge. Instead, we find the distribution of highly weighted voxels to be approximately random, with a modest bias toward more foveal voxels. Our results demonstrate that edge-related activity is likely not necessary for orientation decoding. SIGNIFICANCE STATEMENT: A significant theoretical limitation of multivariate pattern analysis in neuroscience is the ambiguity in interpreting the source of decodable information used by classifiers. For example, orientation can be decoded from BOLD activation patterns in human V1, even though orientation columns are at a finer spatial scale than 3T fMRI. Consequently, the source of decodable information remains controversial. Here we test the proposal that information related to the stimulus edges underlies orientation decoding. We map voxel population receptive fields in V1 and evaluate orientation decoding performance as a function of stimulus location in retinotopic cortex. We find orientation is decodable from voxels whose receptive fields do not overlap with the stimulus edges, suggesting edge-related activity does not substantially drive orientation decoding.

Cortical activation associated with motor preparation can be used to predict the freely chosen effector of an upcoming movement and reflects response time: An fMRI decoding study.

Motor action is prepared in the human brain for rapid initiation at the appropriate time. Recent non-invasive decoding techniques have shown that brain activity for action preparation represents various parameters of an upcoming action. In the present study, we demonstrated that a freely chosen effector can be predicted from brain activity measured using functional magnetic resonance imaging (fMRI) before initiation of the action. Furthermore, the activity was related to response time (RT). We measured brain activity with fMRI while 12 participants performed a finger-tapping task using either the left or right hand, which was freely chosen by them. Using fMRI decoding, we identified brain regions in which activity during the preparatory period could predict the hand used for the upcoming action. We subsequently evaluated the relationship between brain activity and the RT of the upcoming action to determine whether correct decoding was associated with short RT. We observed that activity in the supplementary motor area, dorsal premotor cortex (PMd), and primary motor cortex (M1) measured before action execution predicted the hand used to perform the action with significantly above-chance accuracy (approximately 70%). Furthermore, in most participants, the RT was shorter in trials for which the used hand was correctly predicted by activity in the PMd and M1. The present study showed that preparatory activity in cortical motor areas represents information about the effector used for an upcoming action, and that well-formed motor representations in these regions are associated with reduced response times.

Orientation decoding in human visual cortex: new insights from an unbiased perspective.

The development of multivariate pattern analysis or brain "decoding" methods has substantially altered the field of fMRI research. Although these methods are highly sensitive to whether or not decodable information exists, the information they discover and make use of for decoding is often concealed within complex patterns of activation. This opacity of interpretation is embodied in influential studies showing that the orientation of visual gratings can be decoded from brain activity in human visual cortex with fMRI. Although these studies provided a compelling demonstration of the power of these methods, their findings were somewhat mysterious as the scanning resolution was insufficient to resolve orientation columns, i.e., orientation information should not have been accessible. Two theories have been put forth to account for this result, the hyperacuity account and the biased map account, both of which assume that small biases in fMRI voxels are the source of decodable information. In the present study, we use Hubel and Wiesel's (1972) classic ice-cube model of visual cortex to show that the orientation of gratings can be decoded from an unbiased representation. In our analysis, we identify patterns of activity elicited by the edges of the stimulus as the source of the decodable information. Furthermore, these activation patterns masquerade as a radial bias, a key element of the biased map account. This classic model thus sheds new light on the mystery behind orientation decoding by unveiling a new source of decodable information.

Task-dependent decoding of speaker and vowel identity from auditory cortical response patterns.

Selective attention to relevant sound properties is essential for everyday listening situations. It enables the formation of different perceptual representations of the same acoustic input and is at the basis of flexible and goal-dependent behavior. Here, we investigated the role of the human auditory cortex in forming behavior-dependent representations of sounds. We used single-trial fMRI and analyzed cortical responses collected while subjects listened to the same speech sounds (vowels /a/, /i/, and /u/) spoken by different speakers (boy, girl, male) and performed a delayed-match-to-sample task on either speech sound or speaker identity. Univariate analyses showed a task-specific activation increase in the right superior temporal gyrus/sulcus (STG/STS) during speaker categorization and in the right posterior temporal cortex during vowel categorization. Beyond regional differences in activation levels, multivariate classification of single trial responses demonstrated that the success with which single speakers and vowels can be decoded from auditory cortical activation patterns depends on task demands and subject's behavioral performance. Speaker/vowel classification relied on distinct but overlapping regions across the (right) mid-anterior STG/STS (speakers) and bilateral mid-posterior STG/STS (vowels), as well as the superior temporal plane including Heschl's gyrus/sulcus. The task dependency of speaker/vowel classification demonstrates that the informative fMRI response patterns reflect the top-down enhancement of behaviorally relevant sound representations. Furthermore, our findings suggest that successful selection, processing, and retention of task-relevant sound properties relies on the joint encoding of information across early and higher-order regions of the auditory cortex.

Behavior can be decoded across the cortex when individual differences are considered.

Group-level analyses have typically associated behavioral signatures with a constrained set of brain areas. Here we show that two behavioral metrics - reaction time (RT) and confidence - can be decoded across the cortex when each individual is considered separately. Subjects (N=50) completed a perceptual decision-making task with confidence. We built models decoding trial-level RT and confidence separately for each subject using the activation patterns in one brain area at a time after splitting the entire cortex into 200 regions of interest (ROIs). At the group level, we replicated previous results by showing that both RT and confidence could be decoded from a small number of ROIs (12.0% and 3.5%, respectively). Critically, at the level of the individual, both RT and confidence could be decoded from most brain regions even after Bonferroni correction (90.0% and 72.5%, respectively). Surprisingly, we observed that many brain regions exhibited opposite brain-behavior relationships across individuals, such that, for example, higher activations predicted fast RTs in some subjects but slow RTs in others. These results were further replicated in a second dataset. Lastly, we developed a simple test to determine the robustness of decoding performance, which showed that several hundred trials per subject are required for robust decoding. These results show that behavioral signatures can be decoded from a much broader range of cortical areas than previously recognized and suggest the need to study the brain-behavior relationship at both the group and the individual level.

Influence of Individual Differences in fMRI-Based Pain Prediction Models on Between-Individual Prediction Performance.

Decoding subjective pain perception from functional magnetic resonance imaging (fMRI) data using machine learning technique is gaining a growing interest. Despite the well-documented individual differences in pain experience and brain responses, it still remains unclear how and to what extent these individual differences affect the performance of between-individual fMRI-based pain prediction. The present study is aimed to examine the relationship between individual differences in pain prediction models and between-individual prediction error, and, further, to identify brain regions that contribute to between-individual prediction error. To this end, we collected and analyzed fMRI data and pain ratings in a laser-evoked pain experiment. By correlating different types of individual difference metrics with between-individual prediction error, we are able to quantify the influence of these individual differences on prediction performance and reveal a set of brain regions whose activities are related to prediction error. Interestingly, we found that the precuneus, which does not have predictive capability to pain, could also affect the prediction error. This study elucidates the influence of interindividual variability in pain on the between-individual prediction performance, and the results will be useful for the design of more accurate and robust fMRI-based pain prediction models.