A computational model of prefrontal and striatal interactions in perceptual category learning.

Work on multiple-system theories of cognition mostly focused on the systems themselves, while limited work has been devoted to understanding the interactions between systems. Generally, multiple-system theories include a model-based decision system supported by the prefrontal cortex and a model-free decision system supported by the striatum. Here we propose a neurobiological model to describe the interactions between model-based and model-free decision systems in category learning. The proposed model used spiking neurons to simulate activity of the hyperdirect pathway of the basal ganglia. The hyperdirect pathway acts as a gate for the response signal from the model-free system located in the striatum. We propose that the model-free system's response is inhibited when the model-based system is in control of the response. The new model was used to simulate published data from young adults, people with Parkinson's disease, and aged-matched older adults. The simulation results further suggest that system-switching ability may be related to individual differences in executive function. A new behavioral experiment tested this model prediction. The results show that an updating score predicts the ability to switch system in a categorization task. The article concludes with new model predictions and implications of the results for research on system interactions.

Cognitive flexibility assessment with a new Reversal learning task paradigm compared with the Wisconsin card sorting test exploring the moderating effect of gender and stress.

Cognitive flexibility plays a crucial role in psychological health and this research aimed to investigate its assessment. We developed a novel Reversal learning task (RLT) paradigm adding pure reward (+ 100 points, 0) and punishment (- 100 points, 0) conditions to the classic reward-punishment condition (+ 100, - 100); we also analyzed the RLT convergent validity with approach-avoidance questionnaires (BIS-BAS and Approach-Avoidance Temperament questionnaire) and the Wisconsin card sorting test (WCST) scores through a Principal component analysis. In a sample of 374 participants, we found that these three conditions differently assess flexibility and that high RLT reward sensitivity in the punishment condition (0; - 100) is related with high BAS reward responsiveness. Moreover, we found that RLT and WCST flexibility scores, although associated, detect different facets of cognitive flexibility. Finally, in a second sample (N = 172), we explored the impact of stress, moderated by gender, on RLT and WCST. Whereas, WCST was not impacted by these variables, in RLT stressed women showed increased perseverative errors in punishment condition (- 100, 0) and reduced punishment sensitivity in reward condition (+ 100, 0).Overall, our newly developed RLT paradigm and the WCST seem to provide different ways to assess cognitive flexibility and to be differently affected by moderators, such as gender and stress.

The impact of training methodology and representation on rule-based categorization: An fMRI study.

Hélie, Shamloo, & Ell (2017) showed that regular classification learning instructions (A/B) promote between-category knowledge in rule-based categorization whereas conceptual learning instructions (YES/NO) promote learning within-category knowledge with the same categories. Here we explore how these tasks affect brain activity using fMRI. Participants learned two sets of two categories. Computational models were fit to the behavioral data to determine the type of knowledge learned by each participant. fMRI contrasts were computed to compare BOLD signal between the tasks and between the types of knowledge. The results show that participants in the YES/NO task had more activity in the pre-supplementary motor area, prefrontal cortex, and the angular/supramarginal gyrus. These brain areas are related to working memory and part of the dorsal attention network, which showed increased task-based functional connectivity with the medial temporal lobes. In contrast, participants in the A/B task had more activity in the thalamus and caudate. These results suggest that participants in the YES/NO task used bivalent rules and may have treated each contextual question as a separate task, switching task each time the question changed. Activity in the A/B condition was more consistent with participants applying direct Stimulus → Response rules. With regards to knowledge representation, there was a large shared network of brain areas, but participants learning between-category information showed additional posterior parietal activity, which may be related to the inhibition of incorrect motor programs.

Whole-brain R1 predicts manganese exposure and biological effects in welders.

Manganese (Mn) is a neurotoxicant that, due to its paramagnetic property, also functions as a magnetic resonance imaging (MRI) T1 contrast agent. Previous studies in Mn toxicity have shown that Mn accumulates in the brain, which may lead to parkinsonian symptoms. In this article, we trained support vector machines (SVM) using whole-brain R1 (R1 = 1/T1) maps from 57 welders and 32 controls to classify subjects based on their air Mn concentration ([Mn]Air), Mn brain accumulation (ExMnBrain), gross motor dysfunction (UPDRS), thalamic GABA concentration (GABAThal), and total years welding. R1 was highly predictive of [Mn]Air above a threshold of 0.20 mg/m3 with an accuracy of 88.8% and recall of 88.9%. R1 was also predictive of subjects with GABAThal having less than or equal to 2.6 mM with an accuracy of 82% and recall of 78.9%. Finally, we used an SVM to predict age as a method of verifying that the results could be attributed to Mn exposure. We found that R1 was predictive of age below 48 years of age with accuracies ranging between 75 and 82% with recall between 94.7% and 76.9% but was not predictive above 48 years of age. Together, this suggests that lower levels of exposure (< 0.20 mg/m3 and < 18 years of welding on the job) do not produce discernable signatures, whereas higher air exposures and subjects with more total years welding produce signatures in the brain that are readily identifiable using SVM.

The impact of training methodology and category structure on the formation of new categories from existing knowledge.

Categorization decisions are made thousands of times every day, and a typical adult knows tens of thousands of categories. It is thus relatively rare that adults learn new categories without somehow reorganizing pre-existing knowledge. Yet, most perceptual categorization research has investigated the ability to learn new categories without considering they relation to existing knowledge. In this article, we test the ability of young adults to merge already known categories into new categories as a function of training methodology and category structures using two experiments. Experiment 1 tests participants' ability to merge rule-based or information-integration categories that are either contiguous, semi-contiguous, or non-contiguous in perceptual space using a classification paradigm. Experiment 2 is similar Experiment 1 but uses a YES/NO learning paradigm instead. The results of both experiments suggest a strong effect of the contiguity of the merged categories in perceptual space that depends on the type of category representation that is learned. The type of category representation that is learned, in turn, depends on a complex interaction of the category structures and training task. We conclude by discussing the relevance of these results for categorization outside the laboratory.

Trial-by-trial switching between procedural and declarative categorization systems.

Considerable evidence suggests that human category learning recruits multiple memory systems. A popular assumption is that procedural memory is used to form stimulus-to-response mappings, whereas declarative memory is used to form and test explicit rules about category membership. The multiple systems framework has been successful in motivating and accounting for a broad array of empirical observations over the past 20 years. Even so, only a couple of studies have examined how the different categorization systems interact. Both previous studies suggest that switching between explicit and procedural responding is extremely difficult. But they leave unanswered the critical questions of whether trial-by-trial system switching is possible, and if so, whether it is qualitatively different than trial-by-trial switching between two explicit tasks. The experiment described in this article addressed these questions. The results (1) confirm that effective trial-by-trial system switching, although difficult, is possible; (2) suggest that switching between tasks mediated by different memory systems is more difficult than switching between two declarative memory tasks; and (3) point to a serious shortcoming of current category-learning theories.

The effect of integration masking on visual processing in perceptual categorization.

Learning to recognize and categorize objects is an essential cognitive skill allowing animals to function in the world. However, animals rarely have access to a canonical view of an object in an uncluttered environment. Hence, it is essential to study categorization under noisy, degraded conditions. In this article, we explore how the brain processes categorization stimuli in low signal-to-noise conditions using multivariate pattern analysis. We used an integration masking paradigm with mask opacity of 50%, 60%, and 70% inside a magnetic resonance imaging scanner. The results show that mask opacity affects blood-oxygen-level dependent (BOLD) signal in visual processing areas (V1, V2, V3, and V4) but does not affect the BOLD signal in brain areas traditionally associated with categorization (prefrontal cortex, striatum, hippocampus). This suggests that when a stimulus is difficult to extract from its background (e.g., low signal-to-noise ratio), the visual system extracts the stimulus and that activity in areas typically associated with categorization are not affected by the difficulty level of the visual conditions. We conclude with implications of this result for research on visual attention, categorization, and the integration of these fields.

Trial-by-trial identification of categorization strategy using iterative decision-bound modeling.

Identifying the strategy that participants use in laboratory experiments is crucial in interpreting the results of behavioral experiments. This article introduces a new modeling procedure called iterative decision-bound modeling (iDBM), which iteratively fits decision-bound models to the trial-by-trial responses generated from single participants in perceptual categorization experiments. The goals of iDBM are to identify: (1) all response strategies used by a participant, (2) changes in response strategy, and (3) the trial number at which each change occurs. The new method is validated by testing its ability to identify the response strategies used in noisy simulated data. The benchmark simulation results show that iDBM is able to detect and identify strategy switches during an experiment and accurately estimate the trial number at which the strategy change occurs in low to moderate noise conditions. The new method is then used to reanalyze data from Ell and Ashby (2006). Applying iDBM revealed that increasing category overlap in an information-integration category learning task increased the proportion of participants who abandoned explicit rules, and reduced the number of training trials needed to abandon rules in favor of a procedural strategy. Finally, we discuss new research questions made possible through iDBM.

A neurocomputational model of automatic sequence production.

Most behaviors unfold in time and include a sequence of submovements or cognitive activities. In addition, most behaviors are automatic and repeated daily throughout life. Yet, relatively little is known about the neurobiology of automatic sequence production. Past research suggests a gradual transfer from the associative striatum to the sensorimotor striatum, but a number of more recent studies challenge this role of the BG in automatic sequence production. In this article, we propose a new neurocomputational model of automatic sequence production in which the main role of the BG is to train cortical-cortical connections within the premotor areas that are responsible for automatic sequence production. The new model is used to simulate four different data sets from human and nonhuman animals, including (1) behavioral data (e.g., RTs), (2) electrophysiology data (e.g., single-neuron recordings), (3) macrostructure data (e.g., TMS), and (4) neurological circuit data (e.g., inactivation studies). We conclude with a comparison of the new model with existing models of automatic sequence production and discuss a possible new role for the BG in automaticity and its implication for Parkinson's disease.

Criterion learning in rule-based categorization: simulation of neural mechanism and new data.

In perceptual categorization, rule selection consists of selecting one or several stimulus-dimensions to be used to categorize the stimuli (e.g., categorize lines according to their length). Once a rule has been selected, criterion learning consists of defining how stimuli will be grouped using the selected dimension(s) (e.g., if the selected rule is line length, define 'long' and 'short'). Very little is known about the neuroscience of criterion learning, and most existing computational models do not provide a biological mechanism for this process. In this article, we introduce a new model of rule learning called Heterosynaptic Inhibitory Criterion Learning (HICL). HICL includes a biologically-based explanation of criterion learning, and we use new category-learning data to test key aspects of the model. In HICL, rule selective cells in prefrontal cortex modulate stimulus-response associations using pre-synaptic inhibition. Criterion learning is implemented by a new type of heterosynaptic error-driven Hebbian learning at inhibitory synapses that uses feedback to drive cell activation above/below thresholds representing ionic gating mechanisms. The model is used to account for new human categorization data from two experiments showing that: (1) changing rule criterion on a given dimension is easier if irrelevant dimensions are also changing (Experiment 1), and (2) showing that changing the relevant rule dimension and learning a new criterion is more difficult, but also facilitated by a change in the irrelevant dimension (Experiment 2). We conclude with a discussion of some of HICL's implications for future research on rule learning.

The role of posterior parietal cortex in detecting changes in feedback contingency.

Well-practiced or learned behaviors are extremely resilient. For example, it is extremely difficult for a trained typist to forget how to use a keyboard configuration that they are familiar with. While they can be trained on a new keyboard configuration, the original skill quickly comes back when the old keyboard configuration is used again. This resiliency of learned skills is both a blessing and a curse. It makes useful skills durable, but it also makes maladaptive behaviors difficult to extinguish. Crossley et al. (2013) proposed a computational model and behavioral paradigm aimed at unlearning skills using various feedback contingency manipulations during an extinction phase. They showed that partially-valid feedback during extinction removed evidence for fast reacquisition, which they interpreted as evidence for unlearning. In this article, we replicated the Crossley et al. paradigm using fMRI. Univariate analyses showed differences in BOLD signals between the different experiment phases in the frontoparietal attention network. The superior and inferior parietal lobules (SPL and IPL, respectively) showed the largest cluster differences both between experimental phases and between extinction conditions. In contrast, the prefrontal cortex only showed differences in cluster of activities between extinction conditions. Multivariate pattern analysis was also used with seeds in the SPL and IPL. The results showed that these brain areas were critical in detecting changes in experimental phases. Overall, the fMRI results found mixed evidence for the Crossley et al. model and suggest that while unlearning prevents fast reacquisition, the absence of fast reacquisition does not necessarily implies that unlearning occurred.

The roles of intrinsic motivation and capability-related factors in cognitive effort-based decision-making.

Individual differences in cognitive effort-based decision-making can be used to reveal human motivations to invest effort into a given task. Preferences among options that differ by dimensions related to demand levels (i.e., the interaction of task characteristics and performance measures) are also heavily influenced by how likely a person can succeed at a given option. However, most existing cognitive effort-based research has focused primarily on demand-related factors, leading to confounding inferences about the motivation behind these choices. This study used an adaptive algorithm to adjust relative demand levels for three cognitive tasks to investigate general and individual differences in demand preferences. The results highlight an overall pattern of individual differences in intrinsic motivation to perform challenging tasks, supporting research that found cognitive effort aversive to some but attractive to others. These results suggest that relative demand levels and intrinsic task factors drive the motivation to select an action.

The effect of ostracism on prospective memory in problem solving.

Successfully generating plans, while seemingly straightforward, can be riddled with external and internal interferences. One important possible source of interference is ostracism, which has been consistently shown to induce negative psychological effects in various executive functions. Therefore, understanding the impact of unforeseen ostracism on planning is vital to a broad spectrum of the population, from university students, whose self-esteem partly derives from social acceptance, to healthcare professionals, whose performance oftentimes relies on peer feedback. An individual's ability to navigate through intended actions is an evaluation of their prospective memory (PM), which is traditionally divided into three consecutive phases: (1) planning, (2) recall, and (3) performance. This study primarily focused on the impacts of ostracism via Cyberball simulation on the first two phases of PM in the Tower of London (TOL), an assessment of executive functioning designed specifically to test planning ability during problem solving. Using Bayesian analysis, the study found substantial evidence of there being no difference in planning success between social exclusion and inclusion conditions. However, an individual's sex had significant effects on their planning success at baseline (i.e., inclusion condition). Surprisingly, there was no difference in performance between male participants and female participants when excluded, suggesting that ostracism may play an equalizing role. In addition, male participants both listed more moves at planning and recalled more moves, which led to no difference between sexes in terms of recall percentage. This study underscores a need to consider various factors such as sex and differing perceptions of ostracism when analyzing and addressing problem solving performance.

Brain activity across the development of automatic categorization: a comparison of categorization tasks using multi-voxel pattern analysis.

Previous evidence suggests that relatively separate neural networks underlie initial learning of rule-based and information-integration categorization tasks. With the development of automaticity, categorization behavior in both tasks becomes increasingly similar and exclusively related to activity in cortical regions. The present study uses multi-voxel pattern analysis to directly compare the development of automaticity in different categorization tasks. Each of the three groups of participants received extensive training in a different categorization task: either an information-integration task, or one of two rule-based tasks. Four training sessions were performed inside an MRI scanner. Three different analyses were performed on the imaging data from a number of regions of interest (ROIs). The common patterns analysis had the goal of revealing ROIs with similar patterns of activation across tasks. The unique patterns analysis had the goal of revealing ROIs with dissimilar patterns of activation across tasks. The representational similarity analysis aimed at exploring (1) the similarity of category representations across ROIs and (2) how those patterns of similarities compared across tasks. The results showed that common patterns of activation were present in motor areas and basal ganglia early in training, but only in the former later on. Unique patterns were found in a variety of cortical and subcortical areas early in training, but they were dramatically reduced with training. Finally, patterns of representational similarity between brain regions became increasingly similar across tasks with the development of automaticity.

Learning and transfer of category knowledge in an indirect categorization task.

Knowledge representations acquired during category learning experiments are 'tuned' to the task goal. A useful paradigm to study category representations is indirect category learning. In the present article, we propose a new indirect categorization task called the "same"-"different" categorization task. The same-different categorization task is a regular same-different task, but the question asked to the participants is about the stimulus category membership instead of stimulus identity. Experiment 1 explores the possibility of indirectly learning rule-based and information-integration category structures using the new paradigm. The results suggest that there is little learning about the category structures resulting from an indirect categorization task unless the categories can be separated by a one-dimensional rule. Experiment 2 explores whether a category representation learned indirectly can be used in a direct classification task (and vice versa). The results suggest that previous categorical knowledge acquired during a direct classification task can be expressed in the same-different categorization task only when the categories can be separated by a rule that is easily verbalized. Implications of these results for categorization research are discussed.

When is Psychology Research Useful in Artificial Intelligence? A Case for Reducing Computational Complexity in Problem Solving.

A problem is a situation in which an agent seeks to attain a given goal without knowing how to achieve it. Human problem solving is typically studied as a search in a problem space composed of states (information about the environment) and operators (to move between states). A problem such as playing a game of chess has 10 120 $10^{120}$ possible states, and a traveling salesperson problem with as little as 82 cities already has more than 10 120 $10^{120}$ different tours (similar to chess). Biological neurons are slower than the digital switches in computers. An exhaustive search of the problem space exceeds the capacity of current computers for most interesting problems, and it is fairly clear that humans cannot in their lifetime exhaustively search even small fractions of these problem spaces. Yet, humans play chess and solve logistical problems of similar complexity on a daily basis. Even for simple problems humans do not typically engage in exploring even a small fraction of the problem space. This begs the question: How do humans solve problems on a daily basis in a fast and efficient way? Recent work suggests that humans build a problem representation and solve the represented problem-not the problem that is out there. The problem representation that is built and the process used to solve it are constrained by limits of cognitive capacity and a cost-benefit analysis discounting effort and reward. In this article, we argue that better understanding the way humans represent and solve problems using heuristics can help inform how simpler algorithms and representations can be used in artificial intelligence to lower computational complexity, reduce computation time, and facilitate real-time computation in complex problem solving.

The effect of mode of presentation on Tower of Hanoi problem solving.

The Tower of Hanoi (TOH) is a classic problem that can be solved via multiple strategies. This study used TOH to examine how mode of presentation of a problem influences strategy use and transfer. Undergraduate students (Experiment 1) or Prolific workers (Experiment 2) completed two TOH problems of varying difficulty (4-disk/5-disk). They were randomly assigned to different conditions in which problems were either high in internal representation (mental) or high in external representation (computer). Participants were better able to complete problems successfully when external representations were available but completed problems in fewer moves when relying on internal representations. In addition, participants spent more time between moves when solving problems mentally, suggesting that external representations encourage speed while internal representations promote accuracy when solving recursion problems. Lastly, both experiments provide evidence that first solving a problem mentally encouraged participants to use strategies similar to goal recursion on a second problem.

Evidence for cortical automaticity in rule-based categorization.

There is evidence that rule-based category learning is supported by a broad neural network that includes the prefrontal cortex, the anterior cingulate cortex, the head of the caudate nucleus, and medial temporal lobe structures. Although thousands of studies have examined rule-based category learning, only a few have studied the development of automaticity in rule-based tasks. Categorizing by a newly learned rule makes heavy demands on declarative memory, but after thousands of repetitions rule-based categorizations are made with no apparent effort. Thus, it seems likely that the neural systems that mediate automatic rule-based categorization are substantially different from the systems that mediate initial learning. This research aims at identifying the neural systems responsible for early and late rule-based categorization performances. Toward this end, this article reports the results of an experiment in which human participants each practiced a rule-based categorization task for >10,000 trials distributed over 20 separate sessions. Sessions 1, 4, 10, and 20 were performed inside a magnetic resonance imaging scanner. The main findings are as follows: (1) cortical activation remained approximately constant throughout training, (2) subcortical activation increased with practice (i.e., there were more activated voxels in the striatum), and (3) only cortical activation was correlated with accuracy after extensive training. The results suggest an initial subcortical neural system centered around the head of the caudate that is gradually replaced by a cortical system centered around the ventrolateral prefrontal cortex. With extensive practice, the cortical system progressively becomes more caudal and dorsal, and is eventually centered around the premotor cortex.

A neurocomputational theory of how rule-guided behaviors become automatic.

This article introduces a biologically detailed computational model of how rule-guided behaviors become automatic. The model assumes that initially, rule-guided behaviors are controlled by a distributed neural network centered in the prefrontal cortex, and that in addition to initiating behavior, this network also trains a faster and more direct network that includes projections from sensory association cortex directly to rule-sensitive neurons in the premotor cortex. After much practice, the direct network is sufficient to control the behavior, without prefrontal involvement. The model is implemented as a biologically detailed neural network constructed from spiking neurons and displaying a biologically plausible form of Hebbian learning. The model successfully accounts for single-unit recordings and human behavioral data that are problematic for other models of automaticity. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

A Study of Individual Differences in Categorization with Redundancy.

Humans and other animals are constantly learning new categories and making categorization decisions in their everyday life. However, different individuals may focus on different information when learning categories, which can impact the category representation and the information that is used when making categorization decisions. This article used computational modeling of behavioral data to take a closer look at this possibility in the context of a categorization task with redundancy. Iterative decision bomid modeling and drift diffusion models were used to detect individual differences in human categorization performance. The results show that participants differ in terms of what stimulus features they learned and how they use the learned features. For example, while some participants only learn one stimulus dimension (which is sufficient for perfect accuracy), others learn both stimulus dimensions (which is not required for perfect accuracy). Among participants that learned both dimensions, some used both dimensions, while others show error and RT patterns suggesting the use of only one of the dimensions. The diversity of obtained results is problematic for existing categorization models and suggests that each categorization model may be able to account for the performance of some but not all participants.