Dual-system free-operant avoidance: Extension of a theory.

Our theory of positively reinforced free-operant behavior (Perez & Dickinson, 2020) assumes that responding is controlled by two systems. One system is sensitive to the correlation between response and reinforcement rates and controls goal-directed behavior, whereas a habitual system learns by reward prediction error. We present an extension of this theory to the aversive domain that explains why free-operant avoidance responding increases with both the experienced rate of negative reinforcement and the difference between this rate and that programmed by the avoidance schedule. The theory also assumes that the habitual component is reinforced by the acquisition of aversive inhibitory properties by the feedback stimuli generated by responding, which then act as safety signals that reinforce habit performance. Our analysis suggests that the distinction between habitual and goal-directed control of rewarded behavior can also be applied to the aversive domain. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Why is the Rescorla-Wagner model so influential?

The influence of the Rescorla-Wagner model cannot be overestimated, despite that (1) the model does not differ much computationally from its predecessors and competitors, and (2) its shortcomings are well-known in the learning community. Here we discuss the reasons behind its widespread influence in the cognitive and neural sciences, and argue that it is the constant search for general-process theories by learning scholars which eventually produced a model whose application spans many different areas of research to this day. We focus on the theoretical and empirical background of the model, the theoretical connections that it has with later developments across Marr's levels of analysis, as well as the broad variety of research that it has guided and inspired.

Action sequences, habits, and attention in copying strategies.

Understanding how culture evolves in society is an extremely difficult task. The bifocal stance theory (BST) deploys two copying strategies which can be linked to dual-system theories of behavior. BST would benefit from incorporating results from these theories, such as the evolution of attention to goals or steps of a behavioral sequence, and the role of the environment in prompting different copying strategies.

Ambiguity drives higher-order Pavlovian learning.

In the natural world, stimulus-outcome associations are often ambiguous, and most associations are highly complex and situation-dependent. Learning to disambiguate these complex associations to identify which specific outcomes will occur in which situations is critical for survival. Pavlovian occasion setters are stimuli that determine whether other stimuli will result in a specific outcome. Occasion setting is a well-established phenomenon, but very little investigation has been conducted on how occasion setters are disambiguated when they themselves are ambiguous (i.e., when they do not consistently signal whether another stimulus will be reinforced). In two preregistered studies, we investigated the role of higher-order Pavlovian occasion setting in humans. We developed and tested the first computational model predicting direct associative learning, traditional occasion setting (i.e., 1st-order occasion setting), and 2nd-order occasion setting. This model operationalizes stimulus ambiguity as a mechanism to engage in higher-order Pavlovian learning. Both behavioral and computational modeling results suggest that 2nd-order occasion setting was learned, as evidenced by lack and presence of transfer of occasion setting properties when expected and the superior fit of our 2nd-order occasion setting model compared to the 1st-order occasion setting or direct associations models. These results provide a controlled investigation into highly complex associative learning and may ultimately lead to improvements in the treatment of Pavlovian-based mental health disorders (e.g., anxiety disorders, substance use).

Subsampling of cues in associative learning.

Theories of learning distinguish between elemental and configural stimulus processing depending on whether stimuli are processed independently or as whole configurations. Evidence for elemental processing comes from findings of summation in animals where a compound of two dissimilar stimuli is deemed to be more predictive than each stimulus alone, whereas configural processing is supported by experiments using similar stimuli in which summation is not found. However, in humans the summation effect is robust and impervious to similarity manipulations. In three experiments in human predictive learning, we show that summation can be obliterated when partially reinforced cues are added to the summands in training and tests. This lack of summation only holds when the partially reinforced cues are similar to the reinforced cues (experiment 1) and seems to depend on participants sampling only the most salient cue in each trial (experiments 2a and 2b) in a sequential visual search process. Instead of attributing our and others' instances of lack of summation to the customary idea of configural processing, we offer a formal subsampling rule that might be applied to situations in which the stimuli are hard to parse from each other.

Determining the effects of training duration on the behavioral expression of habitual control in humans: a multilaboratory investigation.

It has been suggested that there are two distinct and parallel mechanisms for controlling instrumental behavior in mammals: goal-directed actions and habits. To gain an understanding of how these two systems interact to control behavior, it is essential to characterize the mechanisms by which the balance between these systems is influenced by experience. Studies in rodents have shown that the amount of training governs the relative expression of these two systems: Behavior is goal-directed following moderate training, but the more extensively an instrumental action is trained, the more it becomes habitual. It is less clear whether humans exhibit similar training effects on the expression of goal-directed and habitual behavior, as human studies have reported contradictory findings. To tackle these contradictory findings, we formed a consortium, where four laboratories undertook a preregistered experimental induction of habits by manipulating the amount of training. There was no statistical evidence for a main effect of the amount of training on the formation and expression of habits. However, exploratory analyses suggest a moderating effect of the affective component of stress on the impact of training over habit expression. Participants who were lower in affective stress appeared to be initially goal-directed, but became habitual with increased training, whereas participants who were high in affective stress were already habitual even after moderate training, thereby manifesting insensitivity to overtraining effects. Our findings highlight the importance of the role of moderating variables such as individual differences in stress and anxiety when studying the experimental induction of habits in humans.

Pavlovian occasion setting in human fear and appetitive conditioning: Effects of trait anxiety and trait depression.

Contexts and discrete stimuli often hierarchically influence the association between a stimulus and outcome. This phenomenon, called occasion setting, is central to modulation-based Pavlovian learning. We conducted two experiments with humans in fear and appetitive conditioning paradigms, training stimuli in differential conditioning, feature-positive discriminations, and feature-negative discriminations. We also investigated the effects of trait anxiety and trait depression on these forms of learning. Results from both experiments showed that participants were able to successfully learn which stimuli predicted the electric shock and monetary reward outcomes. Additionally, as hypothesized, the stimuli trained as occasion setters had little-to-no effect on simple reinforced or non-reinforced stimuli, suggesting the former were indeed occasion setters. Lastly, in fear conditioning, trait anxiety was associated with increases in fear of occasion setter/conditional stimulus compounds; in appetitive conditioning, trait depression was associated with lower expectations of monetary reward for the trained negative occasion setting compound and transfer of the negative occasion setter to the simple reinforced stimulus. These results suggest that clinically anxious individuals may have enhanced fear of occasion setting compounds, and clinically depressed individuals may expect less reward with compounds involving the negative occasion setter.

Instrumental behavior in humans is sensitive to the correlation between response rate and reward rate.

Recent theories of instrumental behavior postulate that the correlation between response and reward rate is a critical factor in instrumental goal-directed performance. However, it is still not clear whether human actions can be sensitive to rate correlation. Using a novel within-subject design, participants were trained under ratio and interval contingencies of reinforcement matching both reward probabilities and reward rates between conditions. The impact of rate correlation on performance was evident in the higher performance observed under ratio contingencies for both types of matching. Moreover, there was no difference in performance between two classes of interval schedules with equivalent correlational properties but different reward probabilities. These results are discussed in terms of a recent dual-system model of instrumental behavior.

A theory of actions and habits: The interaction of rate correlation and contiguity systems in free-operant behavior.

Contemporary theories of instrumental performance assume that responding can be controlled by 2 behavioral systems, 1 goal-directed that encodes the outcome of an action, and 1 habitual that reinforces the response strength of the same action. Here we present a model of free-operant behavior in which goal-directed control is determined by the correlation between the rates of the action and the outcome whereas the total prediction error generated by contiguous reinforcement by the outcome controls habitual response strength. The outputs of these two systems summate to generate a total response strength. This cooperative model addresses the difference in the behavioral impact of ratio and interval schedules, the transition from goal-directed to habitual control with extended training, the persistence of goal-directed control under choice procedures and following extinction, among other phenomena. In these respects, this dual-system model is unique in its account of free-operant behavior. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Evidence for a dissociation between causal beliefs and instrumental actions.

Human experiments have demonstrated that instrumental performance of an action and the causal beliefs of the effectiveness of an action in producing a reward are correlated and controlled by the probability of an action leading to a reward. The animal literature, however, shows that instrumental performance under free-operant training differs even when the reward probabilities are matched while subjects undergo training under ratio or interval schedules of reward. In two experiments, we investigated whether causal beliefs would correlate with instrumental performance under ratio and interval schedules for matched reward probabilities. In both experiments, we found that performance was higher under ratio than under interval training. However, causal beliefs were similar between these two conditions despite these differences in instrumental performance. When reward probabilities were increased by experimental manipulations in Experiment 2, the causal beliefs increased but performance decreased with respect to Experiment 1. This is evidence that instrumental performance and causal action-reward attribution may not follow from the same psychological process under free-operant training.

Human instrumental performance in ratio and interval contingencies: A challenge for associative theory.

Associative learning theories regard the probability of reinforcement as the critical factor determining responding. However, the role of this factor in instrumental conditioning is not completely clear. In fact, free-operant experiments show that participants respond at a higher rate on variable ratio than on variable interval schedules even though the reinforcement probability is matched between the schedules. This difference has been attributed to the differential reinforcement of long inter-response times (IRTs) by interval schedules, which acts to slow responding. In the present study, we used a novel experimental design to investigate human responding under random ratio (RR) and regulated probability interval (RPI) schedules, a type of interval schedule that sets a reinforcement probability independently of the IRT duration. Participants responded on each type of schedule before a final choice test in which they distributed responding between two schedules similar to those experienced during training. Although response rates did not differ during training, the participants responded at a lower rate on the RPI schedule than on the matched RR schedule during the choice test. This preference cannot be attributed to a higher probability of reinforcement for long IRTs and questions the idea that similar associative processes underlie classical and instrumental conditioning.

A re-examination of responding on ratio and regulated-probability interval schedules.

The higher response rates observed on ratio than on matched interval reward schedules has been attributed to the differential reinforcement of longer inter-response times (IRTs) on the interval contingency. Some data, however, seem to contradict this hypothesis, showing that the difference is still observed when the role of IRT reinforcement is neutralized by using a regulated-probability interval schedule (RPI). Given the mixed evidence for these predictions, we re-examined this hypothesis by training three groups of rats to lever press under ratio, interval and RPI schedules across two phases while matching reward rates within triads. At the end of the first phase, the master ratio and RPI groups responded at similar rates. In the second phase, an interval group yoked to the same master ratio group of the first phase responded at a lower rate than the RPI group. Post-hoc analysis showed comparable reward rates for master and yoked schedules. The experienced response-outcome rate correlations were likewise similar and approached zero as training progressed. We discuss these results in terms of a contemporary dual-system model of instrumental conditioning.

Exploring the Effect of Stimulus Similarity on the Summation Effect in Causal Learning.

Several contemporary models anticipate that the summation effect is modulated by the similarity between the cues forming a compound. Here, we explore this hypothesis in a series of causal learning experiments. Participants were presented with two visual cues that separately predicted a common outcome and later asked for the outcome predicted by the compound of the two cues. Similarity was varied between groups through changes in shape, spatial position, color, configuration, and rotation. In variance with the predictions of these models, we observed similar and strong levels of summation in both groups across all manipulations of similarity. The effect, however, was significantly reduced by manipulations intended to impact assumptions about the causal independence of the cues forming the compound, but this reduction was independent of stimulus similarity. These results are problematic for similarity-based models and can be more readily explained by rational approaches to causal learning.

Multiplexed transcriptome analysis to detect ALK, ROS1 and RET rearrangements in lung cancer.

ALK, ROS1 and RET gene fusions are important predictive biomarkers for tyrosine kinase inhibitors in lung cancer. Currently, the gold standard method for gene fusion detection is Fluorescence In Situ Hybridization (FISH) and while highly sensitive and specific, it is also labour intensive, subjective in analysis, and unable to screen a large numbers of gene fusions. Recent developments in high-throughput transcriptome-based methods may provide a suitable alternative to FISH as they are compatible with multiplexing and diagnostic workflows. However, the concordance between these different methods compared with FISH has not been evaluated. In this study we compared the results from three transcriptome-based platforms (Nanostring Elements, Agena LungFusion panel and ThermoFisher NGS fusion panel) to those obtained from ALK, ROS1 and RET FISH on 51 clinical specimens. Overall agreement of results ranged from 86-96% depending on the platform used. While all platforms were highly sensitive, both the Agena panel and Thermo Fisher NGS fusion panel reported minor fusions that were not detectable by FISH. Our proof-of-principle study illustrates that transcriptome-based analyses are sensitive and robust methods for detecting actionable gene fusions in lung cancer and could provide a robust alternative to FISH testing in the diagnostic setting.

Design and selection of Toca 511 for clinical use: modified retroviral replicating vector with improved stability and gene expression.

Retroviral replicating vectors (RRVs) are a nonlytic alternative to oncolytic replicating viruses as anticancer agents, being selective both for dividing cells and for cells that have defects in innate immunity and interferon responsiveness. Tumor cells fit both these descriptions. Previous publications have described a prototype based on an amphotropic murine leukemia virus (MLV), encoding yeast cytosine deaminase (CD) that converts the prodrug 5-fluorocytosine (5-FC) to the potent anticancer drug, 5-fluorouracil (5-FU) in an infected tumor. We report here the selection of one lead clinical candidate based on a general design goal to optimize the genetic stability of the virus and the CD activity produced by the delivered transgene. Vectors were tested for titer, genetic stability, CD protein and enzyme activity, ability to confer susceptibility to 5-FC, and preliminary in vivo antitumor activity and stability. One vector, Toca 511, (aka T5.0002) encoding an optimized CD, shows a threefold increased specific activity in infected cells over infection with the prototype RRV and shows markedly higher genetic stability. Animal testing demonstrated that Toca 511 replicates stably in human tumor xenografts and, after 5-FC administration, causes complete regression of such xenografts. Toca 511 (vocimagene amiretrorepvec) has been taken forward to preclinical and clinical trials.

Brain tumor eradication and prolonged survival from intratumoral conversion of 5-fluorocytosine to 5-fluorouracil using a nonlytic retroviral replicating vector.

Patients with the most common and aggressive form of high-grade glioma, glioblastoma multiforme, have poor prognosis and few treatment options. In 2 immunocompetent mouse brain tumor models (CT26-BALB/c and Tu-2449-B6C3F1), we showed that a nonlytic retroviral replicating vector (Toca 511) stably delivers an optimized cytosine deaminase prodrug activating gene to the tumor lesion and leads to long-term survival after treatment with 5-fluorocytosine (5-FC). Survival benefit is dose dependent for both vector and 5-FC, and as few as 4 cycles of 5-FC dosing after Toca 511 therapy provides significant survival advantage. In the virally permissive CT26-BALB/c model, spread of Toca 511 to other tissues, particularly lymphoid tissues, is detectable by polymerase chain reaction (PCR) over a wide range of levels. In the Tu-2449-B6C3F1 model, Toca 511 PCR signal in nontumor tissues is much lower, spread is not always observed, and when observed, is mainly detected in lymphoid tissues at low levels. The difference in vector genome spread correlates with a more effective antiviral restriction element, APOBEC3, present in the B6C3F1 mice. Despite these differences, neither strain showed signs of treatment-related toxicity. These data support the concept that, in immunocompetent animals, a replicating retroviral vector carrying a prodrug activating gene (Toca 511) can spread through a tumor mass, leading to selective elimination of the tumor after prodrug administration, without local or systemic pathology. This concept is under investigation in an ongoing phase I/II clinical trial of Toca 511 in combination with 5-FC in patients with recurrent high-grade glioma (www.clinicaltrials.gov NCT01156584).

Fibrocyte activation in rheumatoid arthritis.

OBJECTIVES: RA is a common, relapsing autoimmune disease primarily affecting the joints. Fibroblast-like synovial (FLS) cells are thought to be responsible for pannus formation and secretion of factors that recruit leucocytes to affected joints, thereby promoting bone and cartilage destruction. Fibrocytes are multipotent circulating stem cells that may have a role in RA pathogenesis, perhaps as the precursors of the FLS cells, or by regulating FLS cell function. METHODS: We utilized multidimensional phospho-specific flow cytometry to characterize the activation status of peripheral blood (PB) fibrocytes derived from human RA patients at different stages of disease and from mice with CIA. RESULTS: Human PB fibrocytes from RA patients exhibited phosporylation activation of the p44/42 and p38 MAP kinases (MAPKs), and STAT3 (signal transducer and activator of transcription) and STAT-5 early in disease, within the first year of diagnosis. Similarly, in murine CIA, an increase in the total number of PB phosphoSTAT5-positive fibrocytes was observed at early time points in disease. Notably, in the affected paws of mice with CIA, we identified an increased number of fibrocytes, in contrast to the paws of control mice. CONCLUSIONS: These data suggest that activated fibrocytes may influence the disease process in RA and may serve as surrogate markers for disease in the PB of affected patients.

Multiparameter phospho-flow analysis of lymphocytes in early rheumatoid arthritis: implications for diagnosis and monitoring drug therapy.

BACKGROUND: The precise mechanisms involved in the initiation and progression of rheumatoid arthritis (RA) are not known. Early stages of RA often have non-specific symptoms, delaying diagnosis and therapy. Additionally, there are currently no established means to predict clinical responsiveness to therapy. Immune cell activation is a critical component therefore we examined the cellular activation of peripheral blood mononuclear cells (PBMCs) in the early stages of RA, in order to develop a novel diagnostic modality. METHODS AND FINDINGS: PBMCs were isolated from individuals diagnosed with early RA (ERA) (n = 38), longstanding RA (n = 10), osteoarthritis (OA) (n = 19) and from healthy individuals (n = 10). PBMCs were examined for activation of 15 signaling effectors, using phosphorylation status as a measure of activation in immunophenotyped cells, by flow cytometry (phospho-flow). CD3+CD4+, CD3+CD8+ and CD20+ cells isolated from patients with ERA, RA and OA exhibited activation of multiple phospho-epitopes. ERA patient PBMCs showed a bias towards phosphorylation-activation in the CD4+ and CD20+ compartments compared to OA PBMCs, where phospho-activation was primarily observed in CD8+ cells. The ratio of phospho (p)-AKT/p-p38 was significantly elevated in patients with ERA and may have diagnostic potential. The mean fluorescent intensity (MFI) levels for p-AKT and p-H3 in CD4+, CD8+ and CD20+ T cells correlated directly with physician global assessment scores (MDGA) and DAS (disease activity score). Stratification by medications revealed that patients receiving leflunomide, systemic steroids or anti-TNF therapy had significant reductions in phospho-specific activation compared with patients not receiving these therapies. Correlative trends between medication-associated reductions in the levels of phosphorylation of specific signaling effectors and lower disease activity were observed. CONCLUSIONS: Phospho-flow analysis identified phosphorylation-activation of specific signaling effectors in the PB from patients with ERA. Notably, phosphorylation of these signaling effectors did not distinguish ERA from late RA, suggesting that the activation status of discrete cell populations is already established early in disease. However, when the ratio of MFI values for p-AKT and p-p38 is >1.5, there is a high likelihood of having a diagnosis of RA. Our results suggest that longitudinal sampling of patients undergoing therapy may result in phospho-signatures that are predictive of drug responsiveness.

Genomic and proteomic analysis reveals a threshold level of MYC required for tumor maintenance.

MYC overexpression has been implicated in the pathogenesis of most types of human cancers. MYC is likely to contribute to tumorigenesis by its effects on global gene expression. Previously, we have shown that the loss of MYC overexpression is sufficient to reverse tumorigenesis. Here, we show that there is a precise threshold level of MYC expression required for maintaining the tumor phenotype, whereupon there is a switch from a gene expression program of proliferation to a state of proliferative arrest and apoptosis. Oligonucleotide microarray analysis and quantitative PCR were used to identify changes in expression in 3,921 genes, of which 2,348 were down-regulated and 1,573 were up-regulated. Critical changes in gene expression occurred at or near the MYC threshold, including genes implicated in the regulation of the G(1)-S and G(2)-M cell cycle checkpoints and death receptor/apoptosis signaling. Using two-dimensional protein analysis followed by mass spectrometry, phospho-flow fluorescence-activated cell sorting, and antibody arrays, we also identified changes at the protein level that contributed to MYC-dependent tumor regression. Proteins involved in mRNA translation decreased below threshold levels of MYC. Thus, at the MYC threshold, there is a loss of its ability to maintain tumorigenesis, with associated shifts in gene and protein expression that reestablish cell cycle checkpoints, halt protein translation, and promote apoptosis.

Inhibition of HMGcoA reductase by atorvastatin prevents and reverses MYC-induced lymphomagenesis.

Statins are a class of drugs that inhibit 3-hydroxy-3-methylglutaryl-coenzyme A (HMGcoA) reductase, a critical enzyme in the mevalonate pathway. Several reports document that statins may prevent different human cancers. However, whether or not statins can prevent cancer is controversial due to discordant results. One possible explanation for these conflicting conclusions is that only some tumors or specific statins may be effective. Here, we demonstrate in an in vivo transgenic model in which atorvastatin reverses and prevents the onset of MYC-induced lymphomagenesis, but fails to reverse or prevent tumorigenesis in the presence of constitutively activated K-Ras (G12D). Using phosphoprotein fluorescence-activated cell sorter (FACS) analysis, atorvastatin treatment was found to result in the inactivation of the Ras and ERK1/2 signaling pathways associated with the dephosphorylation and inactivation of MYC. Correspondingly, tumors with a constitutively activated K-Ras (G12D) did not exhibit dephosphorylation of ERK1/2 and MYC. Atorvastatin's effects on MYC were specific to the inhibition of HMGcoA reductase, as treatment with mevalonate, the product of HMG-CoA reductase activity, abrogated these effects and inhibited the ability of atorvastatin to reverse or suppress tumorigenesis. Also, RNAi directed at HMGcoA reductase was sufficient to abrogate the neoplastic properties of MYC-induced tumors. Thus, atorvastatin, by inhibiting HMGcoA reductase, induces changes in phosphoprotein signaling that in turn prevent MYC-induced lymphomagenesis.