A reinforcement schedule with two types of temporal information.

Decades of research on reinforcement schedules have demonstrated that temporal information regarding the arrival or nonarrival of biologically significant events controls animal behavior. The fixed interval (FI) schedule, which is a time-based reinforcement schedule, suggests that responses are regulated by the time elapsed since the last reinforcement. This raises the question of how behavior is controlled when two distinct temporal cues regarding the availability of reinforcers are simultaneously presented. We modified the random interval (RI) schedule to create a reinforcement schedule incorporating two types of temporal information: the time elapsed since the last reinforcement, and the inter-reinforcement interval (IRI). Through long-term operant conditioning in rats, we examined how temporal information controls responses. When two temporal cues were available, we found that behavior was influenced by each cue. Moreover, we discovered that the effects of these cues could be analytically separated within the dynamics of response rates. The findings revealed that in controlling behavior, living organisms can utilize two temporal cues rather than relying on a single, clear cue.•We modified the random interval (RI) schedule to create a reinforcement schedule incorporating two types of temporal information.•When two temporal cues were available, we found that behavior was influenced by each cue.•Rats can utilize two temporal cues rather than relying on a single, clear cue.

A model for recovery-from-extinction effects in Pavlovian conditioning and exposure therapy.

Exposure therapy is an effective intervention for anxiety-related problems. The mechanism of this intervention has been the extinction procedure in Pavlovian conditioning, and this application has provided many successful instances for the prevention of relapse. However, traditional associative theories cannot comprehensively explain many findings. In particular, it is difficult to explain the recovery-from-extinction effects, which is the reappearance of the conditioned response following extinction. In this paper, we propose an associative model that is a mathematical extension of Bouton's (1993, Psychological Bulletin, 114, 80-99) model for the extinction procedure. The core of our model is that the asymptotic strength of the inhibitory association depends on the degree of excitatory association retrieved in a context in which a conditioned stimulus (CS) is presented and that the retrieval is determined by the similarity between contexts during both reinforcement and non-reinforcement and the retrieval context. Our model provides an explanation of the recovery-from-extinction effects, and implications for exposure therapy.

Intraday Activity Levels May Better Reflect the Differences Between Major Depressive Disorder and Bipolar Disorder Than Average Daily Activity Levels.

It is important to establish an objective index to differentiate mood disorders (i.e., bipolar disorder; BD and major depressive disorder; MDD). The present study focused on the pattern of changes of physical activity in the amount of activity intraday, and examined the relationship between activity patterns and mood disorders. One hundred and eighteen inpatients with MDD or BD in a depressive state provided the activity data by using wearable activity trackers for 3 weeks. In order to illuminate the characteristic patterns of intraday activities, Principal Component Analysis (PCA) was adopted to extract the main components of intraday activity changes. We found that some of the PCs reflected the differences between the types of mood disorder. BD participants showed high activity pattern in the morning and low activity pattern in evenings. However, MDD showed the opposite. Our results suggest that activity tracking focused on daytime activity patterns may provide objective auxiliary diagnostic information.

Use of redundant sets of landmark information by humans (Homo sapiens) in a goal-searching task in an open field and on a computer screen.

Landmark-based goal-searching tasks that were similar to those for pigeons (Ushitani & Jitsumori, 2011) were provided to human participants to investigate whether they could learn and use multiple sources of spatial information that redundantly indicate the position of a hidden target in both an open field (Experiment 1) and on a computer screen (Experiments 2 and 3). During the training in each experiment, participants learned to locate a target in 1 of 25 objects arranged in a 5 × 5 grid, using two differently colored, arrow-shaped (Experiments 1 and 2) or asymmetrically shaped (Experiment 3) landmarks placed adjacent to the goal and pointing to the goal location. The absolute location and directions of the landmarks varied across trials, but the constant configuration of the goal and the landmarks enabled participants to find the goal using both global configural information and local vector information (pointing to the goal by each individual landmark). On subsequent test trials, the direction was changed for one of the landmarks to conflict with the global configural information. Results of Experiment 1 indicated that participants used vector information from a single landmark but not configural information. Further examinations revealed that the use of global (metric) information was enhanced remarkably by goal searching with nonarrow-shaped landmarks on the computer monitor (Experiment 3) but much less so with arrow-shaped landmarks (Experiment 2). The General Discussion focuses on a comparison between humans in the current study and pigeons in the previous study. (PsycINFO Database Record

The effect of temporal information among events on Bayesian causal inference in rats.

A temporal relationship between events of potential cause and effect is critical to generate a causal relationship because the cause has to be followed by the effect. The present study investigated the role of temporal relationships between events in causal inference in rats via Pavlovian pairings. In Experiment 1A, subjects in Group Successive received training trials whereby Event 1 (tone or light) was followed by Events 2 (light or tone) and 3 (sucrose solution), whereas those in Group Simultaneous received simultaneous pairings of Events 1 and 2, and Events 1 and 3. During testing, a lever was inserted into the experimental chamber, where subjects were allowed to press the lever which produced the occurrence of Event 2 without reward. By measuring nose-poke responses during the presentation of Event 2, assumingly based on the prediction of occurrence of sucrose solution, subjects in Group Successive showed a relatively lower response rate than did those in Group Simultaneous. In Experiment 1B, this difference was not observed if subjects received the presentations of Event 2 which was irrelevant to their lever pressing during testing. These results suggest that rats can differentiate their response based on the elemental temporal information even when the integrated temporal map was the same, and implied that rats use temporal information as well as conditional probability based on causal Bayesian network account.

[The effect of interpersonal dependency on judgment of gaze direction].

This study investigated the effect of interpersonal dependency on judgments of gaze direction of individuals with different facial expressions. Based on interpersonal dependency scores, 46 participants were divided into two groups (high interpersonal dependency and low interpersonal dependency). Participants judged the gaze direction of photographs of faces with angry, neutral or happy expressions. Relative to the low interpersonal dependency group, the high interpersonal dependency group was more accurate in the judgments of gaze direction. This tendency was more salient for the happy and neutral expressions than for the angry expressions. Since people with high interpersonal dependency are highly motivated to seek support from others, this result suggests that they are sensitive to signals with pro-social information such as the gaze direction of others with positive attitudes.

Reward inference by primate prefrontal and striatal neurons.

The brain contains multiple yet distinct systems involved in reward prediction. To understand the nature of these processes, we recorded single-unit activity from the lateral prefrontal cortex (LPFC) and the striatum in monkeys performing a reward inference task using an asymmetric reward schedule. We found that neurons both in the LPFC and in the striatum predicted reward values for stimuli that had been previously well experienced with set reward quantities in the asymmetric reward task. Importantly, these LPFC neurons could predict the reward value of a stimulus using transitive inference even when the monkeys had not yet learned the stimulus-reward association directly; whereas these striatal neurons did not show such an ability. Nevertheless, because there were two set amounts of reward (large and small), the selected striatal neurons were able to exclusively infer the reward value (e.g., large) of one novel stimulus from a pair after directly experiencing the alternative stimulus with the other reward value (e.g., small). Our results suggest that although neurons that predict reward value for old stimuli in the LPFC could also do so for new stimuli via transitive inference, those in the striatum could only predict reward for new stimuli via exclusive inference. Moreover, the striatum showed more complex functions than was surmised previously for model-free learning.

Effects of extended context discrimination training and context extinction on transfer of context dependency of conditioned flavor aversion.

We trained rats in a context discrimination paradigm by pairing a sucrose solution with lithium chloride in one context (conditioning context) and simple exposure to the same fluid in a second (neutral) context to establish a context-dependent aversion to the conditioned fluid. We then investigated whether transfer of the context dependency to a test fluid (a sodium chloride solution) was affected by two post-discrimination training treatments, an extended context discrimination training, and non-reinforced exposure to the conditioning context (context extinction). We found that the context-dependent flavor aversion that had been specific to sucrose transferred to the test fluid after the extensive training (Experiment 1). Context extinction eliminated the transfer effect that had been observed immediately after the context discrimination training (Experiment 2). In addition, an aversion acquired by sucrose through a simple conditioning of sucrose-LiCl pairings did not generalize to the test fluid (Experiment 3). These results emphasize the importance of a Pavlovian excitatory association between the conditioning context and nausea as a primary source of transfer of the context dependency, rather than a generalization of aversion acquired by the conditioned fluid to the test fluid.

The diurnal variation of performance of the novel location recognition task in male rats.

Circadian changes of performance have been reported in various kinds of learning task. However, the diurnal variation of performance in hippocampus-dependent learning tasks remains unclear. In the present study, rats were subjected to the novel location recognition (NLR) task as well as the novel object recognition (NOR) task to examine whether the circadian pattern of hippocampus-dependent task performance was similar to that in tasks in which brain regions other than the hippocampus contribute. The performance in the NOR task was relatively constant irrespective of the time of day, while the performance in the NLR task was higher at night than during the daytime. When the pineal hormone melatonin was injected into rats before the training phase in order to examine its effects on the pattern of circadian changes of NLR performance, rats showed improvement of performance in the daytime, but impairment at night. These results suggest that the pattern of circadian variation of memory performance depends on the type of task, and that the effects of exogenous melatonin on learning performance vary with the time of day.

Conditioned Flavor Preference and the US Postexposure Effect in the House Musk Shrew (Suncus Murinus).

The house musk shrew (Suncus murinus) is the only species of mammalian insectivore that can be domesticated and used as a laboratory animal, and is an interesting subject in terms of evolutionary and comparative aspects. The present study on the learning faculties of shrews examines the possibility of acquiring a conditioned flavor preference and the effects of US postexposure. Subjects were allowed to a drink sucrose solution with flavor A and tap water with flavor B during training. Two extinction tests were administered after every four conditioning trials, and a significant preference for flavor A was observed. After each test, the animals were divided into two groups. Subjects in Group US were presented with a sucrose solution without flavor, while those in Group Water were given tap water. After these trials, all subjects received choice tests where they were presented with water containing the two flavors. The preference ratio was lower in Group US than in Group Water, suggesting a postexposure effect. The findings were discussed in terms of habituation to the US.

Factors that influence negative summation in a spatial-search task with pigeons.

A variant of the standard conditioned inhibition procedure was used to evaluate landmark-based spatial search in a touchscreen preparation. Pigeons were given compound trials with one landmark (A) positioned in a consistent spatial relationship to a hidden goal and another landmark (B) positioned randomly with respect to A and the hidden goal (AB+). On half of the non-reinforced inhibitory trials, A was paired with landmark X (AX-) and on the remaining trials B was paired with Y (BY-). All subjects were also given reinforced trials with a transfer excitor (T+). During conditioned inhibition training, subjects showed no change in overall responding during AX- trials but did show a decrease in the number of pecks to the goal location signaled by A. During non-reinforced summation tests with landmark T, X had a greater suppressive effect than did Y on overall responding but the percentage of pecks at the goal did not differ unless X was positioned near the expected goal signaled by T. These data demonstrate that the effectiveness of a stimulus trained as an inhibitor is dependent on the strength of the association between its training excitor (A) and the US, as well as, the spatial arrangement of stimuli during testing.

Context and the renewal of conditioned taste aversion: the role of rat dorsal hippocampus examined by electrolytic lesion.

An extinguished conditioned response can sometimes be restored. Previous research has shown that this renewal effect depends on the context in which conditioning versus extinction takes place. Here we provide evidence that the dorsal hippocampus is critically involved in the representation of context that underscores the renewal effect. We performed electrolytic lesions in dorsal hippocampus, before or after extinction, in a conditioned taste aversion paradigm with rats. Rats that underwent all conditioning, extinction and testing procedures in the same experimental context showed no renewal during testing in the original context. In contrast, rats that underwent extinction procedures in a different experimental context than the one in which they had acquired the conditioned response, showed a reliable renewal effect during testing in the original context. When electrolytic lesion was performed prior to extinction, the context-dependent renewal effect was disrupted. When electrolytic lesion was undertaken after extinction, we observed a complex pattern of data including the blockage of the conventional renewal effect, and the appearance of an unconventional renewal effect. The implications of these results are discussed with respect to current views on the role of the dorsal hippocampus in processing context information.

Reward prediction based on stimulus categorization in primate lateral prefrontal cortex.

To adapt to changeable or unfamiliar environments, it is important that animals develop strategies for goal-directed behaviors that meet the new challenges. We used a sequential paired-association task with asymmetric reward schedule to investigate how prefrontal neurons integrate multiple already-acquired associations to predict reward. Two types of reward-related neurons were observed in the lateral prefrontal cortex: one type predicted reward independent of physical properties of visual stimuli and the other encoded the reward value specific to a category of stimuli defined by the task requirements. Neurons of the latter type were able to predict reward on the basis of stimuli that had not yet been associated with reward, provided that another stimulus from the same category was paired with reward. The results suggest that prefrontal neurons can represent reward information on the basis of category and propagate this information to category members that have not been linked directly with any experience of reward.

Temporal integration in Pavlovian appetitive conditioning in rats.

We used an appetitive sensory preconditioning procedure to investigate temporal integration in rats in two experiments. In Phase 1, rats were presented with simultaneous compound trials on which a 10-sec conditioned stimulus (CS) X was embedded within a 60-sec CS A. In Group Early, CS X occurred during the early portion of CS A, whereas in Group Late, CS X occurred during the latter portion of CS A. In Phase 2, CS X was paired simultaneously with sucrose. On a subsequent test with CS A, the rate of magazine entries peaked during the early portions of the stimulus in Group Early and during the latter portions of the stimulus in Group Late (Experiments 1 and 2). Similar response peaks were not observed on tests with a control stimulus that had been presented in compound with a stimulus that did not signal reward (Experiment 2).

Causal reasoning in rats.

Empirical research with nonhuman primates appears to support the view that causal reasoning is a key cognitive faculty that divides humans from animals. The claim is that animals approximate causal learning using associative processes. The present results cast doubt on that conclusion. Rats made causal inferences in a basic task that taps into core features of causal reasoning without requiring complex physical knowledge. They derived predictions of the outcomes of interventions after passive observational learning of different kinds of causal models. These competencies cannot be explained by current associative theories but are consistent with causal Bayes net theories.

Sensory preconditioning in spatial learning using a touch screen task in pigeons.

The authors used a touch screen-based visual-search task to investigate spatial integration in pigeons. First, pigeons were presented with a consistent spatial relationship between compound visual landmarks (LMs) A-X and B-Y, separately. Next, pigeons learned to find a hidden goal on the monitor in the presence of LMs A and B. The goal bore a consistent spatial relationship to LM A, but not to LM B. On nonreinforced probe tests, the peak and distribution of responses to LM X suggest that pigeons computed a novel X-goal spatial relationship on the basis of X-A and A-goal spatial vectors. Responses to LM Y, however, revealed no evidence of spatial integration. These results replicate and extend those of A. P. Blaisdell and R. G. Cook (2005) using an open-field task.

Acquired equivalence of flavour cues with a common antecedent in rats.

Equivalence of flavour cues, each of which had been paired with a common antecedent, was demonstrated with rats in a three-stage design. In the first stage, a group of thirsty rats were given each of two target flavour cues after a common antecedent flavour (Xright arrowA and Xright arrowB), while a second group of rats were given A and B after differential antecedent flavours (Xright arrowA and Yright arrowB). Another group of rats was allowed to drink A and B after familiar tap water. In the second stage, aversion to A was established by a lithium chloride injection after drinking A. The acquired equivalence effect was verified in the third stage by strong aversion to B in the group trained with the common antecedent compared with the remaining two groups. The representation-mediation, rather than response-mediation, hypothesis seems to fit the backward acquired equivalence effect obtained here.