Reinforcement signaling of punishment versus relief in fruit flies.

Painful events establish opponent memories: cues that precede pain are remembered negatively, whereas cues that follow pain, thus coinciding with relief are recalled positively. How do individual reinforcement-signaling neurons contribute to this "timing-dependent valence-reversal?" We addressed this question using an optogenetic approach in the fruit fly. Two types of fly dopaminergic neuron, each comprising just one paired cell, indeed established learned avoidance of odors that preceded their photostimulation during training, and learned approach to odors that followed the photostimulation. This is in striking parallel to punishment versus relief memories reinforced by a real noxious event. For only one of these neuron types, both effects were strong enough for further analyses. Notably, interfering with dopamine biosynthesis in these neurons partially impaired the punishing effect, but not the relieving after-effect of their photostimulation. We discuss how this finding constraints existing computational models of punishment versus relief memories and introduce a new model, which also incorporates findings from mammals. Furthermore, whether using dopaminergic neuron photostimulation or a real noxious event, more prolonged punishment led to stronger relief. This parametric feature of relief may also apply to other animals and may explain particular aspects of related behavioral dysfunction in humans.

Aversive olfactory associative memory loses odor specificity over time.

Avoiding associatively learned predictors of danger is crucial for survival. Aversive memories can, however, become counter-adaptive when they are overly generalized to harmless cues and contexts. In a fruit fly odor-electric shock associative memory paradigm, we found that learned avoidance lost its specificity for the trained odor and became general to novel odors within a day of training. We discuss the possible neural circuit mechanisms of this effect and highlight the parallelism to over-generalization of learned fear behavior after an incubation period in rodents and humans, with due relevance for post-traumatic stress disorder.

The Ol1mpiad: concordance of behavioural faculties of stage 1 and stage 3 Drosophila larvae.

Mapping brain function to brain structure is a fundamental task for neuroscience. For such an endeavour, the Drosophila larva is simple enough to be tractable, yet complex enough to be interesting. It features about 10,000 neurons and is capable of various taxes, kineses and Pavlovian conditioning. All its neurons are currently being mapped into a light-microscopical atlas, and Gal4 strains are being generated to experimentally access neurons one at a time. In addition, an electron microscopic reconstruction of its nervous system seems within reach. Notably, this electron microscope-based connectome is being drafted for a stage 1 larva - because stage 1 larvae are much smaller than stage 3 larvae. However, most behaviour analyses have been performed for stage 3 larvae because their larger size makes them easier to handle and observe. It is therefore warranted to either redo the electron microscopic reconstruction for a stage 3 larva or to survey the behavioural faculties of stage 1 larvae. We provide the latter. In a community-based approach we called the Ol1mpiad, we probed stage 1 Drosophila larvae for free locomotion, feeding, responsiveness to substrate vibration, gentle and nociceptive touch, burrowing, olfactory preference and thermotaxis, light avoidance, gustatory choice of various tastants plus odour-taste associative learning, as well as light/dark-electric shock associative learning. Quantitatively, stage 1 larvae show lower scores in most tasks, arguably because of their smaller size and lower speed. Qualitatively, however, stage 1 larvae perform strikingly similar to stage 3 larvae in almost all cases. These results bolster confidence in mapping brain structure and behaviour across developmental stages.

Independent natural genetic variation of punishment- versus relief-memory.

A painful event establishes two opponent memories: cues that are associated with pain onset are remembered negatively, whereas cues that coincide with the relief at pain offset acquire positive valence. Such punishment- versus relief-memories are conserved across species, including humans, and the balance between them is critical for adaptive behaviour with respect to pain and trauma. In the fruit fly, Drosophila melanogaster as a study case, we found that both punishment- and relief-memories display natural variation across wild-derived inbred strains, but they do not covary, suggesting a considerable level of dissociation in their genetic effectors. This provokes the question whether there may be heritable inter-individual differences in the balance between these opponent memories in man, with potential psycho-clinical implications.

Pain-relief learning in flies, rats, and man: basic research and applied perspectives.

Memories relating to a painful, negative event are adaptive and can be stored for a lifetime to support preemptive avoidance, escape, or attack behavior. However, under unfavorable circumstances such memories can become overwhelmingly powerful. They may trigger excessively negative psychological states and uncontrollable avoidance of locations, objects, or social interactions. It is therefore obvious that any process to counteract such effects will be of value. In this context, we stress from a basic-research perspective that painful, negative events are "Janus-faced" in the sense that there are actually two aspects about them that are worth remembering: What made them happen and what made them cease. We review published findings from fruit flies, rats, and man showing that both aspects, respectively related to the onset and the offset of the negative event, induce distinct and oppositely valenced memories: Stimuli experienced before an electric shock acquire negative valence as they signal upcoming punishment, whereas stimuli experienced after an electric shock acquire positive valence because of their association with the relieving cessation of pain. We discuss how memories for such punishment- and relief-learning are organized, how this organization fits into the threat-imminence model of defensive behavior, and what perspectives these considerations offer for applied psychology in the context of trauma, panic, and nonsuicidal self-injury.

Olfactory memories are intensity specific in larval Drosophila.

Learning can rely on stimulus quality, stimulus intensity, or a combination of these. Regarding olfaction, the coding of odour quality is often proposed to be combinatorial along the olfactory pathway, and working hypotheses are available concerning short-term associative memory trace formation of odour quality. However, it is less clear how odour intensity is coded, and whether olfactory memory traces include information about the intensity of the learnt odour. Using odour-sugar associative conditioning in larval Drosophila, we first describe the dose-effect curves of learnability across odour intensities for four different odours (n-amyl acetate, 3-octanol, 1-octen-3-ol and benzaldehyde). We then chose odour intensities such that larvae were trained at an intermediate odour intensity, but were tested for retention with either that trained intermediate odour intensity, or with respectively higher or lower intensities. We observed a specificity of retention for the trained intensity for all four odours used. This adds to the appreciation of the richness in 'content' of olfactory short-term memory traces, even in a system as simple as larval Drosophila, and to define the demands on computational models of associative olfactory memory trace formation. We suggest two kinds of circuit architecture that have the potential to accommodate intensity learning, and discuss how they may be implemented in the insect brain.

Memory decay and susceptibility to amnesia dissociate punishment--from relief-learning.

Painful events shape future behaviour in two ways: stimuli associated with pain onset subsequently support learned avoidance (i.e. punishment-learning) because they signal future, upcoming pain. Stimuli associated with pain offset in turn signal relief and later on support learned approach (i.e. relief-learning). The relative strengths of such punishment- and relief-learning can be crucial for the adaptive organization of behaviour in the aftermath of painful events. Using Drosophila, we compare punishment- and relief-memories in terms of their temporal decay and sensitivity to retrograde amnesia. During the first 75 min following training, relief-memory is stable, whereas punishment-memory decays to half of the initial score. By 24 h after training, however, relief-memory is lost, whereas a third of punishment-memory scores still remain. In accordance with such rapid temporal decay from 75 min on, retrograde amnesia erases relief-memory but leaves a half of punishment-memory scores intact. These findings suggest differential mechanistic bases for punishment- and relief-memory, thus offering possibilities for separately interfering with either of them.

Onset and offset of aversive events establish distinct memories requiring fear and reward networks.

Two things are worth remembering about an aversive event: What made it happen? What made it cease? If a stimulus precedes an aversive event, it becomes a signal for threat and will later elicit behavior indicating conditioned fear. However, if the stimulus is presented upon cessation of the aversive event, it elicits behavior indicating conditioned "relief." What are the neuronal bases for such learning? Using functional magnetic resonance imaging (fMRI) in humans we found that a fear-conditioned stimulus activates amygdala but not striatum, whereas a relief-conditioned stimulus activates striatum but not amygdala. Correspondingly, acute inactivation of amygdala or of ventral striatum in rats respectively abolished only conditioned fear or only conditioned relief. Thus, the behaviorally opponent memories supported by onset and offset of aversive events engage and require fear and reward networks, respectively. This may explain attraction to stimuli associated with the cessation of trauma or of panic attacks.

A rift between implicit and explicit conditioned valence in human pain relief learning.

Pain is aversive, but does the cessation of pain ('relief') have a reward-like effect? Indeed, fruitflies avoid an odour previously presented before a painful event, but approach an odour previously presented after a painful event. Thus, event-timing may turn punishment to reward. However, is event-timing also crucial in humans who can have explicit cognitions about associations? Here, we show that stimuli associated with pain-relief acquire positive implicit valence but are explicitly rated as aversive. Specifically, the startle response, an evolutionarily conserved defence reflex, is attenuated by stimuli that had previously followed a painful event, indicating implicit positive valence of the conditioned stimulus; nevertheless, participants explicitly evaluate these stimuli as 'emotionally negative'. These results demonstrate a rift between the implicit and explicit conditioned valence induced by pain relief. They might explain why humans in some cases are attracted by conditioned stimuli despite explicitly judging them as negative.

Odour intensity learning in fruit flies.

Animals' behaviour towards odours depends on both odour quality and odour intensity. While neuronal coding of odour quality is fairly well studied, how odour intensity is treated by olfactory systems is less clear. Here we study odour intensity processing at the behavioural level, using the fruit fly Drosophila melanogaster. We trained flies by pairing a MEDIUM intensity of an odour with electric shock, and then, at a following test phase, measured flies' conditioned avoidance of either this previously trained MEDIUM intensity or a LOWer or a HIGHer intensity. With respect to 3-octanol, n-amylacetate and 4-methylcyclohexanol, we found that conditioned avoidance is strongest when training and test intensities match, speaking for intensity-specific memories. With respect to a fourth odour, benzaldehyde, on the other hand, we found no such intensity specificity. These results form the basis for further studies of odour intensity processing at the behavioural, neuronal and molecular level.

Genetic distortion of the balance between punishment and relief learning in Drosophila.

An experience with electric shock can support two opposing kinds of behavioral effects: Stimuli that precede shock during training are subsequently avoided as predictors for punishment, whereas stimuli that follow shock during training are later on approached, as they predict relief. We show here, for the fruit fly Drosophila, that upon the loss of white-function, the balance between these two kinds of learning is distorted in favor of punishment learning: white1118 mutants show stronger punishment learning and weaker relief learning, as compared to wild type flies. Thus, white1118 mutants establish, overall, more "negative" memories for the shock experience. This only concerns the mnemonic effects of the shock; the immediate, reflexive responsiveness to shock remains unaltered. Also, learning about reward is apparently unaffected, both in adult and larval Drosophila. Prompted by the proposed function of the White protein as the transporter for biogenic amine precursors, we probed the brains of white1118 mutants for the amounts of biogenic amines (octopamine, tyramine, dopamine, and serotonin) by using high-pressure liquid chromatography coupled to mass spectrometry. Using this method, we found, however, no difference between white1118 and wild type files for any of the probed amines. In any event, analyses of how the white1118 mutation affects the balance between punishment and relief learning should provide a study case of how heritable distortions of such balance can come about. Finally, the effects of the white1118 mutation should be considered as a source of confound when using white as the "marker gene" in behavior-genetic analyses of any sort.

Behavioral analyses of sugar processing in choice, feeding, and learning in larval Drosophila.

Gustatory stimuli have at least 2 kinds of function: They can support immediate, reflexive responses (such as substrate choice and feeding) and they can drive internal reinforcement. We provide behavioral analyses of these functions with respect to sweet taste in larval Drosophila. The idea is to use the dose-effect characteristics as behavioral "fingerprints" to dissociate reflexive and reinforcing functions. For glucose and trehalose, we uncover relatively weak preference. In contrast, for fructose and sucrose, preference responses are strong and the effects on feeding pronounced. Specifically, larvae are attracted to, and feeding is stimulated most strongly for, intermediate concentrations of either sugar: Using very high concentrations (4 M) results in weakened preference and suppression of feeding. In contrast to such an optimum function regarding choice and feeding, an asymptotic dose-effect function is found for reinforcement learning: Learning scores reach asymptote at 2 M and remain stable for a 4-M concentration. A similar parametric discrepancy between the reflexive (choice and feeding) and reinforcing function is also seen for sodium chloride (Niewalda T, Singhal S, Fiala A, Saumweber T, Wegener S, Gerber B, in preparation). We discuss whether these discrepancies are based either on inhibition from high-osmolarity sensors upon specifically the reflexive pathways or whether different sensory pathways, with different effective dose-response characteristics, may have preferential access to drive either reflex responses or modulatory neurons mediating internal reinforcement, respectively.

Reversing Stimulus Timing in Visual Conditioning Leads to Memories with Opposite Valence in Drosophila.

Animals need to associate different environmental stimuli with each other regardless of whether they temporally overlap or not. Drosophila melanogaster displays olfactory trace conditioning, where an odor is followed by electric shock reinforcement after a temporal gap, leading to conditioned odor avoidance. Reversing the stimulus timing in olfactory conditioning results in the reversal of memory valence such that an odor that follows shock is later on approached (i.e. relief conditioning). Here, we explored the effects of stimulus timing on memory in another sensory modality, using a visual conditioning paradigm. We found that flies form visual memories of opposite valence depending on stimulus timing and can associate a visual stimulus with reinforcement despite being presented with a temporal gap. These results suggest that associative memories with non-overlapping stimuli and the effect of stimulus timing on memory valence are shared across sensory modalities.

A model for non-monotonic intensity coding.

Peripheral neurons of most sensory systems increase their response with increasing stimulus intensity. Behavioural responses, however, can be specific to some intermediate intensity level whose particular value might be innate or associatively learned. Learning such a preference requires an adjustable trans- formation from a monotonic stimulus representation at the sensory periphery to a non-monotonic representation for the motor command. How do neural systems accomplish this task? We tackle this general question focusing on odour-intensity learning in the fruit fly, whose first- and second-order olfactory neurons show monotonic stimulus-response curves. Nevertheless, flies form associative memories specific to particular trained odour intensities. Thus, downstream of the first two olfactory processing layers, odour intensity must be re-coded to enable intensity-specific associative learning. We present a minimal, feed-forward, three-layer circuit, which implements the required transformation by combining excitation, inhibition, and, as a decisive third element, homeostatic plasticity. Key features of this circuit motif are consistent with the known architecture and physiology of the fly olfactory system, whereas alternative mechanisms are either not composed of simple, scalable building blocks or not compatible with physiological observations. The simplicity of the circuit and the robustness of its function under parameter changes make this computational motif an attractive candidate for tuneable non-monotonic intensity coding.

Genome-Wide Association Analyses Point to Candidate Genes for Electric Shock Avoidance in Drosophila melanogaster.

Electric shock is a common stimulus for nociception-research and the most widely used reinforcement in aversive associative learning experiments. Yet, nothing is known about the mechanisms it recruits at the periphery. To help fill this gap, we undertook a genome-wide association analysis using 38 inbred Drosophila melanogaster strains, which avoided shock to varying extents. We identified 514 genes whose expression levels and/ or sequences co-varied with shock avoidance scores. We independently scrutinized 14 of these genes using mutants, validating the effect of 7 of them on shock avoidance. This emphasizes the value of our candidate gene list as a guide for follow-up research. In addition, by integrating our association results with external protein-protein interaction data we obtained a shock avoidance-associated network of 38 genes. Both this network and the original candidate list contained a substantial number of genes that affect mechanosensory bristles, which are hair-like organs distributed across the fly's body. These results may point to a potential role for mechanosensory bristles in shock sensation. Thus, we not only provide a first list of candidate genes for shock avoidance, but also point to an interesting new hypothesis on nociceptive mechanisms.

Event timing in associative learning: from biochemical reaction dynamics to behavioural observations.

Associative learning relies on event timing. Fruit flies for example, once trained with an odour that precedes electric shock, subsequently avoid this odour (punishment learning); if, on the other hand the odour follows the shock during training, it is approached later on (relief learning). During training, an odour-induced Ca(++) signal and a shock-induced dopaminergic signal converge in the Kenyon cells, synergistically activating a Ca(++)-calmodulin-sensitive adenylate cyclase, which likely leads to the synaptic plasticity underlying the conditioned avoidance of the odour. In Aplysia, the effect of serotonin on the corresponding adenylate cyclase is bi-directionally modulated by Ca(++), depending on the relative timing of the two inputs. Using a computational approach, we quantitatively explore this biochemical property of the adenylate cyclase and show that it can generate the effect of event timing on associative learning. We overcome the shortage of behavioural data in Aplysia and biochemical data in Drosophila by combining findings from both systems.

A Neurogenetic Dissociation between Punishment-, Reward-, and Relief-Learning in Drosophila.

What is particularly worth remembering about a traumatic experience is what brought it about, and what made it cease. For example, fruit flies avoid an odor which during training had preceded electric shock punishment; on the other hand, if the odor had followed shock during training, it is later on approached as a signal for the relieving end of shock. We provide a neurogenetic analysis of such relief learning. Blocking, using UAS-shibire(ts1), the output from a particular set of dopaminergic neurons defined by the TH-Gal4 driver partially impaired punishment learning, but left relief learning intact. Thus, with respect to these particular neurons, relief learning differs from punishment learning. Targeting another set of dopaminergic/serotonergic neurons defined by the DDC-Gal4 driver on the other hand affected neither punishment nor relief learning. As for the octopaminergic system, the tbh(M18) mutation, compromising octopamine biosynthesis, partially impaired sugar-reward learning, but not relief learning. Thus, with respect to this particular mutation, relief learning, and reward learning are dissociated. Finally, blocking output from the set of octopaminergic/tyraminergic neurons defined by the TDC2-Gal4 driver affected neither reward, nor relief learning. We conclude that regarding the used genetic tools, relief learning is neurogenetically dissociated from both punishment and reward learning. This may be a message relevant also for analyses of relief learning in other experimental systems including man.

No evidence for visual context-dependency of olfactory learning in Drosophila.

How is behaviour organised across sensory modalities? Specifically, we ask concerning the fruit fly Drosophila melanogaster how visual context affects olfactory learning and recall and whether information about visual context is getting integrated into olfactory memory. We find that changing visual context between training and test does not deteriorate olfactory memory scores, suggesting that these olfactory memories can drive behaviour despite a mismatch of visual context between training and test. Rather, both the establishment and the recall of olfactory memory are generally facilitated by light. In a follow-up experiment, we find no evidence for learning about combinations of odours and visual context as predictors for reinforcement even after explicit training in a so-called biconditional discrimination task. Thus, a 'true' interaction between visual and olfactory modalities is not evident; instead, light seems to influence olfactory learning and recall unspecifically, for example by altering motor activity, alertness or olfactory acuity.

Olfactory learning and behaviour are 'insulated' against visual processing in larval Drosophila.

We investigate the organization of behaviour across sensory modalities, using larval Drosophila melanogaster. We ask whether olfactory learning and behaviour are affected by visual processing. We find that: (1) Visual choice does not affect concomitant odour choice. (2) Visual context does not influence odour learning, nor do changes of visual context between training and test affect retrieval of odour memory. (3) Larvae cannot solve a biconditional discrimination task, despite generally permissive conditions. In this task, larvae are required to establish conditional associations: in light, one odour is rewarded and the other one is not, whereas in dark the opposite contingency is established. After such training, choice between the two odours is equal under light and dark testing conditions, suggesting that larvae do not establish odour memories specifically for one visual context only. Together, these data suggest that, in larval Drosophila, olfactory learning and behaviour are 'insulated' against visual processing.

Toll-like receptor 2 (TLR2) and TLR9 signaling results in HIV-long terminal repeat trans-activation and HIV replication in HIV-1 transgenic mouse spleen cells: implications of simultaneous activation of TLRs on HIV replication.

Opportunistic infections are common in HIV-infected patients; they activate HIV replication and contribute to disease progression. In the present study we examined the role of Toll-like receptor 2 (TLR2) and TLR9 in HIV-long terminal repeat (HIV-LTR) trans-activation and assessed whether TLR4 synergized with TLR2 or TLR9 to induce HIV replication. Soluble Mycobacterium tuberculosis factor (STF) and phenol-soluble modulin from Staphylococcus epidermidis induced HIV-LTR trans-activation in human microvessel endothelial cells cotransfected with TLR2 cDNA. Stimulation of ex vivo spleen cells from HIV-1 transgenic mice with TLR4, TLR2, and TLR9 ligands (LPS, STF, and CpG DNA, respectively) induced p24 Ag production in a dose-dependent manner. Costimulation of HIV-1 transgenic mice spleen cells with LPS and STF or CpG DNA induced TNF-alpha and IFN-gamma production in a synergistic manner and p24 production in an additive fashion. In the THP-1 human monocytic cell line stably expressing the HIV-LTR-luciferase construct, LPS and STF also induced HIV-LTR trans-activation in an additive manner. This is the first time that TLR2 and TLR9 and costimulation of TLRs have been shown to induce HIV replication. Together these results underscore the importance of TLRs in bacterial Ag- and CpG DNA-induced HIV-LTR trans-activation and HIV replication. These observations may be important in understanding the role of the innate immune system and the molecular mechanisms involved in the increased HIV replication and HIV disease progression associated with multiple opportunistic infections.