From significance testing to estimation and Open Science: How esci can help.

We argue that researchers should test less, estimate more, and adopt Open Science practices. We outline some of the flaws of null hypothesis significance testing and take three approaches to demonstrating the unreliability of the p value. We explain some advantages of estimation and meta-analysis ("the new statistics"), especially as contributions to Open Science practices, which aim to increase the openness, integrity, and replicability of research. We then describe esci (estimation statistics with confidence intervals): a set of online simulations and an R package for estimation that integrates into jamovi and JASP. This software provides (a) online activities to sharpen understanding of statistical concepts (e.g., "The Dance of the Means"); (b) effects sizes and confidence intervals for a range of study designs, largely by using techniques recently developed by Bonett; (c) publication-ready visualisations that make uncertainty salient; and (d) the option to conduct strong, fair hypothesis evaluation through specification of an interval null. Although developed specifically to support undergraduate learning through the 2nd edition of our textbook, esci should prove a valuable tool for graduate students and researchers interested in adopting the estimation approach. Further information is at ( https://thenewstatistics.com).

Better Inference in Neuroscience: Test Less, Estimate More.

Null-hypothesis significance testing (NHST) has become the main tool of inference in neuroscience, and yet evidence suggests we do not use this tool well: tests are often planned poorly, conducted unfairly, and interpreted invalidly. This editorial makes the case that in addition to reforms to increase rigor we should test less, reserving NHST for clearly confirmatory contexts in which the researcher has derived a quantitative prediction, can provide the inputs needed to plan a quality test, and can specify the criteria not only for confirming their hypothesis but also for rejecting it. A reduction in testing would be accompanied by an expansion of the use of estimation [effect sizes and confidence intervals (CIs)]. Estimation is more suitable for exploratory research, provides the inputs needed to plan strong tests, and provides important contexts for properly interpreting tests.

Savings memory is accompanied by transcriptional changes that persist beyond the decay of recall.

Most long-term memories are forgotten. What happens, then, to the changes in neuronal gene expression that were initially required to encode and maintain the memory? Here we show that the decay of recall for long-term sensitization memory in Aplysia is accompanied both by a form of savings memory (easier relearning) and by persistent transcriptional regulation. A behavioral experiment (N = 14) shows that sensitization training produces a robust long-term sensitization memory, but that recall fades completely within 1 wk. This apparent forgetting, though, is belied by persistent savings memory, as we found that a weak reminder protocol reinstates a long-term sensitization memory only on the previously trained side of the body. Using microarray (N = 8 biological replicates), we found that transcriptional regulation largely decays along with recall. Of the transcripts known to be regulated 1 d after training, 98% (1172/1198) are no longer significantly regulated 7 d after training. Still, there is a small set of transcripts which remain strongly regulated even when recall is absent. Using qPCR (N = 11 additional biological replicates) we confirmed that these include the peptide transmitter FMRFamide, a transcript encoding a putative homolog of spectrin beta chain (Genbank: EB255259) , a transcript encoding a protein with a predicted EF-hand calcium-binding domain (Genbank: EB257711), and eight uncharacterized transcripts. To our knowledge, this is the first work to show that transcriptional changes evoked by learning can outlast recall. The small set of transcriptional changes that persist could mediate the rapid relearning of the memory (savings), or the decay of recall, or both, or neither.

Transcriptional changes before and after forgetting of a long-term sensitization memory in Aplysia californica.

Most long-term memories are forgotten, becoming progressively less likely to be recalled. Still, some memory fragments may persist, as savings memory (easier relearning) can be detected long after recall has become impossible. What happens to a memory trace during forgetting that makes it inaccessible for recall and yet still effective to spark easier re-learning? We are addressing this question by tracking the transcriptional changes that accompany learning and then forgetting of a long-term sensitization memory in the tail-elicited siphon withdrawal reflex of Aplysia californica. First, we tracked savings memory. We found that even though recall of sensitization fades completely within 1 week of training, savings memory is still detectable at 2 weeks post training. Next, we tracked the time-course of regulation of 11 transcripts we previously identified as potentially being regulated after recall has become impossible. Remarkably, 3 transcripts still show strong regulation 2 weeks after training and an additional 4 are regulated for at least 1 week. These long-lasting changes in gene expression always begin early in the memory process, within 1 day of training. We present a synthesis of our results tracking gene expression changes accompanying sensitization and provide a testable model of how sensitization memory is forgotten.

Transcriptional correlates of memory maintenance following long-term sensitization of Aplysia californica.

We characterized the transcriptional response accompanying maintenance of long-term sensitization (LTS) memory in the pleural ganglia of Aplysia californica using microarray (N = 8) and qPCR (N = 11 additional samples). We found that 24 h after memory induction there is strong regulation of 1198 transcripts (748 up and 450 down) in a pattern that is almost completely distinct from what is observed during memory encoding (1 h after training). There is widespread up-regulation of transcripts related to all levels of protein production, from transcription (e.g., subunits of transcription initiation factors) to translation (e.g., subunits of eIF1, eIF2, eIF3, eIF4, eIF5, and eIF2B) to activation of components of the unfolded protein response (e.g., CREB3/Luman, BiP, AATF). In addition, there are widespread changes in transcripts related to cytoskeleton function, synaptic targeting, synaptic function, neurotransmitter regulation, and neuronal signaling. Many of the transcripts identified have previously been linked to memory and plasticity (e.g., Egr, menin, TOB1, IGF2 mRNA binding protein 1/ZBP-1), though the majority are novel and/or uncharacterized. Interestingly, there is regulation that could contribute to metaplasticity potentially opposing or even eroding LTS memory (down-regulation of adenylate cyclase and a putative serotonin receptor, up-regulation of FMRFa and a FMRFa receptor). This study reveals that maintenance of a "simple" nonassociative memory is accompanied by an astonishingly complex transcriptional response.

Cartoon Network Update: New Features for Exploring of Neural Circuits.

Cartoon Network is an open-source simulator for neural circuits. It allows students to playfully explore the remarkable complexity that emerges from networks of interacting neurons. This brief report describes four new features that have been added to Cartoon Network: 1) the ability to save and load different networks, 2) the addition of bursting neurons, and 3) new accelerometer inputs for the Finch robot. In addition, new data from the National Institute of Teaching Psychology is reported showing that psychology faculty report strong interest in using Cartoon Network as a pedagogical tool. The updated version of Cartoon Network is available at: https://github.com/rcalinjageman/cartoon_network.

The New Statistics for Neuroscience Majors: Thinking in Effect Sizes.

An ongoing reform in statistical practice is to report and interpret effect sizes. This paper provides a short tutorial on effect sizes and some tips on how to help your students think in terms of effect sizes when analyzing data. An effect size is just a quantitative answer to a research question. Effect sizes should always be accompanied by a confidence interval or some other means of expressing uncertainty in generalizing from the sample to the population. Effect sizes are best interpreted in raw scores, but can also be expressed in standardized terms; several popular standardized effect score measures are explained and compared. Reporting and interpreting effect sizes has several benefits: it focuses on the practical significance of your findings, helps make clear the remaining uncertainty in your findings, fosters better planning for subsequent experiments, fosters meta-analytic thinking, and can help focus efforts on protocol optimization. You can help your students start to think in effect sizes by giving them tools to visualize and translate between different effect size measures, and by tasking them to build a 'library' of effect sizes in a research field of interest.

Cartoon Network: A tool for open-ended exploration of neural circuits.

Cartoon Network is an open-source simulator for neural circuits. It was designed to provide a microworld for the playful exploration of neural networks (similar to the niche Logo/Scratch fills for computer programming). The simulator makes it easy to construct and experiment with closed-loop circuits, enabling students to explore how small sets of neurons can generate complex patterns of activity (oscillations, reverberation). Cartoon Network can be connected to the Finch robot from BirdBrain Technologies, a cheap USB robotics platform. This enables students to design a nervous system for a Finch, creating sensory neurons which read Finch inputs (touch, light, and temperature), motor neurons which control Finch outputs (wheels, lights, and sound), and interneurons to produce intrinsic activity and/or link together inputs and outputs in ways that can produce complex and surprising behaviors. Students use Cartoon Network by taking on structured challenges (For example, can you make the Finch follow a light? Can it turn and run when it bumps a wall?) that require deep engagement with important principles of neural circuit operation (e.g., lateral inhibition, parallel processing, positive- and negative-feedback). On successfully completing a challenge, students can systematically explore the neural properties that help control the behavior, compare their work to other successful designs, and/or reflect on how the circuit they designed might be modified via evolution to produce different behaviors. Cartoon Network has proven an engaging and effective activity for undergraduates and is accessible for students as young as late elementary school. Download it for free at: https://github.com/rcalinjageman/cartoon_network.

After p Values: The New Statistics for Undergraduate Neuroscience Education.

Statistical inference is a methodological cornerstone for neuroscience education. For many years this has meant inculcating neuroscience majors into null hypothesis significance testing with p values. There is increasing concern, however, about the pervasive misuse of p values. It is time to start planning statistics curricula for neuroscience majors that replaces or de-emphasizes p values. One promising alternative approach is what Cumming has dubbed the "New Statistics", an approach that emphasizes effect sizes, confidence intervals, meta-analysis, and open science. I give an example of the New Statistics in action and describe some of the key benefits of adopting this approach in neuroscience education.

Transcriptional analysis of a whole-body form of long-term habituation in Aplysia californica.

Habituation is the simplest form of learning, but we know little about the transcriptional mechanisms that encode long-term habituation memory. A key obstacle is that habituation is relatively stimulus-specific and is thus encoded in small sets of neurons, providing poor signal/noise ratios for transcriptional analysis. To overcome this obstacle, we have developed a protocol for producing whole-body long-term habituation of the siphon-withdrawal reflex (SWR) of Aplysia californica. Specifically, we constructed a computer-controlled brushing apparatus to apply low-intensity tactile stimulation over the entire dorsal surface of Aplysia at regular intervals. We found that 3 d of training (10 rounds of stimulation/day; each round = 15 min brushing at a 10-sec ISI; 15-min rest between rounds) produces habituation with several characteristics favorable for mechanistic investigation. First, habituation is widespread, with SWR durations reduced whether the reflex is evoked by tactile stimulation to the head, tail, or the siphon. Second, long-term habituation is sensitive to the pattern of training, occurring only when brushing sessions are spaced out over 3 d rather than massed into a single session. Using a custom-designed microarray and quantitative PCR, we show that long-term habituation produces long-term up-regulation of an apparent Aplysia homolog of cornichon, a protein important for glutamate receptor trafficking. Our training paradigm provides a promising starting point for characterizing the transcriptional mechanisms of long-term habituation memory.

ERIN: A Portal to Resources for Higher Education in Neuroscience.

ERIN, Educational Resources in Neuroscience, is the Society for Neuroscience's web portal to selected, high-quality materials for higher education. A Board of Editors approves resources after describing them and classifying them by topic, subtopic, media type, author, and appropriate educational level. Some resources are also accompanied by reviews and ratings from faculty who have used the resource. These features make a search of ERIN far more useful than a typical Google search. ERIN's development was funded by the National Science Foundation with a three-year grant to SfN. Along the way, various unexpected problems arose and solutions were found, many of which are described in this overview of ERIN's history and the various decisions that were made in its design and development.

Characterization of the rapid transcriptional response to long-term sensitization training in Aplysia californica.

We used a custom-designed microarray and quantitative PCR to characterize the rapid transcriptional response to long-term sensitization training in the marine mollusk Aplysia californica. Aplysia were exposed to repeated noxious shocks to one side of the body, a procedure known to induce a long-lasting, transcription-dependent increase in reflex responsiveness that is restricted to the side of training. One hour after training, pleural ganglia from the trained and untrained sides of the body were harvested; these ganglia contain the sensory nociceptors which help mediate the expression of long-term sensitization memory. Microarray analysis from 8 biological replicates suggests that long-term sensitization training rapidly regulates at least 81 transcripts. We used qPCR to test a subset of these transcripts and found that 83% were confirmed in the same samples, and 86% of these were again confirmed in an independent sample. Thus, our new microarray design shows strong convergent and predictive validity for analyzing the transcriptional correlates of memory in Aplysia. Fully validated transcripts include some previously identified as regulated in this paradigm (ApC/EBP and ApEgr) but also include novel findings. Specifically, we show that long-term sensitization training rapidly up-regulates the expression of transcripts which may encode Aplysia homologs of a C/EBPγ transcription factor, a glycine transporter (GlyT2), and a vacuolar-protein-sorting-associated protein (VPS36).

Evidence of Active-Forgetting Mechanisms? Blocking Arachidonic Acid Release May Slow Forgetting of Sensitization in Aplysia.

Long-term sensitization in Aplysia is accompanied by a persistent up-regulation of mRNA encoding the peptide neurotransmitter Phe-Met-Arg-Phe-amide (FMRFa), a neuromodulator that opposes the expression of sensitization through activation of the arachidonic acid second-messenger pathway. We completed a preregistered test of the hypothesis that FMRFa plays a critical role in the forgetting of sensitization. Aplysia received long-term sensitization training and were then given whole-body injections of vehicle (N = 27), FMRFa (N = 26), or 4-bromophenacylbromide (4-BPB; N = 31), a phospholipase inhibitor that prevents the release of arachidonic acid. FMRFa produced no changes in forgetting. 4-BPB decreased forgetting measured 6 d after training [d s = 0.55 95% CI(0.01, 1.09)], though the estimated effect size is uncertain. Our results provide preliminary evidence that forgetting of sensitization may be a regulated, active process in Aplysia, but could also indicate a role for arachidonic acid in stabilizing the induction of sensitization.

An Aplysia Egr homolog is rapidly and persistently regulated by long-term sensitization training.

The Egr family of transcription factors plays a key role in long-term plasticity and memory in a number of vertebrate species. Here we identify and characterize ApEgr (GenBank: KC608221), an Egr homolog in the marine mollusk Aplysia californica. ApEgr codes for a predicted 593-amino acid protein with the highly conserved trio of zinc-fingered domains in the C-terminus that characterizes the Egr family of transcription factors. Promoter analysis shows that the ApEgr protein selectively recognizes the GSG motif recognized by vertebrate Egrs. Like mammalian Egrs, ApEgr is constitutively expressed in a range of tissues, including the CNS. Moreover, expression of ApEgr is bi-directionally regulated by changes in neural activity. Of most interest, the association between ApEgr function and memory may be conserved in Aplysia, as we observe rapid and long-lasting up-regulation of expression after long-term sensitization training. Taken together, our results suggest that Egrs may have memory functions that are conserved from mammals to mollusks.

The diamond ratio: A visual indicator of the extent of heterogeneity in meta-analysis.

The result of a meta-analysis is conventionally pictured in the forest plot as a diamond, whose length is the 95% confidence interval (CI) for the summary measure of interest. The Diamond Ratio (DR) is the ratio of the length of the diamond given by a random effects meta-analysis to that given by a fixed effect meta-analysis. The DR is a simple visual indicator of the amount of change caused by moving from a fixed-effect to a random-effects meta-analysis. Increasing values of DR greater than 1.0 indicate increasing heterogeneity relative to the effect variances. We investigate the properties of the DR, and its relationship to four conventional but more complex measures of heterogeneity. We propose for the first time a CI on the DR, and show that it performs well in terms of coverage. We provide example code to calculate the DR and its CI, and to show these in a forest plot. We conclude that the DR is a useful indicator that can assist students and researchers to understand heterogeneity, and to appreciate its extent in particular cases.

Registered Report: Transcriptional Analysis of Savings Memory Suggests Forgetting is Due to Retrieval Failure.

There is fundamental debate about the nature of forgetting: some have argued that it represents the decay of the memory trace, others that the memory trace persists but becomes inaccessible because of retrieval failure. These different accounts of forgetting lead to different predictions about savings memory, the rapid re-learning of seemingly forgotten information. If forgetting is because of decay, then savings requires re-encoding and should thus involve the same mechanisms as initial learning. If forgetting is because of retrieval failure, then savings should be mechanistically distinct from encoding. In this registered report, we conducted a preregistered and rigorous test between these accounts of forgetting. Specifically, we used microarray to characterize the transcriptional correlates of a new memory (1 d after training), a forgotten memory (8 d after training), and a savings memory (8 d after training but with a reminder on day 7 to evoke a long-term savings memory) for sensitization in Aplysia californica (n = 8 samples/group). We found that the reactivation of sensitization during savings does not involve a substantial transcriptional response. Thus, savings is transcriptionally distinct relative to a newer (1-d-old) memory, with no coregulated transcripts, negligible similarity in regulation-ranked ordering of transcripts, and a negligible correlation in training-induced changes in gene expression (r = 0.04 95% confidence interval (CI) [-0.12, 0.20]). Overall, our results suggest that forgetting of sensitization memory represents retrieval failure.

The New Statistics for Better Science: Ask How Much, How Uncertain, and What Else is Known.

The "New Statistics" emphasizes effect sizes, confidence intervals, meta-analysis, and the use of Open Science practices. We present 3 specific ways in which a New Statistics approach can help improve scientific practice: by reducing over-confidence in small samples, by reducing confirmation bias, and by fostering more cautious judgments of consistency. We illustrate these points through consideration of the literature on oxytocin and human trust, a research area that typifies some of the endemic problems that arise with poor statistical practice.

Estimation for Better Inference in Neuroscience.

The estimation approach to inference emphasizes reporting effect sizes with expressions of uncertainty (interval estimates). In this perspective we explain the estimation approach and describe how it can help nudge neuroscientists toward a more productive research cycle by fostering better planning, more thoughtful interpretation, and more balanced evaluation of evidence.

Meta-Analysis of the Effect of Red on Perceived Attractiveness.

We conducted meta-analyses of studies that test the red-romance hypothesis, which is that the color red enhances heterosexual attraction in romantic contexts. For men rating women, we found a small, statistically significant effect ( d = 0.26 [0.12, 0.40], p = .0004, N = 2,961), with substantial heterogeneity, Q(44) = 172.5, pQ < .0001, I2 = 89% [82, 94], and equivocal results regarding the possibility of upward bias in the estimate. For women rating men, we found a very small effect ( d = 0.13 [0.01, 0.25], p = .03, N = 2,739), with substantial heterogeneity, Q(35) = 73.0, pQ = .0002, I2 = 53% [33, 80], and evidence of upward bias in the estimate. Moderator analyses suggest effect sizes may have declined over time (both genders), may be largest when an original shade of red is used (men only), and may be smaller in preregistered studies (women only). We present contrasting interpretations and suggestions for future research.