Pix2HDR - A pixel-wise acquisition and deep learning-based synthesis approach for high-speed HDR videos.

Accurately capturing dynamic scenes with wideranging motion and light intensity is crucial for many vision applications. However, acquiring high-speed high dynamic range (HDR) video is challenging because the camera's frame rate restricts its dynamic range. Existing methods sacrifice speed to acquire multi-exposure frames. Yet, misaligned motion in these frames can still pose complications for HDR fusion algorithms, resulting in artifacts. Instead of frame-based exposures, we sample the videos using individual pixels at varying exposures and phase offsets. Implemented on a monochrome pixel-wise programmable image sensor, our sampling pattern captures fast motion at a high dynamic range. We then transform pixel-wise outputs into an HDR video using end-to-end learned weights from deep neural networks, achieving high spatiotemporal resolution with minimized motion blurring. We demonstrate aliasing-free HDR video acquisition at 1000 FPS, resolving fast motion under low-light conditions and against bright backgrounds - both challenging conditions for conventional cameras. By combining the versatility of pixel-wise sampling patterns with the strength of deep neural networks at decoding complex scenes, our method greatly enhances the vision system's adaptability and performance in dynamic conditions.

Spike-wave discharges during low-current thalamic deep brain stimulation in mice.

BACKGROUND: Deep brain stimulation of central thalamus (CT-DBS) has potential for modulating states of consciousness, but it can also trigger spike-wave discharges (SWDs). OBJECTIVES: To report the probability of inducing SWDs during CT-DBS in awake mice. METHODS: Mice were implanted with electrodes to deliver unilateral and bilateral CT-DBS at different frequencies while recording EEG. We titrated stimulation current by gradually increasing it at each frequency until an SWD appeared. Subsequent stimulations to test arousal modulation were performed at the current one step below the current that caused an SWD during titration. RESULTS: In 2.21% of the test stimulations (10 out of 12 mice), CT-DBS caused SWDs at currents lower than the titrated current, at currents as low as 20 uA. CONCLUSION: Our study found a small but significant probability of inducing SWDs even after titration and at relatively low currents. EEG should be closely monitored for SWDs when performing CT-DBS in both research and clinical settings.

A unified open-source platform for multimodal neural recording and perturbation during naturalistic behavior.

Behavioral neuroscience faces two conflicting demands: long-duration recordings from large neural populations and unimpeded animal behavior. To meet this challenge, we developed ONIX, an open-source data acquisition system with high data throughput (2GB/sec) and low closed-loop latencies (<1ms) that uses a novel 0.3 mm thin tether to minimize behavioral impact. Head position and rotation are tracked in 3D and used to drive active commutation without torque measurements. ONIX can acquire from combinations of passive electrodes, Neuropixels probes, head-mounted microscopes, cameras, 3D-trackers, and other data sources. We used ONIX to perform uninterrupted, long (~7 hours) neural recordings in mice as they traversed complex 3-dimensional terrain. ONIX allowed exploration with similar mobility as non-implanted animals, in contrast to conventional tethered systems which restricted movement. By combining long recordings with full mobility, our technology will enable new progress on questions that require high-quality neural recordings during ethologically grounded behaviors.

How our understanding of memory replay evolves.

Memory reactivations and replay, widely reported in the hippocampus and cortex across species, have been implicated in memory consolidation, planning, and spatial and skill learning. Technological advances in electrophysiology, calcium imaging, and human neuroimaging techniques have enabled neuroscientists to measure large-scale neural activity with increasing spatiotemporal resolution and have provided opportunities for developing robust analytic methods to identify memory replay. In this article, we first review a large body of historically important and representative memory replay studies from the animal and human literature. We then discuss our current understanding of memory replay functions in learning, planning, and memory consolidation and further discuss the progress in computational modeling that has contributed to these improvements. Next, we review past and present analytic methods for replay analyses and discuss their limitations and challenges. Finally, looking ahead, we discuss some promising analytic methods for detecting nonstereotypical, behaviorally nondecodable structures from large-scale neural recordings. We argue that seamless integration of multisite recordings, real-time replay decoding, and closed-loop manipulation experiments will be essential for delineating the role of memory replay in a wide range of cognitive and motor functions.

Imaging the voltage of neurons distributed across entire brains of larval zebrafish.

Neurons interact in networks distributed throughout the brain. Although much effort has focused on whole-brain calcium imaging, recent advances in genetically encoded voltage indicators (GEVIs) raise the possibility of imaging voltage of neurons distributed across brains. To achieve this, a microscope must image at high volumetric rate and signal-to-noise ratio. We present a remote scanning light-sheet microscope capable of imaging GEVI-expressing neurons distributed throughout entire brains of larval zebrafish at a volumetric rate of 200.8 Hz. We measured voltage of ∼1/3 of the neurons of the brain, distributed throughout. We observed that neurons firing at different times during a sequence were located at different brain locations, for sequences elicited by a visual stimulus, which mapped onto locations throughout the optic tectum, as well as during stimulus-independent bursts, which mapped onto locations in the cerebellum and medulla. Whole-brain voltage imaging may open up frontiers in the fundamental operation of neural systems.

Artificial intelligence insights into hippocampal processing.

Advances in artificial intelligence, machine learning, and deep neural networks have led to new discoveries in human and animal learning and intelligence. A recent artificial intelligence agent in the DeepMind family, muZero, can complete a variety of tasks with limited information about the world in which it is operating and with high uncertainty about features of current and future space. To perform, muZero uses only three functions that are general yet specific enough to allow learning across a variety of tasks without overgeneralization across different contexts. Similarly, humans and animals are able to learn and improve in complex environments while transferring learning from other contexts and without overgeneralizing. In particular, the mammalian extrahippocampal system (eHPCS) can guide spatial decision making while simultaneously encoding and processing spatial and contextual information. Like muZero, the eHPCS is also able to adjust contextual representations depending on the degree and significance of environmental changes and environmental cues. In this opinion, we will argue that the muZero functions parallel those of the hippocampal system. We will show that the different components of the muZero model provide a framework for thinking about generalizable learning in the eHPCS, and that the evaluation of how transitions in cell representations occur between similar and distinct contexts can be informed by advances in artificial intelligence agents such as muZero. We additionally explain how advances in AI agents will provide frameworks and predictions by which to investigate the expected link between state changes and neuronal firing. Specifically, we will discuss testable predictions about the eHPCS, including the functions of replay and remapping, informed by the mechanisms behind muZero learning. We conclude with additional ways in which agents such as muZero can aid in illuminating prospective questions about neural functioning, as well as how these agents may shed light on potential expected answers.

A consensus statement on detection of hippocampal sharp wave ripples and differentiation from other fast oscillations.

Decades of rodent research have established the role of hippocampal sharp wave ripples (SPW-Rs) in consolidating and guiding experience. More recently, intracranial recordings in humans have suggested their role in episodic and semantic memory. Yet, common standards for recording, detection, and reporting do not exist. Here, we outline the methodological challenges involved in detecting ripple events and offer practical recommendations to improve separation from other high-frequency oscillations. We argue that shared experimental, detection, and reporting standards will provide a solid foundation for future translational discovery.

Reversal learning: It's just a phase.

Being able to let go of behaviors that are no longer valuable and adopt actions that achieve the same outcome is fundamental for animal survival. A new study offers clues on the neural mechanisms that allow animals to reverse their behavior as needed.

Animal-to-Animal Variability in Partial Hippocampal Remapping in Repeated Environments.

Hippocampal place cells form a map of the environment of an animal. Changes in the hippocampal map can be brought about in a number of ways, including changes to the environment, task, internal state of the subject, and the passage of time. These changes in the hippocampal map have been called remapping. In this study, we examine remapping during repeated exposure to the same environment. Different animals can have different remapping responses to the same changes. This variability across animals in remapping behavior is not well understood. In this work, we analyzed electrophysiological recordings from the CA3 region of the hippocampus performed by Alme et al. (2014), in which five male rats were exposed to 11 different environments, including a variety of repetitions of those environments. To compare the hippocampal maps between two experiences, we computed average rate map correlation coefficients. We found changes in the hippocampal maps between different sessions in the same environment. These changes consisted of partial remapping, a form of remapping in which some place cells maintain their place fields, whereas other place cells remap their place fields. Each animal exhibited partial remapping differently. We discovered that the heterogeneity in hippocampal representational changes across animals is structured; individual animals had consistently different levels of partial remapping across a range of independent comparisons. Our findings highlight that partial hippocampal remapping between repeated environments depends on animal-specific factors.SIGNIFICANCE STATEMENT Context identification is a difficult problem. Animals are not provided with objective context identity labels, so they must infer which experiences come from which contexts. Different animals may have different strategies for performing this inference. We find that different animals have stereotypically different extents of partial hippocampal remapping, a neural correlate of subjective assessment of context identity.

Dissociating Behavior and Spatial Working Memory Demands Using an H Maze.

The development of mazes for animal experiments has allowed for the investigation of cognitive maps and place cells, spatial working memory, naturalistic navigation, perseverance, exploration, and choice and motivated behavior. However, many mazes, such as the T maze, currently developed to test learning and memory, do not distinguish temporally and spatially between the encoding and recall periods, which makes it difficult to study these stages separately when analyzing animal behavior and electrophysiology. Other mazes, such as the radial maze, rely on single visits to portions of the maze, making maze coverage sparse for place cell and electrophysiology experiments. In this protocol, we present instructions for building and training an animal on a spatial appetitive choice task on a low-cost double-sided T (or H) maze. This maze has several advantages over the traditional T maze and radial mazes. This maze is unique in that it temporally and directionally dissociates the memory encoding and retrieval periods, while requiring the same behaviors of the animal during both periods. This design allows for independent investigation of brain mechanisms, such as cross-region theta coordination, during memory encoding and retrieval, while at least partially dissociating these stages from behavior. This maze has been previously used in our laboratory to investigate cell firing, single-region local field potential (LFP) patterns, and cross region LFP coherence in the hippocampus, lateral septum, prefrontal cortex, and ventral tegmental area, as well as to investigate the effects of hippocampal theta perturbations on task performance.

Lateral septum as a nexus for mood, motivation, and movement.

The lateral septum (LS) has been implicated in a wide variety of functions, including emotional, motivational, and spatial behavior, and the LS may regulate interactions between the hippocampus and other regions that mediate goal directed behavior. In this review, we suggest that the lateral septum incorporates movement into the evaluation of environmental context with respect to motivation, anxiety, and reward to output an 'integrated movement value signal'. Specifically, hippocampally-derived contextual information may be combined with reinforcement or motivational information in the LS to inform task-relevant decisions. We will discuss how movement is represented in the LS and the literature on the LS's involvement in mood and motivation. We will then connect these results to LS movement-related literature and hypotheses about the role of the lateral septum. We suggest that the LS may communicate a movement-scaled reward signal via changes in place-, movement-, and reward-related firing, and that the LS should be considered a fundamental node of affect and locomotor pathways in the brain.

Eszopiclone and Zolpidem Produce Opposite Effects on Hippocampal Ripple Density.

Clinical populations have memory deficits linked to sleep oscillations that can potentially be treated with sleep medications. Eszopiclone and zolpidem (two non-benzodiazepine hypnotics) both enhance sleep spindles. Zolpidem improved sleep-dependent memory consolidation in humans, but eszopiclone did not. These divergent results may reflect that the two drugs have different effects on hippocampal ripple oscillations, which correspond to the reactivation of neuronal ensembles that represent previous waking activity and contribute to memory consolidation. We used extracellular recordings in the CA1 region of rats and systemic dosing of eszopiclone and zolpidem to test the hypothesis that these two drugs differentially affect hippocampal ripples and spike activity. We report evidence that eszopiclone makes ripples sparser, while zolpidem increases ripple density. In addition, eszopiclone led to a drastic decrease in spike firing, both in putative pyramidal cells and interneurons, while zolpidem did not substantially alter spiking. These results provide an explanation of the different effects of eszopiclone and zolpidem on memory in human studies and suggest that sleep medications can be used to regulate hippocampal ripple oscillations, which are causally linked to sleep-dependent memory consolidation.

Hippocampal remapping as hidden state inference.

Cells in the hippocampus tuned to spatial location (place cells) typically change their tuning when an animal changes context, a phenomenon known as remapping. A fundamental challenge to understanding remapping is the fact that what counts as a ''context change'' has never been precisely defined. Furthermore, different remapping phenomena have been classified on the basis of how much the tuning changes after different types and degrees of context change, but the relationship between these variables is not clear. We address these ambiguities by formalizing remapping in terms of hidden state inference. According to this view, remapping does not directly reflect objective, observable properties of the environment, but rather subjective beliefs about the hidden state of the environment. We show how the hidden state framework can resolve a number of puzzles about the nature of remapping.

mPFC spindle cycles organize sparse thalamic activation and recently active CA1 cells during non-REM sleep.

Sleep oscillations in the neocortex and hippocampus are critical for the integration of new memories into stable generalized representations in neocortex. However, the role of the thalamus in this process is poorly understood. To determine the thalamic contribution to non-REM oscillations (sharp-wave ripples, SWRs; slow/delta; spindles), we recorded units and local field potentials (LFPs) simultaneously in the limbic thalamus, mPFC, and CA1 in rats. We report that the cycles of neocortical spindles provide a key temporal window that coordinates CA1 SWRs with sparse but consistent activation of thalamic units. Thalamic units were phase-locked to delta and spindles in mPFC, and fired at consistent lags with other thalamic units within spindles, while CA1 units that were active during spatial exploration were engaged in SWR-coupled spindles after behavior. The sparse thalamic firing could promote an incremental integration of recently acquired memory traces into neocortical schemas through the interleaved activation of thalamocortical cells.

Differences in reward biased spatial representations in the lateral septum and hippocampus.

The lateral septum (LS), which is innervated by the hippocampus, is known to represent spatial information. However, the details of place representation in the LS, and whether this place information is combined with reward signaling, remains unknown. We simultaneously recorded from rat CA1 and caudodorsal lateral septum in rat during a rewarded navigation task and compared spatial firing in the two areas. While LS place cells are less numerous than in hippocampus, they are similar to the hippocampus in field size and number of fields per cell, but with field shape and center distributions that are more skewed toward reward. Spike cross-correlations between the hippocampus and LS are greatest for cells that have reward-proximate place fields, suggesting a role for the LS in relaying task-relevant hippocampal spatial information to downstream areas, such as the VTA.

An easy-to-assemble, robust, and lightweight drive implant for chronic tetrode recordings in freely moving animals.

Tetrode arrays are a standard method for neuronal recordings in behaving animals, especially for chronic recordings of many neurons in freely-moving animals. OBJECTIVE: We sought to simplify tetrode drive designs with the aim of enabling building and implanting a 16-tetrode drive in a single day. APPROACH: Our design makes use of recently developed technologies to reduce the complexity of the drive while maintaining a low weight. MAIN RESULTS: The design presents an improvement over existing implants in terms of robustness, weight, and ease of use. We describe two variants: a 16 tetrode implant weighing ∼2 g for mice, bats, tree shrews and similar animals, and a 64 tetrode implant weighing ∼16 g for rats and similar animals. These designs were co-developed and optimized alongside a new class of drive-mounted feature-rich amplifier boards with ultra-thin radio-frequency tethers, as described in an upcoming paper (Newman, Zhang et al in prep). SIGNIFICANCE: This design significantly improves the data yield of chronic electrophysiology experiments.

Twister3: a simple and fast microwire twister.

OBJECTIVE: Twisted wire probes (TWPs, e.g. stereotrodes and tetrodes) provide a cheap and reliable method for obtaining high quality, multiple single-unit neural recordings in freely moving animals. Despite their ubiquity, TWPs are constructed using a tedious procedure consisting of manually folding, turning, and fusing microwire. This imposes a significant labor burden on research personnel who use TWPs in their experiments. APPROACH: To address this issue, we created Twister3, an open-source microwire twisting machine. This machine features a quick-draw wire feeder that eliminates manual wire folding, an auto-aligning motor attachment mechanism which results in consistently straight probes, and a high speed motor for rapid probe turning. MAIN RESULTS: Twister3 greatly increases the speed and repeatability of constructing twisted microwire probes compared to existing options. Users with less than one hour of experience using the device were able to make ~70 tetrodes per hour, on average. It is cheap, well documented, and all associated designs and source code are open-source. SIGNIFICANCE: Twister3 significantly reduces the labor burden of creating high-quality TWPs so electrophysiologists can spend more of their time performing recordings rather than making probes. Therefore, this device is of interest to any lab performing TWP neural recordings, for example, using microdrives.

A closed-loop, all-electronic pixel-wise adaptive imaging system for high dynamic range videography.

Digital cameras expose and readout all pixels in accordance with a global sample clock. This rigid global control of exposure and sampling is problematic for capturing scenes with large variance in brightness and motion, and may cause regions of motion blur, under- and overexposure. To address these issues, we developed a CMOS imaging system that automatically adjusts each pixel's exposure and sampling rate to fit local motion and brightness. This system consists of an image sensor with pixel-addressable exposure configurability in combination with a real-time, per-pixel exposure controller. It operates in a closed-loop to sample, detect and optimize each pixel's exposure and sampling rate for optimal acquisition. Per-pixel exposure control is implemented using all-integrated electronics without external optical modulation. This reduces system complexity and power consumption compared to existing solutions. Implemented using standard 130nm CMOS process, the chip has 256 × 256 pixels and consumes 7.31mW. To evaluate performance, we used this system to capture scenes with complex lighting and motion conditions that would lead to loss of information for globally-exposed cameras. These results demonstrate the advantage of pixel-wise adaptive imaging for a range of computer vision tasks such as segmentation, motion estimation and object recognition.

Locomotor and Hippocampal Processing Converge in the Lateral Septum.

The lateral septum (LS) has been implicated in anxiety and fear modulation and may regulate interactions between the hippocampus and regions, such as the VTA, that mediate goal-directed behavior. In this study, we simultaneously record from cells in the LS and the hippocampus during navigation and conditioning tasks. In the LS, we identify a speed and acceleration spiking code that does not map to states of anticipation or reward. Additionally, we identify an overlapping population of LS cells that change firing to cue and reward during conditioning. These cells display sharp wave ripple and theta modulation, spatial firing fields, and responses similar to the hippocampus during conditioning. These hippocampus-associated cells are not disproportionately speed or acceleration modulated, suggesting that these movement correlates are not hippocampally derived. Finally, we show that LS theta coordination is selectively enhanced in hippocampus-associated LS cells during navigation behavior that requires working memory. Taken together, these results suggest a role for the LS in transmitting spatial and contextual information, in concert with locomotor information, to downstream areas, such as the VTA, where value weighting may take place.