Self-supervised learning for interventional image analytics: toward robust device trackers.

Purpose: The accurate detection and tracking of devices, such as guiding catheters in live X-ray image acquisitions, are essential prerequisites for endovascular cardiac interventions. This information is leveraged for procedural guidance, e.g., directing stent placements. To ensure procedural safety and efficacy, there is a need for high robustness/no failures during tracking. To achieve this, one needs to efficiently tackle challenges, such as device obscuration by the contrast agent or other external devices or wires and changes in the field-of-view or acquisition angle, as well as the continuous movement due to cardiac and respiratory motion. Approach: To overcome the aforementioned challenges, we propose an approach to learn spatio-temporal features from a very large data cohort of over 16 million interventional X-ray frames using self-supervision for image sequence data. Our approach is based on a masked image modeling technique that leverages frame interpolation-based reconstruction to learn fine inter-frame temporal correspondences. The features encoded in the resulting model are fine-tuned downstream in a light-weight model. Results: Our approach achieves state-of-the-art performance, in particular for robustness, compared to ultra optimized reference solutions (that use multi-stage feature fusion or multi-task and flow regularization). The experiments show that our method achieves a 66.31% reduction in the maximum tracking error against the reference solutions (23.20% when flow regularization is used), achieving a success score of 97.95% at a 3× faster inference speed of 42 frames-per-second (on GPU). In addition, we achieve a 20% reduction in the standard deviation of errors, which indicates a much more stable tracking performance. Conclusions: The proposed data-driven approach achieves superior performance, particularly in robustness and speed compared with the frequently used multi-modular approaches for device tracking. The results encourage the use of our approach in various other tasks within interventional image analytics that require effective understanding of spatio-temporal semantics.

Distinct information conveyed to the olfactory bulb by feedforward input from the nose and feedback from the cortex.

Sensory systems are organized hierarchically, but feedback projections frequently disrupt this order. In the olfactory bulb (OB), cortical feedback projections numerically match sensory inputs. To unravel information carried by these two streams, we imaged the activity of olfactory sensory neurons (OSNs) and cortical axons in the mouse OB using calcium indicators, multiphoton microscopy, and diverse olfactory stimuli. Here, we show that odorant mixtures of increasing complexity evoke progressively denser OSN activity, yet cortical feedback activity is of similar sparsity for all stimuli. Also, representations of complex mixtures are similar in OSNs but are decorrelated in cortical axons. While OSN responses to increasing odorant concentrations exhibit a sigmoidal relationship, cortical axonal responses are complex and nonmonotonic, which can be explained by a model with activity-dependent feedback inhibition in the cortex. Our study indicates that early-stage olfactory circuits have access to local feedforward signals and global, efficiently formatted information about odor scenes through cortical feedback.

The role of prospective contingency in the control of behavior and dopamine signals during associative learning.

Associative learning depends on contingency, the degree to which a stimulus predicts an outcome. Despite its importance, the neural mechanisms linking contingency to behavior remain elusive. Here we examined the dopamine activity in the ventral striatum - a signal implicated in associative learning - in a Pavlovian contingency degradation task in mice. We show that both anticipatory licking and dopamine responses to a conditioned stimulus decreased when additional rewards were delivered uncued, but remained unchanged if additional rewards were cued. These results conflict with contingency-based accounts using a traditional definition of contingency or a novel causal learning model (ANCCR), but can be explained by temporal difference (TD) learning models equipped with an appropriate inter-trial-interval (ITI) state representation. Recurrent neural networks trained within a TD framework develop state representations like our best 'handcrafted' model. Our findings suggest that the TD error can be a measure that describes both contingency and dopaminergic activity.

Interpretable deep learning for deconvolutional analysis of neural signals.

The widespread adoption of deep learning to build models that capture the dynamics of neural populations is typically based on "black-box" approaches that lack an interpretable link between neural activity and function. Here, we propose to apply algorithm unrolling, a method for interpretable deep learning, to design the architecture of sparse deconvolutional neural networks and obtain a direct interpretation of network weights in relation to stimulus-driven single-neuron activity through a generative model. We characterize our method, referred to as deconvolutional unrolled neural learning (DUNL), and show its versatility by applying it to deconvolve single-trial local signals across multiple brain areas and recording modalities. To exemplify use cases of our decomposition method, we uncover multiplexed salience and reward prediction error signals from midbrain dopamine neurons in an unbiased manner, perform simultaneous event detection and characterization in somatosensory thalamus recordings, and characterize the responses of neurons in the piriform cortex. Our work leverages the advances in interpretable deep learning to gain a mechanistic understanding of neural dynamics.

Adaptive algorithms for shaping behavior.

Dogs and laboratory mice are commonly trained to perform complex tasks by guiding them through a curriculum of simpler tasks ('shaping'). What are the principles behind effective shaping strategies? Here, we propose a machine learning framework for shaping animal behavior, where an autonomous teacher agent decides its student's task based on the student's transcript of successes and failures on previously assigned tasks. Using autonomous teachers that plan a curriculum in a common sequence learning task, we show that near-optimal shaping algorithms adaptively alternate between simpler and harder tasks to carefully balance reinforcement and extinction. Based on this intuition, we derive an adaptive shaping heuristic with minimal parameters, which we show is near-optimal on the sequence learning task and robustly trains deep reinforcement learning agents on navigation tasks that involve sparse, delayed rewards. Extensions to continuous curricula are explored. Our work provides a starting point towards a general computational framework for shaping animal behavior.

Neural Circuits for Fast Poisson Compressed Sensing in the Olfactory Bulb.

Within a single sniff, the mammalian olfactory system can decode the identity and concentration of odorants wafted on turbulent plumes of air. Yet, it must do so given access only to the noisy, dimensionally-reduced representation of the odor world provided by olfactory receptor neurons. As a result, the olfactory system must solve a compressed sensing problem, relying on the fact that only a handful of the millions of possible odorants are present in a given scene. Inspired by this principle, past works have proposed normative compressed sensing models for olfactory decoding. However, these models have not captured the unique anatomy and physiology of the olfactory bulb, nor have they shown that sensing can be achieved within the 100-millisecond timescale of a single sniff. Here, we propose a rate-based Poisson compressed sensing circuit model for the olfactory bulb. This model maps onto the neuron classes of the olfactory bulb, and recapitulates salient features of their connectivity and physiology. For circuit sizes comparable to the human olfactory bulb, we show that this model can accurately detect tens of odors within the timescale of a single sniff. We also show that this model can perform Bayesian posterior sampling for accurate uncertainty estimation. Fast inference is possible only if the geometry of the neural code is chosen to match receptor properties, yielding a distributed neural code that is not axis-aligned to individual odor identities. Our results illustrate how normative modeling can help us map function onto specific neural circuits to generate new hypotheses.

Nonequilibrium sensing of volatile compounds using active and passive analyte delivery.

Although sensor technologies have allowed us to outperform the human senses of sight, hearing, and touch, the development of artificial noses is significantly behind their biological counterparts. This largely stems from the sophistication of natural olfaction, which relies on both fluid dynamics within the nasal anatomy and the response patterns of hundreds to thousands of unique molecular-scale receptors. We designed a sensing approach to identify volatiles inspired by the fluid dynamics of the nose, allowing us to extract information from a single sensor (here, the reflectance spectra from a mesoporous one-dimensional photonic crystal) rather than relying on a large sensor array. By accentuating differences in the nonequilibrium mass-transport dynamics of vapors and training a machine learning algorithm on the sensor output, we clearly identified polar and nonpolar volatile compounds, determined the mixing ratios of binary mixtures, and accurately predicted the boiling point, flash point, vapor pressure, and viscosity of a number of volatile liquids, including several that had not been used for training the model. We further implemented a bioinspired active sniffing approach, in which the analyte delivery was performed in well-controlled 'inhale-exhale' sequences, enabling an additional modality of differentiation and reducing the duration of data collection and analysis to seconds. Our results outline a strategy to build accurate and rapid artificial noses for volatile compounds that can provide useful information such as the composition and physical properties of chemicals, and can be applied in a variety of fields, including disease diagnosis, hazardous waste management, and healthy building monitoring.

Experience-dependent evolution of odor mixture representations in piriform cortex.

Rodents can learn from exposure to rewarding odors to make better and quicker decisions. The piriform cortex is thought to be important for learning complex odor associations; however, it is not understood exactly how it learns to remember discriminations between many, sometimes overlapping, odor mixtures. We investigated how odor mixtures are represented in the posterior piriform cortex (pPC) of mice while they learn to discriminate a unique target odor mixture against hundreds of nontarget mixtures. We find that a significant proportion of pPC neurons discriminate between the target and all other nontarget odor mixtures. Neurons that prefer the target odor mixture tend to respond with brief increases in firing rate at odor onset compared to other neurons, which exhibit sustained and/or decreased firing. We allowed mice to continue training after they had reached high levels of performance and find that pPC neurons become more selective for target odor mixtures as well as for randomly chosen repeated nontarget odor mixtures that mice did not have to discriminate from other nontargets. These single unit changes during overtraining are accompanied by better categorization decoding at the population level, even though behavioral metrics of mice such as reward rate and latency to respond do not change. However, when difficult ambiguous trial types are introduced, the robustness of the target selectivity is correlated with better performance on the difficult trials. Taken together, these data reveal pPC as a dynamic and robust system that can optimize for both current and possible future task demands at once.

Dynamics of cooperative excavation in ant and robot collectives.

The solution of complex problems by the collective action of simple agents in both biologically evolved and synthetically engineered systems involves cooperative action. Understanding the resulting emergent solutions requires integrating across the organismal behavior of many individuals. Here, we investigate an ecologically relevant collective task in black carpenter ants Camponotus pennsylvanicus: excavation of a soft, erodible confining corral. These ants show a transition from individual exploratory excavation at random locations to spatially localized collective exploitative excavation and escape from the corral. Agent-based simulations and a minimal continuum theory that coarse-grains over individual actions and considers their integrated influence on the environment leads to the emergence of an effective phase space of behaviors, characterized in terms of excavation strength and cooperation intensity. To test the theory over the range of both observed and predicted behaviors, we use custom-built robots (RAnts) that respond to stimuli to characterize the phase space of emergence (and failure) of cooperative excavation. Tuning the amount of cooperation between RAnts, allows us to vary the efficiency of excavation and synthetically generate the entire range of macroscopic phases predicted by our theory. Overall, our approach shows how the cooperative completion of tasks can arise from simple rules that involve the interaction of agents with a dynamically changing environment that serves as both an enabler and a modulator of behavior.

Distinct representation of cue-outcome association by D1 and D2 neurons in the ventral striatum's olfactory tubercle.

Positive and negative associations acquired through olfactory experience are thought to be especially strong and long-lasting. The conserved direct olfactory sensory input to the ventral striatal olfactory tubercle (OT) and its convergence with dense dopaminergic input to the OT could underlie this privileged form of associative memory, but how this process occurs is not well understood. We imaged the activity of the two canonical types of striatal neurons, expressing D1- or D2-type dopamine receptors, in the OT at cellular resolution while mice learned odor-outcome associations ranging from aversive to rewarding. D1 and D2 neurons both responded to rewarding and aversive odors. D1 neurons in the OT robustly and bidirectionally represented odor valence, responding similarly to odors predicting similar outcomes regardless of odor identity. This valence representation persisted even in the absence of a licking response to the odors and in the absence of the outcomes, indicating a true transformation of odor sensory information by D1 OT neurons. In contrast, D2 neuronal representation of the odor-outcome associations was weaker, contingent on a licking response by the mouse, and D2 neurons were more selective for odor identity than valence. Stimulus valence coding in the OT was modality-sensitive, with separate sets of D1 neurons responding to odors and sounds predicting the same outcomes, suggesting that integration of multimodal valence information happens downstream of the OT. Our results point to distinct representation of identity and valence of odor stimuli by D1 and D2 neurons in the OT.

Multi-animal pose estimation, identification and tracking with DeepLabCut.

Estimating the pose of multiple animals is a challenging computer vision problem: frequent interactions cause occlusions and complicate the association of detected keypoints to the correct individuals, as well as having highly similar looking animals that interact more closely than in typical multi-human scenarios. To take up this challenge, we build on DeepLabCut, an open-source pose estimation toolbox, and provide high-performance animal assembly and tracking-features required for multi-animal scenarios. Furthermore, we integrate the ability to predict an animal's identity to assist tracking (in case of occlusions). We illustrate the power of this framework with four datasets varying in complexity, which we release to serve as a benchmark for future algorithm development.

Natural gradient enables fast sampling in spiking neural networks.

For animals to navigate an uncertain world, their brains need to estimate uncertainty at the timescales of sensations and actions. Sampling-based algorithms afford a theoretically-grounded framework for probabilistic inference in neural circuits, but it remains unknown how one can implement fast sampling algorithms in biologically-plausible spiking networks. Here, we propose to leverage the population geometry, controlled by the neural code and the neural dynamics, to implement fast samplers in spiking neural networks. We first show that two classes of spiking samplers-efficient balanced spiking networks that simulate Langevin sampling, and networks with probabilistic spike rules that implement Metropolis-Hastings sampling-can be unified within a common framework. We then show that careful choice of population geometry, corresponding to the natural space of parameters, enables rapid inference of parameters drawn from strongly-correlated high-dimensional distributions in both networks. Our results suggest design principles for algorithms for sampling-based probabilistic inference in spiking neural networks, yielding potential inspiration for neuromorphic computing and testable predictions for neurobiology.

Deletion of TrkB in parvalbumin interneurons alters cortical neural dynamics.

Signaling by neurotrophins such as the brain-derived neurotrophic factor (BDNF) is known to modulate development of interneurons, but the circuit effects of this modulation remain unclear. Here, we examined the impact of deleting TrkB, a BDNF receptor, in parvalbumin-expressing (PV) interneurons on the balance of excitation and inhibition (E-I) in cortical circuits. In the mouse olfactory cortex, TrkB deletion impairs multiple aspects of PV neuronal function including synaptic excitation, intrinsic excitability, and the innervation pattern of principal neurons. Impaired PV cell function resulted in aberrant spiking patterns in principal neurons in response to stimulation of sensory inputs. Surprisingly, dampened PV neuronal function leads to a paradoxical decrease in overall excitability in cortical circuits. Our study demonstrates that, by modulating PV circuit plasticity and development, TrkB plays a critical role in shaping the evoked pattern of activity in a cortical network.

Antagonistic odor interactions in olfactory sensory neurons are widespread in freely breathing mice.

Odor landscapes contain complex blends of molecules that each activate unique, overlapping populations of olfactory sensory neurons (OSNs). Despite the presence of hundreds of OSN subtypes in many animals, the overlapping nature of odor inputs may lead to saturation of neural responses at the early stages of stimulus encoding. Information loss due to saturation could be mitigated by normalizing mechanisms such as antagonism at the level of receptor-ligand interactions, whose existence and prevalence remains uncertain. By imaging OSN axon terminals in olfactory bulb glomeruli as well as OSN cell bodies within the olfactory epithelium in freely breathing mice, we find widespread antagonistic interactions in binary odor mixtures. In complex mixtures of up to 12 odorants, antagonistic interactions are stronger and more prevalent with increasing mixture complexity. Therefore, antagonism is a common feature of odor mixture encoding in OSNs and helps in normalizing activity to reduce saturation and increase information transfer.

Rapid Learning of Odor-Value Association in the Olfactory Striatum.

Rodents can successfully learn multiple novel stimulus-response associations after only a few repetitions when the contingencies predict reward. The circuits modified during such reinforcement learning to support decision-making are not known, but the olfactory tubercle (OT) and posterior piriform cortex (pPC) are candidates for decoding reward category from olfactory sensory input and relaying this information to cognitive and motor areas. Through single-cell recordings in behaving male and female C57BL/6 mice, we show here that an explicit representation for reward category emerges in the OT within minutes of learning a novel odor-reward association, whereas the pPC lacks an explicit representation even after weeks of overtraining. The explicit reward category representation in OT is visible in the first sniff (50-100 ms) of an odor on each trial, and precedes the motor action. Together, these results suggest that the coding of stimulus information required for reward prediction does not occur within olfactory cortex, but rather in circuits involving the olfactory striatum.SIGNIFICANCE STATEMENT Rodents are olfactory specialists and can use odors to learn contingencies quickly and well. We have found that mice can readily learn to place multiple odors into rewarded and unrewarded categories. Once they have learned the rule, they can do such categorization in a matter of minutes (<10 trials). We found that neural activity in olfactory cortex largely reflects sensory coding, with very little explicit information about categories. By contrast, neural activity in a brain region in the ventral striatum is rapidly modified in a matter of minutes to reflect reward category. Our experiments set up a paradigm for studying rapid sensorimotor reinforcement in a circuit that is right at the interface of sensory input and reward areas.

Microglial depletion disrupts normal functional development of adult-born neurons in the olfactory bulb.

Microglia play key roles in regulating synapse development and refinement in the developing brain, but it is unknown whether they are similarly involved during adult neurogenesis. By transiently depleting microglia from the healthy adult mouse brain, we show that microglia are necessary for the normal functional development of adult-born granule cells (abGCs) in the olfactory bulb. Microglial depletion reduces the odor responses of developing, but not preexisting GCs in vivo in both awake and anesthetized mice. Microglia preferentially target their motile processes to interact with mushroom spines on abGCs, and when microglia are absent, abGCs develop smaller spines and receive weaker excitatory synaptic inputs. These results suggest that microglia promote the development of excitatory synapses onto developing abGCs, which may impact the function of these cells in the olfactory circuit.

The brain has its own population of resident immune cells known as microglia, which defend against infections and are involved in conditions such as Alzheimer's, Parkinson's and other diseases. In the last decade, new studies have suggested that these cells also sculpt brain circuits during early development. They can 'eat' weak connections between neurons, and help strong ones to mature. Most of brain 'wiring' happens during development, when the majority of neurons is born and connects together. However, a few brain areas can incorporate new neurons during adulthood into existing circuits. In mice for example, this process takes place in the olfactory bulb, the area that first processes smells: it is believed that new neurons connecting to existing ones helps to detect new odors. It is unclear, however, whether microglia also help to shape these connections, or if their role is confined to early development. To investigate this question, Wallace et al. gave adult mice a drug that kills only microglia, and then examined how the neurons respond when the animals are exposed to smells. The results show that the new neurons that developed without microglia responded to fewer odors. These neurons also formed weaker connections and had physical features that indicated they might not have been properly incorporated into the circuit. It may be possible to encourage new neurons to be born in brain areas that normally do not produce these cells in adulthood. Ultimately, this could potentially help to repair the damages of age or disease, but this will rely on understanding exactly how new neurons are integrated into existing brain circuits. Future work, however, is still necessary to figure out how much these new neurons could compensate for cells damaged by injury or disease.