Differentiable optimization of the Debye-Wolf integral for light shaping and adaptive optics in two-photon microscopy.

Control of light through a microscope objective with a high numerical aperture is a common requirement in applications such as optogenetics, adaptive optics, or laser processing. Light propagation, including polarization effects, can be described under these conditions using the Debye-Wolf diffraction integral. Here, we take advantage of differentiable optimization and machine learning for efficiently optimizing the Debye-Wolf integral for such applications. For light shaping we show that this optimization approach is suitable for engineering arbitrary three-dimensional point spread functions in a two-photon microscope. For differentiable model-based adaptive optics (DAO), the developed method can find aberration corrections with intrinsic image features, for example neurons labeled with genetically encoded calcium indicators, without requiring guide stars. Using computational modeling we further discuss the range of spatial frequencies and magnitudes of aberrations which can be corrected with this approach.

Integration of sleep homeostasis and navigation in Drosophila.

During sleep, the brain undergoes dynamic and structural changes. In Drosophila, such changes have been observed in the central complex, a brain area important for sleep control and navigation. The connectivity of the central complex raises the question about how navigation, and specifically the head direction system, can operate in the face of sleep related plasticity. To address this question, we develop a model that integrates sleep homeostasis and head direction. We show that by introducing plasticity, the head direction system can function in a stable way by balancing plasticity in connected circuits that encode sleep pressure. With increasing sleep pressure, the head direction system nevertheless becomes unstable and a sleep phase with a different plasticity mechanism is introduced to reset network connectivity. The proposed integration of sleep homeostasis and head direction circuits captures features of their neural dynamics observed in flies and mice.

Differentiable model-based adaptive optics for two-photon microscopy.

Aberrations limit scanning fluorescence microscopy when imaging in scattering materials such as biological tissue. Model-based approaches for adaptive optics take advantage of a computational model of the optical setup. Such models can be combined with the optimization techniques of machine learning frameworks to find aberration corrections, as was demonstrated for focusing a laser beam through aberrations onto a camera [Opt. Express2826436 (26436)10.1364/OE.403487]. Here, we extend this approach to two-photon scanning microscopy. The developed sensorless technique finds corrections for aberrations in scattering samples and will be useful for a range of imaging application, for example in brain tissue.

Neural dynamics for landmark orientation and angular path integration.

Many animals navigate using a combination of visual landmarks and path integration. In mammalian brains, head direction cells integrate these two streams of information by representing an animal's heading relative to landmarks, yet maintaining their directional tuning in darkness based on self-motion cues. Here we use two-photon calcium imaging in head-fixed Drosophila melanogaster walking on a ball in a virtual reality arena to demonstrate that landmark-based orientation and angular path integration are combined in the population responses of neurons whose dendrites tile the ellipsoid body, a toroidal structure in the centre of the fly brain. The neural population encodes the fly's azimuth relative to its environment, tracking visual landmarks when available and relying on self-motion cues in darkness. When both visual and self-motion cues are absent, a representation of the animal's orientation is maintained in this network through persistent activity, a potential substrate for short-term memory. Several features of the population dynamics of these neurons and their circular anatomical arrangement are suggestive of ring attractors, network structures that have been proposed to support the function of navigational brain circuits.

Differentiable model-based adaptive optics with transmitted and reflected light.

Aberrations limit optical systems in many situations, for example when imaging in biological tissue. Machine learning offers novel ways to improve imaging under such conditions by learning inverse models of aberrations. Learning requires datasets that cover a wide range of possible aberrations, which however becomes limiting for more strongly scattering samples, and does not take advantage of prior information about the imaging process. Here, we show that combining model-based adaptive optics with the optimization techniques of machine learning frameworks can find aberration corrections with a small number of measurements. Corrections are determined in a transmission configuration through a single aberrating layer and in a reflection configuration through two different layers at the same time. Additionally, corrections are not limited by a predetermined model of aberrations (such as combinations of Zernike modes). Focusing in transmission can be achieved based only on reflected light, compatible with an epidetection imaging configuration.

Wavefront correction for adaptive optics with reflected light and deep neural networks.

Light scattering and aberrations limit optical microscopy in biological tissue, which motivates the development of adaptive optics techniques. Here, we develop a method for wavefront correction in adaptive optics with reflected light and deep neural networks compatible with an epi-detection configuration. Large datasets of sample aberrations which consist of excitation and detection path aberrations as well as the corresponding reflected focus images are generated. These datasets are used for training deep neural networks. After training, these networks can disentangle and independently correct excitation and detection aberrations based on reflected light images recorded from scattering samples. A similar deep learning approach is also demonstrated with scattering guide stars. The predicted aberration corrections are validated using two photon imaging.

Two-photon Bessel beam tomography for fast volume imaging.

Light microscopy on dynamic samples, for example neural activity in the brain, often requires imaging volumes that extend over several 100 µm in axial direction at a rate of at least several tens of Hertz. Here, we develop a tomography approach for scanning fluorescence microscopy which allows recording a volume image in a single frame scan. Volumes are imaged by simultaneously recording four independent projections at different angles using temporally multiplexed, tilted Bessel beams. From the resulting projections, three-dimensional images are reconstructed using inverse Radon transforms combined with convolutional neural networks (U-net).

Feature detection and orientation tuning in the Drosophila central complex.

Many animals, including insects, are known to use visual landmarks to orient in their environment. In Drosophila melanogaster, behavioural genetics studies have identified a higher brain structure called the central complex as being required for the fly's innate responses to vertical visual features and its short- and long-term memory for visual patterns. But whether and how neurons of the fly central complex represent visual features are unknown. Here we use two-photon calcium imaging in head-fixed walking and flying flies to probe visuomotor responses of ring neurons--a class of central complex neurons that have been implicated in landmark-driven spatial memory in walking flies and memory for visual patterns in tethered flying flies. We show that dendrites of ring neurons are visually responsive and arranged retinotopically. Ring neuron receptive fields comprise both excitatory and inhibitory subfields, resembling those of simple cells in the mammalian primary visual cortex. Ring neurons show strong and, in some cases, direction-selective orientation tuning, with a notable preference for vertically oriented features similar to those that evoke innate responses in flies. Visual responses were diminished during flight, but, in contrast with the hypothesized role of the central complex in the control of locomotion, not modulated during walking. Taken together, these results indicate that ring neurons represent behaviourally relevant visual features in the fly's environment, enabling downstream central complex circuits to produce appropriate motor commands. More broadly, this study opens the door to mechanistic investigations of circuit computations underlying visually guided action selection in the Drosophila central complex.

Light scattering control in transmission and reflection with neural networks.

Scattering often limits the controlled delivery of light in applications such as biomedical imaging, optogenetics, optical trapping, and fiber-optic communication or imaging. Such scattering can be controlled by appropriately shaping the light wavefront entering the material. Here, we develop a machine-learning approach for light control. Using pairs of binary intensity patterns and intensity measurements we train neural networks (NNs) to provide the wavefront corrections necessary to shape the beam after the scatterer. Additionally, we demonstrate that NNs can be used to find a functional relationship between transmitted and reflected speckle patterns. Establishing the validity of this relationship, we focus and scan in transmission through opaque media using reflected light. Our approach shows the versatility of NNs for light shaping, for efficiently and flexibly correcting for scattering, and in particular the feasibility of transmission control based on reflected light.

Two-photon calcium imaging from head-fixed Drosophila during optomotor walking behavior.

Drosophila melanogaster is a model organism rich in genetic tools to manipulate and identify neural circuits involved in specific behaviors. Here we present a technique for two-photon calcium imaging in the central brain of head-fixed Drosophila walking on an air-supported ball. The ball's motion is tracked at high resolution and can be treated as a proxy for the fly's own movements. We used the genetically encoded calcium sensor, GCaMP3.0, to record from important elements of the motion-processing pathway, the horizontal-system lobula plate tangential cells (LPTCs) in the fly optic lobe. We presented motion stimuli to the tethered fly and found that calcium transients in horizontal-system neurons correlated with robust optomotor behavior during walking. Our technique allows both behavior and physiology in identified neurons to be monitored in a genetic model organism with an extensive repertoire of walking behaviors.

Axial motion estimation and correction for simultaneous multi-plane two-photon calcium imaging.

Two-photon imaging in behaving animals is typically accompanied by brain motion. For functional imaging experiments, for example with genetically encoded calcium indicators, such brain motion induces changes in fluorescence intensity. These motion-related intensity changes or motion artifacts can typically not be separated from neural activity-induced signals. While lateral motion, within the focal plane, can be corrected by computationally aligning images, axial motion, out of the focal plane, cannot easily be corrected. Here, we developed an algorithm for axial motion correction for non-ratiometric calcium indicators taking advantage of simultaneous multi-plane imaging. Using temporally multiplexed beams, recording simultaneously from at least two focal planes at different z positions, and recording a z-stack for each beam as a calibration step, the algorithm separates motion-related and neural activity-induced changes in fluorescence intensity. The algorithm is based on a maximum likelihood optimisation approach; it assumes (as a first order approximation) that no distortions of the sample occurs during axial motion and that neural activity increases uniformly along the optical axis in each region of interest. The developed motion correction approach allows axial motion estimation and correction at high frame rates for isolated structures in the imaging volume in vivo, such as sparse expression patterns in the fruit fly brain.

A place learning assay for tethered walking Drosophila.

BACKGROUND: Drosophila shows a range of visually guided memory and learning behaviors, including place learning. Investigating the dynamics of neural circuits underlying such behaviors requires learning assays in tethered animals, compatible with in vivo imaging experiments. NEW METHOD: Here, we introduce an assay for place learning for tethered walking flies. A cylindrical arena is rotated and translated in real time around the fly in concert with the rotational and translational walking activity measured with an air supported ball, resulting in a mechanical virtual reality (VR). RESULTS: Navigation together with heat-based operant conditioning allows flies to learn the location of a cool spot with respect to a visual landmark. Flies optimize the time and distance required to find the cool spot over a similar number of trials as observed in assays with freely moving flies. Additionally, a fraction of flies remembers the location of the cool spot also after the conditioning heat is removed. COMPARISON WITH EXISTING METHODS: Learning tasks have been implemented in tethered flying as well as walking flies. Mechanically translating and rotating an arena in concert with the fly's walking activity enables navigation in a three dimensional environment. CONCLUSION: In the developed mechanical VR flies can learn to remember the location of a cool place within an otherwise hot environment with respect to a visual landmark. Implementing place learning in a tethered walking configuration is a precondition for investigating the underlying circuit dynamics using functional imaging.

Automated long-term two-photon imaging in head-fixed walking Drosophila.

BACKGROUND: The brain of Drosophila shows dynamics at multiple timescales, from the millisecond range of fast voltage or calcium transients to functional and structural changes occurring over multiple days. To relate such dynamics to behavior requires monitoring neural circuits across these multiple timescales in behaving animals. NEW METHOD: Here, we develop a technique for automated long-term two-photon imaging in fruit flies, during wakefulness and extended bouts of immobility, as typically observed during sleep, navigating in virtual reality over up to seven days. The method is enabled by laser surgery, a microrobotic arm for controlling forceps for dissection assistance, an automated feeding robot, as well as volumetric, simultaneous multiplane imaging. RESULTS: The approach is validated in the fly's head direction system and walking behavior as well a neural activity are recorded. The head direction system tracks the fly's walking direction over multiple days. COMPARISON WITH EXISTING METHODS: In comparison with previous head-fixed preparations, the time span over which tethered behavior and neural activity can be recorded at the same time is extended from hours to days. Additionally, the reproducibility and ease of dissections are improved compared with manual approaches. Different from previous laser surgery approaches, only continuous wave lasers are required. Lastly, an automated feeding system allows continuously maintaining the fly for several days in the virtual reality setup without user intervention. CONCLUSIONS: Imaging in behaving flies over multiple timescales will be useful for understanding circadian activity, learning and long-term memory, or sleep.

Virtual reality for animal navigation with camera-based optical flow tracking.

BACKGROUND: Virtual reality combined with a spherical treadmill is used across species for studying neural circuits underlying navigation and learning. NEW METHOD: We developed an optical flow-based method for tracking treadmill ball motion in real time using a single high-resolution camera. RESULTS: Tracking accuracy and timing were determined using calibration data. Ball tracking was performed at 500 Hz and integrated with an open source game engine for virtual reality projection. The projection was updated at 120 Hz with a latency with respect to ball motion of 30 ±â€¯8 ms. The system was tested for behavior with fruit flies. The application and source code are available at https://github.com/ivan-vishniakou/neural-circuits-vr. COMPARISON WITH EXISTING METHOD(S): Optical flow-based tracking of treadmill motion is typically achieved using optical mice. The camera-based optical flow tracking system developed here is based on off-the-shelf components and offers control over the image acquisition and processing parameters. This results in flexibility with respect to tracking conditions - such as ball surface texture, lighting conditions, or ball size - as well as camera alignment and calibration. CONCLUSIONS: A fast system for rotational ball motion tracking suitable for virtual reality behavior with fruit flies was developed and characterized.

Angular velocity integration in a fly heading circuit.

Many animals maintain an internal representation of their heading as they move through their surroundings. Such a compass representation was recently discovered in a neural population in the Drosophila melanogaster central complex, a brain region implicated in spatial navigation. Here, we use two-photon calcium imaging and electrophysiology in head-fixed walking flies to identify a different neural population that conjunctively encodes heading and angular velocity, and is excited selectively by turns in either the clockwise or counterclockwise direction. We show how these mirror-symmetric turn responses combine with the neurons' connectivity to the compass neurons to create an elegant mechanism for updating the fly's heading representation when the animal turns in darkness. This mechanism, which employs recurrent loops with an angular shift, bears a resemblance to those proposed in theoretical models for rodent head direction cells. Our results provide a striking example of structure matching function for a broadly relevant computation.

Video-rate volumetric functional imaging of the brain at synaptic resolution.

Neurons and neural networks often extend hundreds of micrometers in three dimensions. Capturing the calcium transients associated with their activity requires volume imaging methods with subsecond temporal resolution. Such speed is a challenge for conventional two-photon laser-scanning microscopy, because it depends on serial focal scanning in 3D and indicators with limited brightness. Here we present an optical module that is easily integrated into standard two-photon laser-scanning microscopes to generate an axially elongated Bessel focus, which when scanned in 2D turns frame rate into volume rate. We demonstrated the power of this approach in enabling discoveries for neurobiology by imaging the calcium dynamics of volumes of neurons and synapses in fruit flies, zebrafish larvae, mice and ferrets in vivo. Calcium signals in objects as small as dendritic spines could be resolved at video rates, provided that the samples were sparsely labeled to limit overlap in their axially projected images.

Studying sensorimotor processing with physiology in behaving Drosophila.

The neural underpinnings of sensorimotor integration are best studied in the context of well-characterized behavior. A rich trove of Drosophila behavioral genetics research offers a variety of well-studied behaviors and candidate brain regions that can form the bases of such studies. The development of tools to perform in vivo physiology from the Drosophila brain has made it possible to monitor activity in defined neurons in response to sensory stimuli. More recently still, it has become possible to perform recordings from identified neurons in the brain of head-fixed flies during walking or flight behaviors. In this chapter, we discuss how experiments that simultaneously monitor behavior and physiology in Drosophila can be combined with other techniques to produce testable models of sensorimotor circuit function.

Walking modulates speed sensitivity in Drosophila motion vision.

Changes in behavioral state modify neural activity in many systems. In some vertebrates such modulation has been observed and interpreted in the context of attention and sensorimotor coordinate transformations. Here we report state-dependent activity modulations during walking in a visual-motor pathway of Drosophila. We used two-photon imaging to monitor intracellular calcium activity in motion-sensitive lobula plate tangential cells (LPTCs) in head-fixed Drosophila walking on an air-supported ball. Cells of the horizontal system (HS)--a subgroup of LPTCs--showed stronger calcium transients in response to visual motion when flies were walking rather than resting. The amplified responses were also correlated with walking speed. Moreover, HS neurons showed a relatively higher gain in response strength at higher temporal frequencies, and their optimum temporal frequency was shifted toward higher motion speeds. Walking-dependent modulation of HS neurons in the Drosophila visual system may constitute a mechanism to facilitate processing of higher image speeds in behavioral contexts where these speeds of visual motion are relevant for course stabilization.