Neurons in Primate Area MSTd Signal Eye Movement Direction Inferred from Dynamic Perspective Cues in Optic Flow.

Smooth eye movements are common during natural viewing; we frequently rotate our eyes to track moving objects or to maintain fixation on an object during self-movement. Reliable information about smooth eye movements is crucial to various neural computations, such as estimating heading from optic flow or judging depth from motion parallax. While it is well established that extraretinal signals (e.g., efference copies of motor commands) carry critical information about eye velocity, the rotational optic flow field produced by eye rotations also carries valuable information. Although previous work has shown that dynamic perspective cues in optic flow can be used in computations that require estimates of eye velocity, it has remained unclear where and how the brain processes these visual cues and how they are integrated with extraretinal signals regarding eye rotation. We examined how neurons in the dorsal region of the medial superior temporal area (MSTd) of two male rhesus monkeys represent the direction of smooth pursuit eye movements based on both visual cues (dynamic perspective) and extraretinal signals. We find that most MSTd neurons have matched preferences for the direction of eye rotation based on visual and extraretinal signals. Moreover, neural responses to combinations of these signals are well predicted by a weighted linear summation model. These findings demonstrate a neural substrate for representing the velocity of smooth eye movements based on rotational optic flow and establish area MSTd as a key node for integrating visual and extraretinal signals into a more generalized representation of smooth eye movements.SIGNIFICANCE STATEMENT We frequently rotate our eyes to smoothly track objects of interest during self-motion. Information about eye velocity is crucial for a variety of computations performed by the brain, including depth perception and heading perception. Traditionally, information about eye rotation has been thought to arise mainly from extraretinal signals, such as efference copies of motor commands. Previous work shows that eye velocity can also be inferred from rotational optic flow that accompanies smooth eye movements, but the neural origins of these visual signals about eye rotation have remained unknown. We demonstrate that macaque neurons signal the direction of smooth eye rotation based on visual signals, and that they integrate both visual and extraretinal signals regarding eye rotation in a congruent fashion.

Neural sensitivity to translational self- and object-motion velocities.

The ability to detect and assess world-relative object-motion is a critical computation performed by the visual system. This computation, however, is greatly complicated by the observer's movements, which generate a global pattern of motion on the observer's retina. How the visual system implements this computation is poorly understood. Since we are potentially able to detect a moving object if its motion differs in velocity (or direction) from the expected optic flow generated by our own motion, here we manipulated the relative motion velocity between the observer and the object within a stationary scene as a strategy to test how the brain accomplishes object-motion detection. Specifically, we tested the neural sensitivity of brain regions that are known to respond to egomotion-compatible visual motion (i.e., egomotion areas: cingulate sulcus visual area, posterior cingulate sulcus area, posterior insular cortex [PIC], V6+, V3A, IPSmot/VIP, and MT+) to a combination of different velocities of visually induced translational self- and object-motion within a virtual scene while participants were instructed to detect object-motion. To this aim, we combined individual surface-based brain mapping, task-evoked activity by functional magnetic resonance imaging, and parametric and representational similarity analyses. We found that all the egomotion regions (except area PIC) responded to all the possible combinations of self- and object-motion and were modulated by the self-motion velocity. Interestingly, we found that, among all the egomotion areas, only MT+, V6+, and V3A were further modulated by object-motion velocities, hence reflecting their possible role in discriminating between distinct velocities of self- and object-motion. We suggest that these egomotion regions may be involved in the complex computation required for detecting scene-relative object-motion during self-motion.

Landmark knowledge overrides optic flow in honeybee waggle dance distance estimation.

Honeybees encode in their waggle dances the vector (distance and direction) of an outbound flight to a food source or a new nest site. Optic flow has been identified as the major source of information in the distance estimation. Additional components of distance estimation were also identified, e.g. the sequence of experienced landmarks. Here we address the question of whether bees also use the landscape memory developed during exploratory orientation flights to estimate distance. We took advantage of the fact that flights in a narrow tunnel lead to further distance measures due to higher optic flow. We find that this effect is lost when bees had explored the area in which the tunnel is located and when they have somewhat restricted visual access to the surrounding environment through the mesh on top of the tunnel. These data are interpreted in the context of other findings about the structure of navigational memory in bees that develops during exploratory orientation flights. In particular, the data suggest that bees embed distance measures into a representation of navigational space that stores previously experienced landscape features.

The influence of travel time on perceived traveled distance varies by spatiotemporal scale.

The influence of travel time on perceived traveled distance has often been studied, but the results are inconsistent regarding the relationship between the two magnitudes. We argue that this is due to differences in the lengths of investigated travel distances and hypothesize that the influence of travel time differs for rather short compared to rather long traveled distances. We tested this hypothesis in a virtual environment presented on a desktop as well as through a head-mounted display. Our results show that, for longer distances, more travel time leads to longer perceived distance, while we do not find an influence of travel time on shorter distances. The presentation through an HMD vs. desktop only influenced distance judgments in the short distance condition. These results are in line with the idea that the influence of travel time varies by the length of the traveled distance, and provide insights on the question of how distance perception in path integration studies is affected by travel time, thereby resolving inconsistencies reported in previous studies.

Tunnel motion: Pupil dilations to optic flow within illusory dark holes.

We showed to the same observers both dynamic and static 2D patterns that can both evoke distinctive perceptions of motion or optic flow, as if moving in a tunnel or into a dark hole. At all times pupil diameters were monitored with an infrared eye tracker. We found a converging set of results indicating stronger pupil dilations to expansive growth of shapes or optic flows evoking a forward motion into a dark tunnel. Multiple regression analyses showed that the pupil responses to the illusory expanding black holes of static patterns were predicted by the individuals' pupil response to optic flows showing spiraling motion or "free fall" into a black hole. Also, individuals' pupil responses to spiraling motion into dark tunnels predicted the individuals' sense of illusory expansion with the static, illusory expanding, dark holes. This correspondence across individuals between their pupil responses to both dynamic and static, illusory expanding, holes suggests that these percepts reflect a common perceptual mechanism, deriving motion from 2D scenes, and that the observers' pupil adjustments reflect the direction and strength of motion they perceive and the expected outcome of an increase in darkness.

From Multisensory Integration to Multisensory Decision-Making.

Organisms live in a dynamic environment in which sensory information from multiple sources is ever changing. A conceptually complex task for the organisms is to accumulate evidence across sensory modalities and over time, a process known as multisensory decision-making. This is a new concept, in terms of that previous researches have been largely conducted in parallel disciplines. That is, much efforts have been put either in sensory integration across modalities using activity summed over a duration of time, or in decision-making with only one sensory modality that evolves over time. Recently, a few studies with neurophysiological measurements emerge to study how different sensory modality information is processed, accumulated, and integrated over time in decision-related areas such as the parietal or frontal lobes in mammals. In this review, we summarize and comment on these studies that combine the long-existed two parallel fields of multisensory integration and decision-making. We show how the new findings provide insight into our understanding about neural mechanisms mediating multisensory information processing in a more complete way.

Facilitation of neural responses to targets moving against optic flow.

For the human observer, it can be difficult to follow the motion of small objects, especially when they move against background clutter. In contrast, insects efficiently do this, as evidenced by their ability to capture prey, pursue conspecifics, or defend territories, even in highly textured surrounds. We here recorded from target selective descending neurons (TSDNs), which likely subserve these impressive behaviors. To simulate the type of optic flow that would be generated by the pursuer's own movements through the world, we used the motion of a perspective corrected sparse dot field. We show that hoverfly TSDN responses to target motion are suppressed when such optic flow moves syn-directional to the target. Indeed, neural responses are strongly suppressed when targets move over either translational sideslip or rotational yaw. More strikingly, we show that TSDNs are facilitated by optic flow moving counterdirectional to the target, if the target moves horizontally. Furthermore, we show that a small, frontal spatial window of optic flow is enough to fully facilitate or suppress TSDN responses to target motion. We argue that such TSDN response facilitation could be beneficial in modulating corrective turns during target pursuit.

Cortical Motion Perception Emerges from Dimensionality Reduction with Evolved Spike-Timing-Dependent Plasticity Rules.

The nervous system is under tight energy constraints and must represent information efficiently. This is particularly relevant in the dorsal part of the medial superior temporal area (MSTd) in primates where neurons encode complex motion patterns to support a variety of behaviors. A sparse decomposition model based on a dimensionality reduction principle known as non-negative matrix factorization (NMF) was previously shown to account for a wide range of monkey MSTd visual response properties. This model resulted in sparse, parts-based representations that could be regarded as basis flow fields, a linear superposition of which accurately reconstructed the input stimuli. This model provided evidence that the seemingly complex response properties of MSTd may be a by-product of MSTd neurons performing dimensionality reduction on their input. However, an open question is how a neural circuit could carry out this function. In the current study, we propose a spiking neural network (SNN) model of MSTd based on evolved spike-timing-dependent plasticity and homeostatic synaptic scaling (STDP-H) learning rules. We demonstrate that the SNN model learns compressed and efficient representations of the input patterns similar to the patterns that emerge from NMF, resulting in MSTd-like receptive fields observed in monkeys. This SNN model suggests that STDP-H observed in the nervous system may be performing a similar function as NMF with sparsity constraints, which provides a test bed for mechanistic theories of how MSTd may efficiently encode complex patterns of visual motion to support robust self-motion perception.SIGNIFICANCE STATEMENT The brain may use dimensionality reduction and sparse coding to efficiently represent stimuli under metabolic constraints. Neurons in monkey area MSTd respond to complex optic flow patterns resulting from self-motion. We developed a spiking neural network model that showed MSTd-like response properties can emerge from evolving spike-timing-dependent plasticity with STDP-H parameters of the connections between then middle temporal area and MSTd. Simulated MSTd neurons formed a sparse, reduced population code capable of encoding perceptual variables important for self-motion perception. This model demonstrates that complex neuronal responses observed in MSTd may emerge from efficient coding and suggests that neurobiological plasticity, like STDP-H, may contribute to reducing the dimensions of input stimuli and allowing spiking neurons to learn sparse representations.

From the eye to the wing: neural circuits for transforming optic flow into motor output in avian flight.

Avian flight is guided by optic flow-the movement across the retina of images of surfaces and edges in the environment due to self-motion. In all vertebrates, there is a short pathway for optic flow information to reach pre-motor areas: retinal-recipient regions in the midbrain encode optic flow, which is then sent to the cerebellum. One well-known role for optic flow pathways to the cerebellum is the control of stabilizing eye movements (the optokinetic response). However, the role of this pathway in controlling locomotion is less well understood. Electrophysiological and tract tracing studies are revealing the functional connectivity of a more elaborate circuit through the avian cerebellum, which integrates optic flow with other sensory signals. Here we review the research supporting this framework and identify the cerebellar output centres, the lateral (CbL) and medial (CbM) cerebellar nuclei, as two key nodes with potentially distinct roles in flight control. The CbM receives bilateral optic flow information and projects to sites in the brainstem that suggest a primary role for flight control over time, such as during forward flight. The CbL receives monocular optic flow and other types of visual information. This site provides feedback to sensory areas throughout the brain and has a strong projection the nucleus ruber, which is known to have a dominant role in forelimb muscle control. This arrangement suggests primary roles for the CbL in the control of wing morphing and for rapid maneuvers.

The Presidential Symposium at the International Congress of Neuroethology 2022 in Lisbon, Portugal.

In this special issue of articles from leading neuroethologists-all of whom gave outstanding presentations within the Presidential Symposium of the 2022 International Congress of Neuroethology held in Lisbon, Portugal-we learn about the role of cryptochrome molecules in the magnetic sense of animals, how honeybees construct their honeycombs, why fish eyes are built the way they are in species from different depths, how archerfish intercept their newly downed prey with a swift muscular curving of the body (known as a C-start) and how birds process optic flow information to control flight. Each contribution showcases how nervous systems have evolved to control behaviour, the raison d'être of neuroethology.

Optic flow based spatial vision in insects.

The optic flow, i.e., the displacement of retinal images of objects in the environment induced by self-motion, is an important source of spatial information, especially for fast-flying insects. Spatial information over a wide range of distances, from the animal's immediate surroundings over several hundred metres to kilometres, is necessary for mediating behaviours, such as landing manoeuvres, collision avoidance in spatially complex environments, learning environmental object constellations and path integration in spatial navigation. To facilitate the processing of spatial information, the complexity of the optic flow is often reduced by active vision strategies. These result in translations and rotations being largely separated by a saccadic flight and gaze mode. Only the translational components of the optic flow contain spatial information. In the first step of optic flow processing, an array of local motion detectors provides a retinotopic spatial proximity map of the environment. This local motion information is then processed in parallel neural pathways in a task-specific manner and used to control the different components of spatial behaviour. A particular challenge here is that the distance information extracted from the optic flow does not represent the distances unambiguously, but these are scaled by the animal's speed of locomotion. Possible ways of coping with this ambiguity are discussed.

Mallard landing behavior on water follows a -constant braking strategy.

Many flying animals use optic flow to control their flight. During landing maneuvers, pigeons, hummingbirds, bats, Draco lizards and bees use the -constant braking strategy. This strategy regulates the approach by keeping the ratio of distance to an object and the rate of change of that distance constant. In keeping this ratio, , constant, a variety of deceleration profiles can lead to different collision avoidance behaviors. The landing behaviors listed above all qualify as controlled collisions, where the animal is decelerating into the object. We examined whether the same regulatory strategy is employed by mallards when landing on water. Video of mallard landing behavior was recorded at a local pond and digitized. Kinematic and τ parameters were calculated for each landing (N=177). The Pearson correlation coefficient for τ with respect to time to land was 0.99±0.02, indicating mallards employ a controlled-collision strategy. This result implies regulation by the birds to fix as constant while landing (on average, 0.90±0.13). In comparison with other active flyers, mallards use a higher value of when landing (0.775±0.109, 0.710±0.132 and 0.702±0.052 for pigeons, hummingbirds and bats, respectively). This higher may reflect physical differences in substrate from solid to liquid. The higher compliance of water in comparison to a solid substrate may reduce impact forces that could be injurious on a solid substrate, thereby enabling mallards to approach faster and expend less energy for costly, slow flight.

A mental workload and biomechanical assessment during split-belt locomotor adaptation with and without optic flow.

Adaptive human performance relies on the central nervous system to regulate the engagement of cognitive-motor resources as task demands vary. Despite numerous studies which employed a split-belt induced perturbation to examine biomechanical outcomes during locomotor adaptation, none concurrently examined the cerebral cortical dynamics to assess changes in mental workload. Additionally, while prior work suggests that optic flow provides critical information for walking regulation, a few studies have manipulated visual inputs during adaption to split-belt walking. This study aimed to examine the concurrent modulation of gait and Electroencephalography (EEG) cortical dynamics underlying mental workload during split-belt locomotor adaptation, with and without optic flow. Thirteen uninjured participants with minimal inherent walking asymmetries at baseline underwent adaptation, while temporal-spatial gait and EEG spectral metrics were recorded. The results revealed a reduction in step length and time asymmetry from early to late adaptation, accompanied by an elevated frontal and temporal theta power; the former being well corelated to biomechanical changes. While the absence of optic flow during adaptation did not affect temporal-spatial gait metrics, it led to an increase of theta and low-alpha power. Thus, as individuals adapt their locomotor patterns, the cognitive-motor resources underlying the encoding and consolidation processes of the procedural memory were recruited to acquire a new internal model of the perturbation. Also, when adaption occurs without optic flow, a further reduction of arousal is accompanied with an elevation of attentional engagement due to enhanced neurocognitive resources likely to maintain adaptive walking patterns.

Increased variability but intact integration during visual navigation in Autism Spectrum Disorder.

Autism Spectrum Disorder (ASD) is a common neurodevelopmental disturbance afflicting a variety of functions. The recent computational focus suggesting aberrant Bayesian inference in ASD has yielded promising but conflicting results in attempting to explain a wide variety of phenotypes by canonical computations. Here, we used a naturalistic visual path integration task that combines continuous action with active sensing and allows tracking of subjects' dynamic belief states. Both groups showed a previously documented bias pattern by overshooting the radial distance and angular eccentricity of targets. For both control and ASD groups, these errors were driven by misestimated velocity signals due to a nonuniform speed prior rather than imperfect integration. We tracked participants' beliefs and found no difference in the speed prior, but there was heightened variability in the ASD group. Both end point variance and trajectory irregularities correlated with ASD symptom severity. With feedback, variance was reduced, and ASD performance approached that of controls. These findings highlight the need for both more naturalistic tasks and a broader computational perspective to understand the ASD phenotype and pathology.

Humans navigate with stereo olfaction.

Human navigation relies on inputs to our paired eyes and ears. Although we also have two nasal passages, there has been little empirical indication that internostril differences yield directionality in human olfaction without involving the trigeminal system. By using optic flow that captures the pattern of apparent motion of surface elements in a visual scene, we demonstrate through formal psychophysical testing that a moderate binaral concentration disparity of a nontrigeminal odorant consistently biases recipients' perceived direction of self-motion toward the higher-concentration side, despite that they cannot verbalize which nostril smells a stronger odor. We further show that the effect depends on the internostril ratio of odor concentrations and not the numeric difference in concentration between the two nostrils. Taken together, our findings provide behavioral evidence that humans smell in stereo and subconsciously utilize stereo olfactory cues in spatial navigation.

Heading perception depends on time-varying evolution of optic flow.

There is considerable support for the hypothesis that perception of heading in the presence of rotation is mediated by instantaneous optic flow. This hypothesis, however, has never been tested. We introduce a method, termed "nonvarying phase motion," for generating a stimulus that conveys a single instantaneous optic flow field, even though the stimulus is presented for an extended period of time. In this experiment, observers viewed stimulus videos and performed a forced-choice heading discrimination task. For nonvarying phase motion, observers made large errors in heading judgments. This suggests that instantaneous optic flow is insufficient for heading perception in the presence of rotation. These errors were mostly eliminated when the velocity of phase motion was varied over time to convey the evolving sequence of optic flow fields corresponding to a particular heading. This demonstrates that heading perception in the presence of rotation relies on the time-varying evolution of optic flow. We hypothesize that the visual system accurately computes heading, despite rotation, based on optic acceleration, the temporal derivative of optic flow.

Virtual reality-enhanced walking in people post-stroke: effect of optic flow speed and level of immersion on the gait biomechanics.

BACKGROUND: Optic flow-the apparent visual motion experienced while moving-is absent during treadmill walking. With virtual reality (VR), optic flow can be controlled to mediate alterations in human walking. The aim of this study was to investigate (1) the effects of fully immersive VR and optic flow speed manipulation on gait biomechanics, simulator sickness, and enjoyment in people post-stroke and healthy people, and (2) the effects of the level of immersion on optic flow speed and sense of presence. METHODS: Sixteen people post-stroke and 16 healthy controls performed two VR-enhanced treadmill walking sessions: the semi-immersive GRAIL session and fully immersive head-mounted display (HMD) session. Both consisted of five walking trials. After two habituation trials (without and with VR), participants walked three more trials under the following conditions: matched, slow, and fast optic flow. Primary outcome measures were spatiotemporal parameters and lower limb kinematics. Secondary outcomes (simulator sickness, enjoyment, and sense of presence) were assessed with the Simulator Sickness Questionnaire, Visual Analogue Scales, and Igroup Presence Questionnaire. RESULTS: When walking with the immersive HMD, the stroke group walked with a significantly slower cadence (-3.69strides/min, p = 0.006), longer stride time (+ 0.10 s, p = 0.017) and stance time for the unaffected leg (+ 1.47%, p = 0.001) and reduced swing time for the unaffected leg (- 1.47%, p = 0.001). Both groups responded to the optic flow speed manipulation such that people accelerated with a slow optic flow and decelerated with a fast optic flow. Compared to the semi-immersive GRAIL session, manipulating the optic flow speed with the fully immersive HMD had a greater effect on gait biomechanics whilst also eliciting a higher sense of presence. CONCLUSION: Adding fully immersive VR while walking on a self-paced treadmill led to a more cautious gait pattern in people post-stroke. However, walking with the HMD was well tolerated and enjoyable. People post-stroke altered their gait parameters when optic flow speed was manipulated and showed greater alterations with the fully-immersive HMD. Further work is needed to determine the most effective type of optic flow speed manipulation as well as which other principles need to be implemented to positively influence the gait pattern of people post-stroke. TRIAL REGISTRATION NUMBER: The study was pre-registered at ClinicalTrials.gov (NCT04521829).

FlyDetector-Automated Monitoring Platform for the Visual-Motor Coordination of Honeybees in a Dynamic Obstacle Scene Using Digital Paradigm.

Vision plays a crucial role in the ability of compound-eyed insects to perceive the characteristics of their surroundings. Compound-eyed insects (such as the honeybee) can change the optical flow input of the visual system by autonomously controlling their behavior, and this is referred to as visual-motor coordination (VMC). To analyze an insect's VMC mechanism in dynamic scenes, we developed a platform for studying insects that actively shape the optic flow of visual stimuli by adapting their flight behavior. Image-processing technology was applied to detect the posture and direction of insects' movement, and automatic control technology provided dynamic scene stimulation and automatic acquisition of perceptual insect behavior. In addition, a virtual mapping technique was used to reconstruct the visual cues of insects for VMC analysis in a dynamic obstacle scene. A simulation experiment at different target speeds of 1-12 m/s was performed to verify the applicability and accuracy of the platform. Our findings showed that the maximum detection speed was 8 m/s, and triggers were 95% accurate. The outdoor experiments showed that flight speed in the longitudinal axis of honeybees was more stable when facing dynamic barriers than static barriers after analyzing the change in geometric optic flow. Finally, several experiments showed that the platform can automatically and efficiently monitor honeybees' perception behavior, and can be applied to study most insects and their VMC.

Response properties of optic flow neurons in the accessory optic system of hummingbirds versus zebra finches and pigeons.

Optokinetic responses function to maintain retinal image stabilization by minimizing optic flow that occurs during self-motion. The hovering ability of hummingbirds is an extreme example of this behavior. Optokinetic responses are mediated by direction-selective neurons with large receptive fields in the accessory optic system (AOS) and pretectum. Recent studies in hummingbirds showed that, compared with other bird species, 1) the pretectal nucleus lentiformis mesencephali (LM) is hypertrophied, 2) LM has a unique distribution of direction preferences, and 3) LM neurons are more tightly tuned to stimulus velocity. In this study, we sought to determine if there are concomitant changes in the nucleus of the basal optic root (nBOR) of the AOS. We recorded the visual response properties of nBOR neurons to large-field-drifting random dot patterns and sine-wave gratings in Anna's hummingbirds and zebra finches and compared these with archival data from pigeons. We found no differences with respect to the distribution of direction preferences: Neurons responsive to upward, downward, and nasal-to-temporal motion were equally represented in all three species, and neurons responsive to temporal-to-nasal motion were rare or absent (<5%). Compared with zebra finches and pigeons, however, hummingbird nBOR neurons were more tightly tuned to stimulus velocity of random dot stimuli. Moreover, in response to drifting gratings, hummingbird nBOR neurons are more tightly tuned in the spatiotemporal domain. These results, in combination with specialization in LM, support a hypothesis that hummingbirds have evolved to be "optic flow specialists" to cope with the optomotor demands of sustained hovering flight.NEW & NOTEWORTHY Hummingbirds have specialized response properties to optic flow in the pretectal nucleus lentiformis mesencephali (LM). The LM works with the nucleus of the basal optic root (nBOR) of the accessory optic system (AOS) to process global visual motion, but whether the neural response specializations observed in the LM extend to the nBOR is unknown. Hummingbird nBOR neurons are more tightly tuned to visual stimulus velocity, and in the spatiotemporal domain, compared with two nonhovering species.

Distance estimation in the goldfish (Carassius auratus).

Neurophysiological advances have given us exciting insights into the systems responsible for spatial mapping in mammals. However, we are still lacking information on the evolution of these systems and whether the underlying mechanisms identified are universal across phyla, or specific to the species studied. Here we address these questions by exploring whether a species that is evolutionarily distant from mammals can perform a task central to mammalian spatial mapping-distance estimation. We developed a behavioural paradigm allowing us to test whether goldfish (Carassius auratus) can estimate distance and explored the behavioural mechanisms that underpin this ability. Fish were trained to swim a set distance within a narrow tank covered with a striped pattern. After changing the background pattern, we found that goldfish use the spatial frequency of their visual environment to estimate distance, doubling the spatial frequency of the background pattern resulted in a large overestimation of the swimming distance. We present robust evidence that goldfish can accurately estimate distance and show that they use local optic flow to do so. These results provide a compelling basis to use goldfish as a model system to interrogate the evolution of the mechanisms that underpin spatial cognition, from brain to behaviour.