V-NeuroStack: Open-source 3D time stack software for identifying patterns in neuronal data.

Understanding functional correlations between the activities of neuron populations is vital for the analysis of neuronal networks. Analyzing large-scale neuroimaging data obtained from hundreds of neurons simultaneously poses significant visualization challenges. We developed V-NeuroStack, a novel network visualization tool to visualize data obtained using calcium imaging of spontaneous activity of neurons in a mouse brain slice as well as in vivo using two-photon imaging. V-NeuroStack creates 3D time stacks by stacking 2D time frames for a time-series dataset. It provides a web interface to explore and analyze data using both 3D and 2D visualization techniques. Previous attempts to analyze such data have been limited by the tools available to visualize large numbers of correlated activity traces. V-NeuroStack's 3D view is used to explore patterns in dynamic large-scale correlations between neurons over time. The 2D view is used to examine any timestep of interest in greater detail. Furthermore, a dual-line graph provides the ability to explore the raw and first-derivative values of activity from an individual or a functional cluster of neurons. V-NeuroStack can scale to datasets with at least a few thousand temporal snapshots. It can potentially support future advancements in in vitro and in vivo data capturing techniques to bring forth novel hypotheses by allowing unambiguous visualization of massive patterns in neuronal activity data.

RemBrain: Exploring Dynamic Biospatial Networks with Mosaic Matrices and Mirror Glyphs.

We introduce a web-based visual comparison approach for the systematic exploration of dynamic activation networks across biological datasets. Understanding the dynamics of such networks in the context of demographic factors like age is a fundamental problem in computational systems biology and neuroscience. We design visual encodings for the dynamic and community characteristics of these temporal networks. Our multi-scale approach blends nested mosaic matrices that capture temporal characteristics of the data, spatial views of the network data, Kiviat diagrams and mirror glyphs that detail the temporal behavior and community assignment of specific nodes. A top design specifically targeted at pairwise visual comparison further supports the comparative analysis of multiple dataset activations. We demonstrate the effectiveness of this approach through a case study on mouse brain network data. Domain expert feedback indicates this approach can help identify trends and anomalies in the data.

Mirror versus parallel bimanual reaching.

BACKGROUND: In spite of their importance to everyday function, tasks that require both hands to work together such as lifting and carrying large objects have not been well studied and the full potential of how new technology might facilitate recovery remains unknown. METHODS: To help identify the best modes for self-teleoperated bimanual training, we used an advanced haptic/graphic environment to compare several modes of practice. In a 2-by-2 study, we compared mirror vs. parallel reaching movements, and also compared veridical display to one that transforms the right hand's cursor to the opposite side, reducing the area that the visual system has to monitor. Twenty healthy, right-handed subjects (5 in each group) practiced 200 movements. We hypothesized that parallel reaching movements would be the best performing, and attending to one visual area would reduce the task difficulty. RESULTS: The two-way comparison revealed that mirror movement times took an average 1.24 s longer to complete than parallel. Surprisingly, subjects' movement times moving to one target (attending to one visual area) also took an average of 1.66 s longer than subjects moving to two targets. For both hands, there was also a significant interaction effect, revealing the lowest errors for parallel movements moving to two targets (p < 0.001). This was the only group that began and maintained low errors throughout training. CONCLUSION: Combined with other evidence, these results suggest that the most intuitive reaching performance can be observed with parallel movements with a veridical display (moving to two separate targets). These results point to the expected levels of challenge for these bimanual training modes, which could be used to advise therapy choices in self-neurorehabilitation.

Self versus environment motion in postural control.

To stabilize our position in space we use visual information as well as non-visual physical motion cues. However, visual cues can be ambiguous: visually perceived motion may be caused by self-movement, movement of the environment, or both. The nervous system must combine the ambiguous visual cues with noisy physical motion cues to resolve this ambiguity and control our body posture. Here we have developed a Bayesian model that formalizes how the nervous system could solve this problem. In this model, the nervous system combines the sensory cues to estimate the movement of the body. We analytically demonstrate that, as long as visual stimulation is fast in comparison to the uncertainty in our perception of body movement, the optimal strategy is to weight visually perceived movement velocities proportional to a power law. We find that this model accounts for the nonlinear influence of experimentally induced visual motion on human postural behavior both in our data and in previously published results.

A novel dynamic network imaging analysis method reveals aging-related fragmentation of cortical networks in mouse.

Network analysis of large-scale neuroimaging data is a particularly challenging computational problem. Here, we adapt a novel analytical tool, the community dynamic inference method (CommDy), for brain imaging data from young and aged mice. CommDy, which was inspired by social network theory, has been successfully used in other domains in biology; this report represents its first use in neuroscience. We used CommDy to investigate aging-related changes in network metrics in the auditory and motor cortices by using flavoprotein autofluorescence imaging in brain slices and in vivo. We observed that auditory cortical networks in slices taken from aged brains were highly fragmented compared to networks observed in young animals. CommDy network metrics were then used to build a random-forests classifier based on NMDA receptor blockade data, which successfully reproduced the aging findings, suggesting that the excitatory cortical connections may be altered during aging. A similar aging-related decline in network connectivity was also observed in spontaneous activity in the awake motor cortex, suggesting that the findings in the auditory cortex reflect general mechanisms during aging. These data suggest that CommDy provides a new dynamic network analytical tool to study the brain and that aging is associated with fragmentation of intracortical networks.

Identifying the control of physically and perceptually evoked sway responses with coincident visual scene velocities and tilt of the base of support.

In this study, we have explored whether the impact of visual information on postural reactions is due to the same perceptual mechanisms that produce vection. Pitch motion of the visual field was presented at varying velocities to eight healthy subjects (29.9+/-2.8 years) standing quietly on a stationary base of support or receiving a 3 degrees toes-up tilt of the base of support. An infrared motion system recorded markers placed on body segments to record angular displacement of head and ankle and calculate whole body center of mass. Onset of the visual field motion and base of support movement were synchronized in all trials. We found that in the first 2 s following onset of visual field motion, both direction and amplitude of the linear displacement of whole body center of mass and angular displacement of the head, hip, and ankle were modulated by the velocity of visual scene motion. When the visual scene rotated in upward pitch, subjects overshot their initial vertical position with amplitudes that increased as velocity of the visual field increased. This behavior was even more evident when the base of support was tilted. These responses were much shorter than those observed in studies of vection. The dependence of the postural response amplitudes on the velocity of the visual field suggests, however, that there might be well-shared control pathways for visual influences on postural reactions and postural sway elicited by an illusion of self-motion.

Signal Propagation via Open-Loop Intrathalamic Architectures: A Computational Model.

Propagation of signals across the cerebral cortex is a core component of many cognitive processes and is generally thought to be mediated by direct intracortical connectivity. The thalamus, by contrast, is considered to be devoid of internal connections and organized as a collection of parallel inputs to the cortex. Here, we provide evidence that "open-loop" intrathalamic pathways involving the thalamic reticular nucleus (TRN) can support propagation of oscillatory activity across the cortex. Recent studies support the existence of open-loop thalamo-reticulo-thalamic (TC-TRN-TC) synaptic motifs in addition to traditional closed-loop architectures. We hypothesized that open-loop structural modules, when connected in series, might underlie thalamic and, therefore cortical, signal propagation. Using a supercomputing platform to simulate thousands of permutations of a thalamocortical network based on physiological data collected in mice, rats, ferrets, and cats and in which select synapses were allowed to vary both by class and individually, we evaluated the relative capacities of closed-loop and open-loop TC-TRN-TC synaptic configurations to support both propagation and oscillation. We observed that (1) signal propagation was best supported in networks possessing strong open-loop TC-TRN-TC connectivity; (2) intrareticular synapses were neither primary substrates of propagation nor oscillation; and (3) heterogeneous synaptic networks supported more robust propagation of oscillation than their homogeneous counterparts. These findings suggest that open-loop, heterogeneous intrathalamic architectures might complement direct intracortical connectivity to facilitate cortical signal propagation.

Effects of roll visual motion on online control of arm movement: reaching within a dynamic virtual environment.

Reaching toward a visual target involves the transformation of visual information into appropriate motor commands. Complex movements often occur either while we are moving or when objects in the world move around us, thus changing the spatial relationship between our hand and the space in which we plan to reach. This study investigated whether rotation of a wide field-of-view immersive scene produced by a virtual environment affected online visuomotor control during a double-step reaching task. A total of 20 seated healthy subjects reached for a visual target that remained stationary in space or unpredictably shifted to a second position (either to the right or left of its initial position) with different inter-stimulus intervals. Eleven subjects completed two experiments which were similar except for the duration of the target's appearance. The final target was either visible throughout the entire trial or only for a period of 200 ms. Movements were performed under two visual field conditions: the virtual scene was matched to the subject's head motion or rolled about the line of sight counterclockwise at 130 degrees/s. Nine additional subjects completed a third experiment in which the direction of the rolling scene was manipulated (i.e., clockwise and counterclockwise). Our results showed that while all subjects were able to modify their hand trajectory in response to the target shift with both visual scenes, some of the double-step movements contained a pause prior to modifying trajectory direction. Furthermore, our findings indicated that both the timing and kinematic adjustments of the reach were affected by roll motion of the scene. Both planning and execution of the reach were affected by roll motion. Changes in proportion of trajectory types, and significantly longer pauses that occurred during the reach in the presence of roll motion suggest that background roll motion mainly interfered with the ability to update the visuomotor response to the target displacement. Furthermore, the reaching movement was affected differentially by the direction of roll motion. Subjects demonstrated a stronger effect of visual motion on movements taking place in the direction of visual roll (e.g., leftward movements during counterclockwise roll). Further investigation of the hand path revealed significant changes during roll motion for both the area and shape of the 95% tolerance ellipses that were constructed from the hand position following the main movement termination. These changes corresponded with a hand drift that would suggest that subjects were relying more on proprioceptive information to estimate the arm position in space during roll motion of the visual field. We conclude that both the spatial and temporal kinematics of the reach movement were affected by the motion of the visual field, suggesting interference with the ability to simultaneously process two consecutive stimuli.

Postural and spatial orientation driven by virtual reality.

Orientation in space is a perceptual variable intimately related to postural orientation that relies on visual and vestibular signals to correctly identify our position relative to vertical. We have combined a virtual environment with motion of a posture platform to produce visual-vestibular conditions that allow us to explore how motion of the visual environment may affect perception of vertical and, consequently, affect postural stabilizing responses. In order to involve a higher level perceptual process, we needed to create a visual environment that was immersive. We did this by developing visual scenes that possess contextual information using color, texture, and 3-dimensional structures. Update latency of the visual scene was close to physiological latencies of the vestibulo-ocular reflex. Using this system we found that even when healthy young adults stand and walk on a stable support surface, they are unable to ignore wide field of view visual motion and they adapt their postural orientation to the parameters of the visual motion. Balance training within our environment elicited measurable rehabilitation outcomes. Thus we believe that virtual environments can serve as a clinical tool for evaluation and training of movement in situations that closely reflect conditions found in the physical world.

Error-augmented bimanual therapy for stroke survivors.

BACKGROUND: Stroke recovery studies have shown the efficacy of bimanual training on upper limb functional recovery and others have shown the efficacy of feedback technology that augments error. OBJECTIVE: In a double-blinded randomized controlled study (N = 26), we evaluated the short-term effects of bilateral arm training to foster functional recovery of a hemiparetic arm, with half of our subjects unknowingly also receiving error augmentation (where errors were visually and haptically enhanced by a robot). METHODS: Twenty-six individuals with chronic stroke were randomly assigned to practice an equivalent amount of bimanual reaching either with or without error augmentation. Participants were instructed to coordinate both arms while reaching to two targets (one for each arm) in three 45-minute treatments per week for two weeks, with a follow-up visit after one week without treatment. RESULTS: Subjects' 2-week gains in Fugl-Meyer score averaged 2.92, and we also observed improvements Wolf Motor Functional Ability Scale average 0.21, and Motor Activity Log of 0.58 for quantity and 0.63 for quality of life scores. The extra benefit of error augmentation over the three weeks became apparent in Fugl-Meyer score only after removing an outlier from consideration. CONCLUSIONS: This modest advantage of error augmentation was detectable over a short interval encouraging further research in interactive self-rehabilitation systems that can enhance error motor recovery.

Field of view and base of support width influence postural responses to visual stimuli during quiet stance.

We explored the destabilizing effect of visual field motion as the base of support (BOS) and the field of view (FOV) were narrowed. Visual field motion was achieved using an immersive virtual environment (scene) that moved realistically with head motion (natural motion) and translated sinusoidally at 0.1Hz in the fore-aft direction (augmented motion). Natural motion was presented in stereo while augmented motion was presented in both stereo and non-stereo. Subjects viewed scene motion under wide (90 degrees and 55 degrees in the horizontal and vertical directions) and narrow (25 degrees in both directions) FOV conditions while standing flatfooted (100% BOS) and on two blocks (45% and 35% BOS). Head and whole body center of mass (COM) and ankle angle root mean square (RMS) were determined as were head, whole body, and shank COM FFTs. During natural motion, the primary effect emerged in the head RMS which was significantly smaller with a 35% BOS and the wide FOV compared to the narrow FOV. However, the primary effect of augmented motion emerged in the power analysis of head and whole body COM which significantly increased with the wide FOV for a 35% BOS compared to 100% BOS. Statistical analysis indicated an effect of BOS on depth perception for head and whole body RMS; however, post hoc comparisons revealed no significant differences between stereo and non-stereo augmented motion. We conclude that reducing the BOS increased reliance on peripheral visual information to stabilize the head in space even when the augmented visual motion promoted postural instability.

Reaching within a dynamic virtual environment.

BACKGROUND: Planning and execution of reaching requires a series of computational processes that involve localization of both the target and initial arm position, and the translation of this spatial information into appropriate motor commands that bring the hand to the target. We have investigated the effects of shifting the visual field on visuomotor control using a virtual visual environment in order to determine how changes in visuo-spatial relations alter motor planning during a reach. METHODS: Five healthy subjects were seated in front of an immersive, stereo virtual scene while reaching for a visual target that remained stationary in space or unpredictably shifted to a second position (either to the right or left of the first target) with different inter-stimulus intervals. Motion of the scene either matched the motion of their head or was rotated counter clockwise at 130 deg/s in the roll plane. RESULTS: Initial results suggested that both the temporal and spatial aspects of reaching were affected by a rolling visual field. Subjects were able to amend ongoing motion to match target position regardless of scene motion, but the presence of visual field motion produced significantly longer pauses during the reach movement when the target was shifted in space. In addition, terminal arm posture exhibited a drift in the direction opposite to the roll motion. CONCLUSION: These findings suggest that roll motion of the visual field of view interfered with the ability to imultaneously process two consecutive stimuli. Observed changes in arm position following the termination of the reach suggest that subjects were compensating for a perceived change in their visual reference frame.

Hand rehabilitation following stroke: a pilot study of assisted finger extension training in a virtual environment.

BACKGROUND AND PURPOSE: The purpose of this pilot study was to investigate the impact of assisted motor training in a virtual environment on hand function in stroke survivors. PARTICIPANTS: Fifteen volunteer stroke survivors (32-88 years old) with chronic upper extremity hemiparesis (1-38 years post incident) took part. METHOD: Participants had 6 weeks of training in reach-to-grasp of virtual and actual objects. They were randomized to one of three groups: assistance of digit extension provided by a novel cable orthosis, assistance provided by a novel pneumatic orthosis, or no assistance provided. Hand performance was evaluated at baseline, immediately following training, and 1 month after completion of training. Clinical assessments included the Wolf Motor Function Test (WMFT), Box and Blocks Test (BB), Upper Extremity Fugl-Meyer Test (FM), and Rancho Los Amigos Functional Test of the Hemiparetic Upper Extremity (RLA). Biomechanical assessments included grip strength, extension range of motion and velocity, spasticity, and isometric strength. RESULTS: Participants demonstrated a significant decrease in time to perform functional tasks for the WMFT (p = .02), an increase in the number of blocks successfully grasped and released during the BB (p = .09), and an increase for the FM score (p = .08). There were no statistically significant changes in time to complete tasks on the RLA or any of the biomechanical measures. Assistance of extension did not have a significant effect. DISCUSSION AND CONCLUSION: After the training period, participants in all 3 groups demonstrated a decrease in time to perform some of the functional tasks. Although the overall gains were slight, the general acceptance of the novel rehabilitation tools by a population with substantial impairment suggests that a larger randomized controlled trial, potentially in a subacute population, may be warranted.

Considerations for the future development of virtual technology as a rehabilitation tool.

BACKGROUND: Virtual environments (VE) are a powerful tool for various forms of rehabilitation. Coupling VE with high-speed networking [Tele-Immersion] that approaches speeds of 100 Gb/sec can greatly expand its influence in rehabilitation. Accordingly, these new networks will permit various peripherals attached to computers on this network to be connected and to act as fast as if connected to a local PC. This innovation may soon allow the development of previously unheard of networked rehabilitation systems. Rapid advances in this technology need to be coupled with an understanding of how human behavior is affected when immersed in the VE. METHODS: This paper will discuss various forms of VE that are currently available for rehabilitation. The characteristic of these new networks and examine how such networks might be used for extending the rehabilitation clinic to remote areas will be explained. In addition, we will present data from an immersive dynamic virtual environment united with motion of a posture platform to record biomechanical and physiological responses to combined visual, vestibular, and proprioceptive inputs. A 6 degree-of-freedom force plate provides measurements of moments exerted on the base of support. Kinematic data from the head, trunk, and lower limb was collected using 3-D video motion analysis. RESULTS: Our data suggest that when there is a confluence of meaningful inputs, neither vision, vestibular, or proprioceptive inputs are suppressed in healthy adults; the postural response is modulated by all existing sensory signals in a non-additive fashion. Individual perception of the sensory structure appears to be a significant component of the response to these protocols and underlies much of the observed response variability. CONCLUSION: The ability to provide new technology for rehabilitation services is emerging as an important option for clinicians and patients. The use of data mining software would help analyze the incoming data to provide both the patient and the therapist with evaluation of the current treatment and modifications needed for future therapies. Quantification of individual perceptual styles in the VE will support development of individualized treatment programs. The virtual environment can be a valuable tool for therapeutic interventions that require adaptation to complex, multimodal environments.

Postural responses exhibit multisensory dependencies with discordant visual and support surface motion.

The purpose of this study was to identify how the postural system weights coincident yet discordant disturbances of the visual and proprioceptive/vestibular systems. Eleven healthy subjects (25-38 yrs) received either fore-aft translations of an immersive, wide field-of-view visual environment (0.1 Hz, +/- 3.7 m/sec), or anterior-posterior translations of the support surface (0.25 Hz, +/- 15 cm/sec), or both concurrently. Kinematics of the head, trunk, and shank were collected with an Optotrak system and angular motion of each segment plotted across time. With only support surface translation, segmental responses were small (1 degrees -2 degrees ) and mostly opposed the direction of sled translation. When only the visual scene was moving, segmental responses increased as the trial progressed. When the inputs were presented coincidentally, response amplitudes were large even at the onset of the trial. Mean RMS values across subjects were significantly greater with combined stimuli than for either stimulus presented alone and areas under the power curve across subjects were significantly increased at the frequency of the visual input when both inputs were presented. Thus, intra-modality dependencies were observed, such that responses to the visual inputs significantly increased and responses to the somatosensory signals reflected the stimulus amplitude only when the two inputs were combined. We believe it unlikely that the role of any single pathway contributing to postural control can be accurately characterized in a static environment if the function of that pathway is context dependent.

Using immersive technology for postural research and rehabilitation.

Posture has traditionally been examined by isolating individual control pathways to determine their specific contributions. However, if these pathways are responsive to functional contexts, then their responses may differ when the system is receiving simultaneous inputs from multiple pathways. Thus, we may never fully understand how the central nervous system (CNS) organizes behaviors in the real world from studies conducted in the minimized environment of the laboratory. The consequence of this is that when findings from the laboratory are applied to therapeutic intervention, the intervention may not be appropriate for all circumstances and will not fully meet the needs of the patient. We have united an immersive dynamic virtual environment with motion of a posture platform to record the biomechanical and physiological responses to combined visual, vestibular, and proprioceptive inputs. The virtual environment possesses content, contrast, and texture so that we can examine postural responses as they might occur in a complex, real-world environment. In this paper we specifically describe the factors guiding our choices of virtual technology and present data from young adults, elderly adults, and an individual with bilateral labyrinthine loss to demonstrate how multimodal inputs influence their postural response organization. Significant implications for future experimental and rehabilitation protocols are also discussed.

Error augmentation enhancing arm recovery in individuals with chronic stroke: a randomized crossover design.

BACKGROUND: Neurorehabilitation studies suggest that manipulation of error signals during practice can stimulate improvement in coordination after stroke. OBJECTIVE: To test visual display and robotic technology that delivers augmented error signals during training, in participants with stroke. METHODS: A total of 26 participants with chronic hemiparesis were trained with haptic (via robot-rendered forces) and graphic (via a virtual environment) distortions to amplify upper-extremity (UE) tracking error. In a randomized crossover design, the intervention was compared with an equivalent amount of practice without error augmentation (EA). Interventions involved three 45-minute sessions per week for 2 weeks, then 1 week of no treatment, and then 2 additional weeks of the alternate treatment. A therapist provided a visual cursor using a tracking device, and participants were instructed to match it with their hand. Haptic and visual EA was used with blinding of participant, therapist, technician-operator, and evaluator. Clinical measures of impairment were obtained at the beginning and end of each 2-week treatment phase as well as at 1 week and at 45 days after the last treatment. RESULTS: Outcomes showed a small, but significant benefit to EA training over simple repetitive practice, with a mean 2-week improvement in Fugl-Meyer UE motor score of 2.08 and Wolf Motor Function Test of timed tasks of 1.48 s. CONCLUSIONS: This interactive technology may improve UE motor recovery of stroke-related hemiparesis.

Human Augmentics: augmenting human evolution.

Human Augmentics (HA) refers to technologies for expanding the capabilities, and characteristics of humans. One can think of Human Augmentics as the driving force in the non-biological evolution of humans. HA devices will provide technology to compensate for human biological limitations either natural or acquired. The strengths of HA lie in its applicability to all humans. Its interoperability enables the formation of ecosystems whereby augmented humans can draw from other realms such as "the Cloud" and other augmented humans for strength. The exponential growth in new technologies portends such a system but must be designed for interaction through the use of open-standards and open-APIs for system development. We discuss the conditions needed for HA to flourish with an emphasis on devices that provide non-biological rehabilitation.

Arm control recovery enhanced by error augmentation.

Here we present results where nineteen stroke survivors with chronic hemiparesis simultaneously employed the trio of patient, therapist, and machine. Massed practice combined with error augmentation, where haptic (robotic forces) and graphic (visual display) distortions are used to enhance the feedback of error, was compared to massed practice alone. The 6-week randomized crossover design involved approximately 60 minutes of daily treatment three times per week for two weeks, followed by one week of rest, and then repeated using the alternate treatment protocol. A therapist provided a visual target using a tracking device that moved a cursor in front of the patient, who was instructed to maintain the cursor on the target. The patient, therapist, technician-operator, and rater were blinded to treatment type. Several clinical measures gauged outcomes at the beginning and end of each 2-week period and one week post training. Results showed incremental benefit across most but not all days, abrupt gains in performance, and a benefit to error augmentation training in final evaluations. This application of interactive technology may be a compelling new method for enhancing a therapist's productivity in stroke-rehabilitation.

A pneumatic glove and immersive virtual reality environment for hand rehabilitative training after stroke.

While a number of devices have recently been developed to facilitate hand rehabilitation after stroke, most place some restrictions on movement of the digits or arm. Thus, a novel glove was developed which can provide independent extension assistance to each digit while still allowing full arm movement. This pneumatic glove, the PneuGlove, can be used for training grasp-and-release movements either with real objects or with virtual objects in a virtual reality environment. Two groups of stroke survivors, with seven subjects in each group, completed a six-week rehabilitation training protocol, consisting of three 1-h sessions held each week. One group wore the PneuGlove during training, performed both within a novel virtual reality environment and outside of it with physical objects, while the other group completed the same training without the device. Across subjects, significant improvements were observed in the Fugl-Meyer Assessment for the upper extremity (p < 0.001), the hand/wrist portion of the Fugl-Meyer Assessment (p < 0.001), the Box and Blocks test (p < 0.005), and palmar pinch strength (p < 0.005). While changes in the two groups were not statistically different, the group using the PneuGlove did show greater mean improvement on each of these measures, such as gains of 3.7 versus 2.4 points on the hand/wrist portion of the Fugl-Meyer Assessment and 14 N versus 5 N in palmar pinch.