Evaluation of a Portable fMRI Compatible Robotic Wrist Interface.

This paper presents evaluation of a portable fMRI compatible haptic interface to study the brain correlates of sensorimotor control during wrist motion. The interface is actuated by a shielded DC motor located more than 2 m away from the 3T MR scanner's bore. The achievable wrist torque of the interface is up to 2 Nm, and the interface provides sufficient bandwidth for human motor control experiments. Ergonomic and fMRI compatibility testing with a 3T MR scanner showed that the interface is MR safe, compatible with a strong static magnetic field and radio frequency emission, and its operation does not affect the quality of the acquired images. Clinical Relevance- We present and evaluate an fMRI compatible robotic interface to study human wrist joint motor function.

Neural signatures of visuo-motor integration during human-robot interactions.

Visuo-motor integration shapes our daily experience and underpins the sense of feeling in control over our actions. The last decade has seen a surge in robotically and virtually mediated interactions, whereby bodily actions ultimately result in an artificial movement. But despite the growing number of applications, the neurophysiological correlates of visuo-motor processing during human-machine interactions under dynamic conditions remain scarce. Here we address this issue by employing a bimanual robotic interface able to track voluntary hands movement, rendered in real-time into the motion of two virtual hands. We experimentally manipulated the visual feedback in the virtual reality with spatial and temporal conflicts and investigated their impact on (1) visuo-motor integration and (2) the subjective experience of being the author of one's action (i.e., sense of agency). Using somatosensory evoked responses measured with electroencephalography, we investigated neural differences occurring when the integration between motor commands and visual feedback is disrupted. Our results show that the right posterior parietal cortex encodes for differences between congruent and spatially-incongruent interactions. The experimental manipulations also induced a decrease in the sense of agency over the robotically-mediated actions. These findings offer solid neurophysiological grounds that can be used in the future to monitor integration mechanisms during movements and ultimately enhance subjective experience during human-machine interactions.

Thought consciousness and source monitoring depend on robotically controlled sensorimotor conflicts and illusory states.

Thought insertion (TI) is characterized by the experience that certain thoughts, occurring in one's mind, are not one's own, but the thoughts of somebody else and suggestive of a psychotic disorder. We report a robotics-based method able to investigate the behavioral and subjective mechanisms of TI in healthy participants. We used a robotic device to alter body perception by providing online sensorimotor stimulation, while participants performed cognitive tasks implying source monitoring of mental states attributed to either oneself or another person. Across several experiments, conflicting sensorimotor stimulation reduced the distinction between self- and other-generated thoughts and was, moreover, associated with the experimentally generated feeling of being in the presence of an alien agent and subjective aspects of TI. Introducing a new robotics-based approach that enables the experimental study of the brain mechanisms of TI, these results link TI to predictable self-other shifts in source monitoring and specific sensorimotor processes.

Balance Control Strategies during Standing in a Locked-Ankle Passive Exoskeleton.

This paper investigates sensorimotor adaptation strategies of sagittal postural control in healthy subjects under kinematic constraints. A passive exoskeleton named CAPTUR, with locked ankle joints and legs motion restrained to the sagittal plane is used to restrict and measure participant's movements. The aim is to assess the role of the orientation of the shank and the trunk segments in maintaining the body center of mass above its support base, while the ankle strategy is inhibited. Five young healthy participants were asked to keep standing, while their balance was challenged by five experimental conditions. Participants mainly regulated quiet standing balance by flexing/extending the knees, in order to affect the shank and feet angles, and move the contact patch along the sagittal axis. In this case, the orientation of the trunk segment changes synchronously with the shank angle to keep an upright posture. Responses to more dramatic excursions of the center of pressure are ensured by changing the trunk tilt angle in opposition of phase with the shank angle. These observations could be used to implement a bioinspired balance controller for such constrained lower-limb exoskeletons.

Error Augmentation Improves Visuomotor Adaptation during a Full-Body Balance Task.

Visual amplification of kinematic errors has successfully been applied to improve performance for upper limb movements. In this study, we investigated whether visual error augmentation can promote faster adaptation during a full-body balance task. Healthy volunteers controlled a cursor by shifting their weight on the THERA-Trainer coro platform. Two experimental groups and one control group were asked to reach visual targets. For the two experimental groups, the cursor's deviation from the ideal straight line trajectory was augmented by a gain of 1.5 and 2, respectively, while the control group did not experience visual error amplification (gain of 1). Error augmentation with a gain of 1.5 enhanced the speed and the amount of motor adaptation, while the highest gain might have decreased the stability of adaptation. As visual feedback is commonly used in balance training, our preliminary data suggest that integrating visual error augmentation in postural exercises may facilitate balance control.

Positioning the endoscope in laparoscopic surgery by foot: Influential factors on surgeons' performance in virtual trainer.

We have investigated how surgeons can use the foot to position a laparoscopic endoscope, a task that normally requires an extra assistant. Surgeons need to train in order to exploit the possibilities offered by this new technique and safely manipulate the endoscope together with the hands movements. A realistic abdominal cavity has been developed as training simulator to investigate this multi-arm manipulation. In this virtual environment, the surgeon's biological hands are modelled as laparoscopic graspers while the viewpoint is controlled by the dominant foot. 23 surgeons and medical students performed single-handed and bimanual manipulation in this environment. The results show that residents had superior performance compared to both medical students and more experienced surgeons, suggesting that residency is an ideal period for this training. Performing the single-handed task improves the performance in the bimanual task, whereas the converse was not true.

TWIICE - A lightweight lower-limb exoskeleton for complete paraplegics.

This paper introduces TWIICE, a lower-limb exoskeleton that enables people suffering from complete paraplegia to stand up and walk again. TWIICE provides complete mobilization of the lower-limbs, which is a first step toward enabling the user to regain independence in activities of the daily living. The tasks it can perform include level and inclined walking (up to 20° slope), stairs ascent and descent, sitting on a seat, and standing up. Participation in the world's first Cybathlon (Zurich, 2016) demonstrated good performance at these demanding tasks. In this paper, we describe the implementation details of the device and comment on preliminary results from a single user case study.

Cortical and subcortical mechanisms of brain-machine interfaces.

Technical advances in the field of Brain-Machine Interfaces (BMIs) enable users to control a variety of external devices such as robotic arms, wheelchairs, virtual entities and communication systems through the decoding of brain signals in real time. Most BMI systems sample activity from restricted brain regions, typically the motor and premotor cortex, with limited spatial resolution. Despite the growing number of applications, the cortical and subcortical systems involved in BMI control are currently unknown at the whole-brain level. Here, we provide a comprehensive and detailed report of the areas active during on-line BMI control. We recorded functional magnetic resonance imaging (fMRI) data while participants controlled an EEG-based BMI inside the scanner. We identified the regions activated during BMI control and how they overlap with those involved in motor imagery (without any BMI control). In addition, we investigated which regions reflect the subjective sense of controlling a BMI, the sense of agency for BMI-actions. Our data revealed an extended cortical-subcortical network involved in operating a motor-imagery BMI. This includes not only sensorimotor regions but also the posterior parietal cortex, the insula and the lateral occipital cortex. Interestingly, the basal ganglia and the anterior cingulate cortex were involved in the subjective sense of controlling the BMI. These results inform basic neuroscience by showing that the mechanisms of BMI control extend beyond sensorimotor cortices. This knowledge may be useful for the development of BMIs that offer a more natural and embodied feeling of control for the user. Hum Brain Mapp 38:2971-2989, 2017. © 2017 Wiley Periodicals, Inc.

EXiO-A Brain-Controlled Lower Limb Exoskeleton for Rhesus Macaques.

Recent advances in the field of brain-machine interfaces (BMIs) have demonstrated enormous potential to shape the future of rehabilitation and prosthetic devices. Here, a lower-limb exoskeleton controlled by the intracortical activity of an awake behaving rhesus macaque is presented as a proof-of-concept for a locomotorBMI. A detailed description of the mechanical device, including its innovative features and first experimental results, is provided. During operation, BMI-decoded position and velocity are directly mapped onto the bipedal exoskeleton's motions, which then move the monkey's legs as the monkey remains physicallypassive. To meet the unique requirements of such an application, the exoskeleton's features include: high output torque with backdrivable actuation, size adjustability, and safe user-robot interface. In addition, a novel rope transmission is introduced and implemented. To test the performance of the exoskeleton, a mechanical assessment was conducted, which yielded quantifiable results for transparency, efficiency, stiffness, and tracking performance. Usage under both brain control and automated actuation demonstrates the device's capability to fulfill the demanding needs of this application. These results lay the groundwork for further advancement in BMI-controlled devices for primates including humans.

Assimilation of virtual legs and perception of floor texture by complete paraplegic patients receiving artificial tactile feedback.

Spinal cord injuries disrupt bidirectional communication between the patient's brain and body. Here, we demonstrate a new approach for reproducing lower limb somatosensory feedback in paraplegics by remapping missing leg/foot tactile sensations onto the skin of patients' forearms. A portable haptic display was tested in eight patients in a setup where the lower limbs were simulated using immersive virtual reality (VR). For six out of eight patients, the haptic display induced the realistic illusion of walking on three different types of floor surfaces: beach sand, a paved street or grass. Additionally, patients experienced the movements of the virtual legs during the swing phase or the sensation of the foot rolling on the floor while walking. Relying solely on this tactile feedback, patients reported the position of the avatar leg during virtual walking. Crossmodal interference between vision of the virtual legs and tactile feedback revealed that patients assimilated the virtual lower limbs as if they were their own legs. We propose that the addition of tactile feedback to neuroprosthetic devices is essential to restore a full lower limb perceptual experience in spinal cord injury (SCI) patients, and will ultimately, lead to a higher rate of prosthetic acceptance/use and a better level of motor proficiency.

Long-Term Training with a Brain-Machine Interface-Based Gait Protocol Induces Partial Neurological Recovery in Paraplegic Patients.

Brain-machine interfaces (BMIs) provide a new assistive strategy aimed at restoring mobility in severely paralyzed patients. Yet, no study in animals or in human subjects has indicated that long-term BMI training could induce any type of clinical recovery. Eight chronic (3-13 years) spinal cord injury (SCI) paraplegics were subjected to long-term training (12 months) with a multi-stage BMI-based gait neurorehabilitation paradigm aimed at restoring locomotion. This paradigm combined intense immersive virtual reality training, enriched visual-tactile feedback, and walking with two EEG-controlled robotic actuators, including a custom-designed lower limb exoskeleton capable of delivering tactile feedback to subjects. Following 12 months of training with this paradigm, all eight patients experienced neurological improvements in somatic sensation (pain localization, fine/crude touch, and proprioceptive sensing) in multiple dermatomes. Patients also regained voluntary motor control in key muscles below the SCI level, as measured by EMGs, resulting in marked improvement in their walking index. As a result, 50% of these patients were upgraded to an incomplete paraplegia classification. Neurological recovery was paralleled by the reemergence of lower limb motor imagery at cortical level. We hypothesize that this unprecedented neurological recovery results from both cortical and spinal cord plasticity triggered by long-term BMI usage.

Quantifying the role of motor imagery in brain-machine interfaces.

Despite technical advances in brain machine interfaces (BMI), for as-yet unknown reasons the ability to control a BMI remains limited to a subset of users. We investigate whether individual differences in BMI control based on motor imagery (MI) are related to differences in MI ability. We assessed whether differences in kinesthetic and visual MI, in the behavioral accuracy of MI, and in electroencephalographic variables, were able to differentiate between high- versus low-aptitude BMI users. High-aptitude BMI users showed higher MI accuracy as captured by subjective and behavioral measurements, pointing to a prominent role of kinesthetic rather than visual imagery. Additionally, for the first time, we applied mental chronometry, a measure quantifying the degree to which imagined and executed movements share a similar temporal profile. We also identified enhanced lateralized µ-band oscillations over sensorimotor cortices during MI in high- versus low-aptitude BMI users. These findings reveal that subjective, behavioral, and EEG measurements of MI are intimately linked to BMI control. We propose that poor BMI control cannot be ascribed only to intrinsic limitations of EEG recordings and that specific questionnaires and mental chronometry can be used as predictors of BMI performance (without the need to record EEG activity).

In a demanding task, three-handed manipulation is preferred to two-handed manipulation.

Equipped with a third hand under their direct control, surgeons may be able to perform certain surgical interventions alone; this would reduce the need for a human assistant and related coordination difficulties. However, does human performance improve with three hands compared to two hands? To evaluate this possibility, we carried out a behavioural study on the performance of naive adults catching objects with three virtual hands controlled by their two hands and right foot. The subjects could successfully control the virtual hands in a few trials. With this control strategy, the workspace of the hands was inversely correlated with the task velocity. The comparison of performance between the three and two hands control revealed no significant difference of success in catching falling objects and in average effort during the tasks. Subjects preferred the three handed control strategy, found it easier, with less physical and mental burden. Although the coordination of the foot with the natural hands increased trial after trial, about two minutes of practice was not sufficient to develop a sense of ownership towards the third arm.

Control of a Supernumerary Robotic Hand by Foot: An Experimental Study in Virtual Reality.

In the operational theater, the surgical team could highly benefit from a robotic supplementary hand under the surgeon's full control. The surgeon may so become more autonomous; this may reduce communication errors with the assistants and take over difficult tasks such as holding tools without tremor. In this paper, we therefore examine the possibility to control a third robotic hand with one foot's movements. Three experiments in virtual reality were designed to assess the feasibility of this control strategy, the learning curve of the subjects in different tasks and the coordination of foot movements with the two natural hands. Results show that the limbs are moved simultaneously, in parallel rather than serially. Participants' performance improved within a few minutes of practice without any specific difficulty to complete the tasks. Subjective assessment by the subjects indicated that controlling a third hand by foot has been easy and required only negligible physical and mental efforts. The sense of ownership was reported to improve through the experiments. The mental burden was not directly related to the level of motion required by a task, but depended on the type of activity and practice. The most difficult task was moving two hands and foot in opposite directions. These results suggest that a combination of practice and appropriate tasks can enhance the learning process for controlling a robotic hand by foot.

Control strategies for active lower extremity prosthetics and orthotics: a review.

: Technological advancements have led to the development of numerous wearable robotic devices for the physical assistance and restoration of human locomotion. While many challenges remain with respect to the mechanical design of such devices, it is at least equally challenging and important to develop strategies to control them in concert with the intentions of the user.This work reviews the state-of-the-art techniques for controlling portable active lower limb prosthetic and orthotic (P/O) devices in the context of locomotive activities of daily living (ADL), and considers how these can be interfaced with the user's sensory-motor control system. This review underscores the practical challenges and opportunities associated with P/O control, which can be used to accelerate future developments in this field. Furthermore, this work provides a classification scheme for the comparison of the various control strategies.As a novel contribution, a general framework for the control of portable gait-assistance devices is proposed. This framework accounts for the physical and informatic interactions between the controller, the user, the environment, and the mechanical device itself. Such a treatment of P/Os--not as independent devices, but as actors within an ecosystem--is suggested to be necessary to structure the next generation of intelligent and multifunctional controllers.Each element of the proposed framework is discussed with respect to the role that it plays in the assistance of locomotion, along with how its states can be sensed as inputs to the controller. The reviewed controllers are shown to fit within different levels of a hierarchical scheme, which loosely resembles the structure and functionality of the nominal human central nervous system (CNS). Active and passive safety mechanisms are considered to be central aspects underlying all of P/O design and control, and are shown to be critical for regulatory approval of such devices for real-world use.The works discussed herein provide evidence that, while we are getting ever closer, significant challenges still exist for the development of controllers for portable powered P/O devices that can seamlessly integrate with the user's neuromusculoskeletal system and are practical for use in locomotive ADL.

Neurological and robot-controlled induction of an apparition.

Tales of ghosts, wraiths, and other apparitions have been reported in virtually all cultures. The strange sensation that somebody is nearby when no one is actually present and cannot be seen (feeling of a presence, FoP) is a fascinating feat of the human mind, and this apparition is often covered in the literature of divinity, occultism, and fiction. Although it is described by neurological and psychiatric patients and healthy individuals in different situations, it is not yet understood how the phenomenon is triggered by the brain. Here, we performed lesion analysis in neurological FoP patients, supported by an analysis of associated neurological deficits. Our data show that the FoP is an illusory own-body perception with well-defined characteristics that is associated with sensorimotor loss and caused by lesions in three distinct brain regions: temporoparietal, insular, and especially frontoparietal cortex. Based on these data and recent experimental advances of multisensory own-body illusions, we designed a master-slave robotic system that generated specific sensorimotor conflicts and enabled us to induce the FoP and related illusory own-body perceptions experimentally in normal participants. These data show that the illusion of feeling another person nearby is caused by misperceiving the source and identity of sensorimotor (tactile, proprioceptive, and motor) signals of one's own body. Our findings reveal the neural mechanisms of the FoP, highlight the subtle balance of brain mechanisms that generate the experience of "self" and "other," and advance the understanding of the brain mechanisms responsible for hallucinations in schizophrenia.

Development of an assistive motorized hip orthosis: kinematics analysis and mechanical design.

With the increase of life expectancy, a higher number of elderly need assistance to maintain their mobility and their independance. The hip joint is crucial for walking and is problematic for a large number of aged people. In this paper we present a novel design of a motorized hip orthosis to assist elderly people while walking, stair climbing and during the sit-to-stand transistions. The kinematics was developed based on biomechanics considerations. To be able to achieve a large assistance rate, velocity and torques of the hip joint were studied from the literature. In order to fit with these requirements, an amplification mechanism inspired by excavators was developed and implemented. Comfort considerations were also taken into account and a custom interface was designed with the collaboration of a professional orthopaedic technician. First tests with the prototype showed that the workspace is sufficient for walking, for stair climbing as well as for sit-to-stand transitions. The assistance rate can go up to 30% for a 70 kg subject during walking at a cadence of 100 steps/min. The comfort is guaranteed despite the important weight (4.3 kg) of this first prototype.

Lifeguard for robotic surgery assistance "LIGRA": an interactive platform centralizing information and control in robotic surgery.

A crying need for standardized safety management in health-care in conventional and in robotic surgery in particular has been identified. The same will, which has led to safer air transportation, can be a great source of inspiration for health-care. This paper proposes an interactive platform for the operating room with robotic surgery in view of an efficient safety implementation.

Expanding the primate body schema in sensorimotor cortex by virtual touches of an avatar.

The brain representation of the body, called the body schema, is susceptible to plasticity. For instance, subjects experiencing a rubber hand illusion develop a sense of ownership of a mannequin hand when they view it being touched while tactile stimuli are simultaneously applied to their own hand. Here, the cortical basis of such an embodiment was investigated through concurrent recordings from primary somatosensory (i.e., S1) and motor (i.e., M1) cortical neuronal ensembles while two monkeys observed an avatar arm being touched by a virtual ball. Following a period when virtual touches occurred synchronously with physical brushes of the monkeys' arms, neurons in S1 and M1 started to respond to virtual touches applied alone. Responses to virtual touch occurred 50 to 70 ms later than to physical touch, consistent with the involvement of polysynaptic pathways linking the visual cortex to S1 and M1. We propose that S1 and M1 contribute to the rubber hand illusion and that, by taking advantage of plasticity in these areas, patients may assimilate neuroprosthetic limbs as parts of their body schema.

Force feedback facilitates multisensory integration during robotic tool use.

The present study investigated the effects of force feedback in relation to tool use on the multisensory integration of visuo-tactile information. Participants learned to control a robotic tool through a surgical robotic interface. Following tool-use training, participants performed a crossmodal congruency task, by responding to tactile vibrations applied to their hands, while ignoring visual distractors superimposed on the robotic tools. In the first experiment it was found that tool-use training with force feedback facilitates multisensory integration of signals from the tool, as reflected in a stronger crossmodal congruency effect with the force feedback training compared to training without force feedback and to no training. The second experiment extends these findings by showing that training with realistic online force feedback resulted in a stronger crossmodal congruency effect compared to training in which force feedback was delayed. The present study highlights the importance of haptic information for multisensory integration and extends findings from classical tool-use studies to the domain of robotic tools. We argue that such crossmodal congruency effects are an objective measure of robotic tool integration and propose some potential applications in surgical robotics, robotic tools, and human-tool interaction.