An Analysis of Intention Detection Strategies to Control Advanced Assistive Technologies at the CYBATHLON.

With the increasing range of functionalities of advanced assistive technologies (AAT), reliable control and initiation of the desired actions become increasingly challenging for users. In this work, we present an analysis of current practices, user preferences, and usability of AAT intention detection strategies based on a survey among participants with disabilities at the CYBATHLON 2020 Global Edition. We collected data from 35 respondents, using devices in various disciplines and levels of technology maturity. We found that conventional, direct inputs such as buttons and joysticks are used by the majority of AAT (71.4%) due to their simplicity and learnability. However, 22 respondents (62.8%) reported a desire for more natural control using muscle or non-invasive brain signals, and 37.1% even reported an openness to invasive strategies for potentially improved control. The usability of the used strategies in terms of the explored attributes (reliability, mental effort, required learning) was mainly perceived positively, whereas no significant difference was observed across intention detection strategies and device types. It can be assumed that the strategies used during the CYBATHLON realistically represent options to control an AAT in a dynamic, physically and mentally demanding environment. Thus, this work underlines the need for carefully considering user needs and preferences for the selection of intention detection strategies in a context of use outside the laboratory.

A survey on the influence of CYBATHLON on the development and acceptance of advanced assistive technologies.

BACKGROUND: Advanced assistive technologies (AAT) aim to exploit the vast potential of technological developments made in the past decades to improve the quality of life of people with disabilities. Combining complex robotic technologies with the unique needs of people with disabilities requires a strong focus on user-centered design to ensure that the AAT appropriately addresses the daily life struggles of target users. The CYBATHLON aims to promote this mindset by empowering the AAT target users ("pilots") to compete on race tracks that represent approximations of daily life obstacles. The objective of this work was to investigate the AAT technology development, usability, and user involvement (i.e., application of user-centered design) in the context of the CYBATHLON. METHODS: With an online survey targeting the pilots and technical leads of teams preparing for the CYBATHLON 2020 Global Edition, we investigated to what extent the pilots were involved in device development and how this influences the perceived usability of the showcased AAT. Furthermore, the effects of user-centered design variables on the individual race performances were analyzed. RESULTS: A total of 81 responses from 35 pilots and 46 technical leads were collected in the two weeks prior to the event. Of all teams partaking in the included disciplines of the CYBATHLON 2020 Global Edition, 81.8% (36 of 44) were included in the study. User-centered design appeared to be a prevalent practice among the teams, as 85.7% of all pilots reported a certain level of involvement. However, only 25.5% of the pilots reported daily life usage, despite QUEST usability scores of both respondent groups showing moderate to high satisfaction with the respected AAT across all investigated disciplines. An explorative linear mixed model indicated that daily life usage (p < 0.05) and prolonged user involvement (e.g., more than 2 years, p < 0.001) have a significant positive effect on the race performance at the competition. CONCLUSIONS: We conclude that the CYBATHLON positively fulfills its conceptual goals of promoting active participation and inclusion of people with disabilities in the design and evaluation of AAT, thereby stimulating the development of promising novel technological solutions. Also, our data could underline the value of the competition as a benchmark, highlighting remaining usability limitations or technology adoption hurdles.

When a robot teaches humans: Automated feedback selection accelerates motor learning.

A multitude of robotic systems have been developed to foster motor learning. Some of these robotic systems featured augmented visual or haptic feedback, which was automatically adjusted to the trainee's performance. However, selecting the type of feedback to achieve the training goal usually remained up to a human trainer. We automated this feedback selection within a robotic rowing simulator: Four spatial errors and one velocity error were considered, all related to trunk-arm sweep rowing set as the training goal to be learned. In an alternating sequence of assessments without augmented feedback and training sessions with augmented, concurrent feedback, the experimental group received feedback, thus addressing the main shortcoming of the previous assessment. With this approach, each participant of the experimental group received an individual sequence of 10 training sessions with feedback. The training sequences from participants in the experimental group were consecutively applied for participants in the control group. Both groups were able to reduce spatial and velocity errors due to training. The learning rate of the requested velocity profile was significantly higher for the experimental group compared with the control group. Thus, our robotic rowing simulator accelerated motor learning by automated feedback selection. This demonstration of a working, closed-loop selection of types of feedback, i.e., training conditions, could serve as the basis for other robotic trainers incorporating further human expertise and artificial intelligence.

Benchmarking Brain-Computer Interfaces Outside the Laboratory: The Cybathlon 2016.

This paper presents a new approach to benchmarking brain-computer interfaces (BCIs) outside the lab. A computer game was created that mimics a real-world application of assistive BCIs, with the main outcome metric being the time needed to complete the game. This approach was used at the Cybathlon 2016, a competition for people with disabilities who use assistive technology to achieve tasks. The paper summarizes the technical challenges of BCIs, describes the design of the benchmarking game, then describes the rules for acceptable hardware, software and inclusion of human pilots in the BCI competition at the Cybathlon. The 11 participating teams, their approaches, and their results at the Cybathlon are presented. Though the benchmarking procedure has some limitations (for instance, we were unable to identify any factors that clearly contribute to BCI performance), it can be successfully used to analyze BCI performance in realistic, less structured conditions. In the future, the parameters of the benchmarking game could be modified to better mimic different applications (e.g., the need to use some commands more frequently than others). Furthermore, the Cybathlon has the potential to showcase such devices to the general public.

Learning of Temporal and Spatial Movement Aspects: A Comparison of Four Types of Haptic Control and Concurrent Visual Feedback.

In literature, the effectiveness of haptics for motor learning is controversially discussed. Haptics is believed to be effective for motor learning in general; however, different types of haptic control enhance different movement aspects. Thus, in dependence on the movement aspects of interest, one type of haptic control may be effective whereas another one is not. Therefore, in the current work, it was investigated if and how different types of haptic controllers affect learning of spatial and temporal movement aspects. In particular, haptic controllers that enforce active participation of the participants were expected to improve spatial aspects. Only haptic controllers that provide feedback about the task's velocity profile were expected to improve temporal aspects. In a study on learning a complex trunk-arm rowing task, the effect of training with four different types of haptic control was investigated: position control, path control, adaptive path control, and reactive path control. A fifth group (control) trained with visual concurrent augmented feedback. As hypothesized, the position controller was most effective for learning of temporal movement aspects, while the path controller was most effective in teaching spatial movement aspects of the rowing task. Visual feedback was also effective for learning temporal and spatial movement aspects.

Sonification and haptic feedback in addition to visual feedback enhances complex motor task learning.

Concurrent augmented feedback has been shown to be less effective for learning simple motor tasks than for complex tasks. However, as mostly artificial tasks have been investigated, transfer of results to tasks in sports and rehabilitation remains unknown. Therefore, in this study, the effect of different concurrent feedback was evaluated in trunk-arm rowing. It was then investigated whether multimodal audiovisual and visuohaptic feedback are more effective for learning than visual feedback only. Naïve subjects (N = 24) trained in three groups on a highly realistic virtual reality-based rowing simulator. In the visual feedback group, the subject's oar was superimposed to the target oar, which continuously became more transparent when the deviation between the oars decreased. Moreover, a trace of the subject's trajectory emerged if deviations exceeded a threshold. The audiovisual feedback group trained with oar movement sonification in addition to visual feedback to facilitate learning of the velocity profile. In the visuohaptic group, the oar movement was inhibited by path deviation-dependent braking forces to enhance learning of spatial aspects. All groups significantly decreased the spatial error (tendency in visual group) and velocity error from baseline to the retention tests. Audiovisual feedback fostered learning of the velocity profile significantly more than visuohaptic feedback. The study revealed that well-designed concurrent feedback fosters complex task learning, especially if the advantages of different modalities are exploited. Further studies should analyze the impact of within-feedback design parameters and the transferability of the results to other tasks in sports and rehabilitation.

Transfer of complex skill learning from virtual to real rowing.

Simulators are commonly used to train complex tasks. In particular, simulators are applied to train dangerous tasks, to save costs, and to investigate the impact of different factors on task performance. However, in most cases, the transfer of simulator training to the real task has not been investigated. Without a proof for successful skill transfer, simulators might not be helpful at all or even counter-productive for learning the real task. In this paper, the skill transfer of complex technical aspects trained on a scull rowing simulator to sculling on water was investigated. We assume if a simulator provides high fidelity rendering of the interactions with the environment even without augmented feedback, training on such a realistic simulator would allow similar skill gains as training in the real environment. These learned skills were expected to transfer to the real environment. Two groups of four recreational rowers participated. One group trained on water, the other group trained on a simulator. Within two weeks, both groups performed four training sessions with the same licensed rowing trainer. The development in performance was assessed by quantitative biomechanical performance measures and by a qualitative video evaluation of an independent, blinded trainer. In general, both groups could improve their performance on water. The used biomechanical measures seem to allow only a limited insight into the rowers' development, while the independent trainer could also rate the rowers' overall impression. The simulator quality and naturalism was confirmed by the participants in a questionnaire. In conclusion, realistic simulator training fostered skill gains to a similar extent as training in the real environment and enabled skill transfer to the real environment. In combination with augmented feedback, simulator training can be further exploited to foster motor learning even to a higher extent, which is subject to future work.

Terminal feedback outperforms concurrent visual, auditory, and haptic feedback in learning a complex rowing-type task.

Augmented feedback, provided by coaches or displays, is a well-established strategy to accelerate motor learning. Frequent terminal feedback and concurrent feedback have been shown to be detrimental for simple motor task learning but supportive for complex motor task learning. However, conclusions on optimal feedback strategies have been mainly drawn from studies on artificial laboratory tasks with visual feedback only. Therefore, the authors compared the effectiveness of learning a complex, 3-dimensional rowing-type task with either concurrent visual, auditory, or haptic feedback to self-controlled terminal visual feedback. Results revealed that terminal visual feedback was most effective because it emphasized the internalization of task-relevant aspects. In contrast, concurrent feedback fostered the correction of task-irrelevant errors, which hindered learning. The concurrent visual and haptic feedback group performed much better during training with the feedback than in nonfeedback trials. Auditory feedback based on sonification of the movement error was not practical for training the 3-dimensional movement for most participants. Concurrent multimodal feedback in combination with terminal feedback may be most effective, especially if the feedback strategy is adapted to individual preferences and skill level.

Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review.

It is generally accepted that augmented feedback, provided by a human expert or a technical display, effectively enhances motor learning. However, discussion of the way to most effectively provide augmented feedback has been controversial. Related studies have focused primarily on simple or artificial tasks enhanced by visual feedback. Recently, technical advances have made it possible also to investigate more complex, realistic motor tasks and to implement not only visual, but also auditory, haptic, or multimodal augmented feedback. The aim of this review is to address the potential of augmented unimodal and multimodal feedback in the framework of motor learning theories. The review addresses the reasons for the different impacts of feedback strategies within or between the visual, auditory, and haptic modalities and the challenges that need to be overcome to provide appropriate feedback in these modalities, either in isolation or in combination. Accordingly, the design criteria for successful visual, auditory, haptic, and multimodal feedback are elaborated.