User Evaluation of a Shared Robot Control System Combining BCI and Eye Tracking in a Portable Augmented Reality User Interface.

This study evaluates an innovative control approach to assistive robotics by integrating brain-computer interface (BCI) technology and eye tracking into a shared control system for a mobile augmented reality user interface. Aimed at enhancing the autonomy of individuals with physical disabilities, particularly those with impaired motor function due to conditions such as stroke, the system utilizes BCI to interpret user intentions from electroencephalography signals and eye tracking to identify the object of focus, thus refining control commands. This integration seeks to create a more intuitive and responsive assistive robot control strategy. The real-world usability was evaluated, demonstrating significant potential to improve autonomy for individuals with severe motor impairments. The control system was compared with an eye-tracking-based alternative to identify areas needing improvement. Although BCI achieved an acceptable success rate of 0.83 in the final phase, eye tracking was more effective with a perfect success rate and consistently lower completion times (p<0.001). The user experience responses favored eye tracking in 11 out of 26 questions, with no significant differences in the remaining questions, and subjective fatigue was higher with BCI use (p=0.04). While BCI performance lagged behind eye tracking, the user evaluation supports the validity of our control strategy, showing that it could be deployed in real-world conditions and suggesting a pathway for further advancements.

Talk with Ted: an embodied conversational agent for caregivers.

Communication is key to the success of any relationship. When it comes to caregivers, having a conversation with a person living with some form of cognitive impairment, such as dementia, can be a struggle. Most people living with dementia experience some form of communication impairment that reduces their ability to express their needs. In this case study, we present the design of an embodied conversation agent (ECA), Ted, designed to educate caregivers about the importance of good communication principles when engaging with people living with dementia. This training tool was trialed and compared to an online training tool, with 23 caregivers divided into two cohorts (12 in the ECA condition, and 11 in the online training tool condition), over a period of 8 weeks using a mixed evaluation approach. Our findings suggest that (a) caregivers developed an emotional connection with the ECA and retained the learning from their interactions with Ted even after 8 weeks had elapsed, (b) caregivers implemented the learnings in their practice, and (c) the changes in care practice were well received by people living with dementia.

Family-Centered Design: Interactive Performance Testing and User Interface Evaluation of the Slovenian eDavki Public Tax Portal.

This is the latest article in a series of research on the family-centered design concept. The theoretical context was revisited and expounded to support its usefulness in conjunction with a user-centered design approach within distinct application domains. A very important contribution is applied through the development of the instruments-website capture, a public testing platform, results processing and the Web Content Accessibility Guide 2.1 recommendation tool-to conduct unmoderated remote testing of user interfaces that corresponds to the requirements of general digitalization efforts as well as the response to current and future health risks. With this set of instruments, an experiment was conducted to address the differences in usage, and performance-wise and user-based evaluation methods, of the eDavki public tax portal, among two generations, adults and elderly citizens, and between an original and an adapted user interface that respects accessibility and other recommendations. The differences found are further discussed and are congruent to particularities that have been modified within interfaces.

Evaluative Research of Technologies for Prehospital Communication and Coordination: a Systematic Review.

Various technologies have been designed and developed to support communication and coordination between the field and hospital during a medical emergency. The usability issues and human factors entailed in these new technologies are important to their application and effectiveness, suggesting the need to examine this information in a systematic review. The systematic review aims to synthesize the user-centered evaluative research of prehospital communication technologies. We conducted a systematic literature search in four databases (Medline, Cochrane, Embase, and Web of Science) for articles published between the years 2000 and 2019. We included articles that evaluated the technologies developed for supporting prehospital communication and collaboration, and were published in English. A total of 918 articles were retrieved and screened, with 17 articles included for in-depth analysis. Two authors conducted independent screens and reviews of the articles using a list of inclusion/exclusion criteria and defined factors. The types of technology of reviewed articles included ambulance-based telemedicine, wearable, handheld, and Internet of Things (IoT) devices. Even though these technologies have demonstrated high levels of user acceptance, the reviewed studies noted a variety of challenges faced by emergency care providers, which were grouped into three categories-technical, usability, and organizational challenges. Our review also highlighted the paucity of evaluative research of prehospital communication technologies and the lack of user engagement throughout system design process. Based on the results, we discuss the importance of adopting user-centered design approaches and accounting for three social-technical factors in designing technologies for time-critical medical settings, including cognitive and physical stressor, workflow, and context. This systematic review presents an overview of key evaluative research of prehospital communication technologies. The paucity of evaluative research in prehospital communication technology and challenges faced in adopting advanced technological solutions in emergency care highlight the need to adopt user-centered design and take into account socio-technical issues at the point of system design.

Interaction-centred design: an end user evaluation of road intersection concepts developed using the cognitive work analysis design toolkit (CWA-DT).

Crashes at intersections represent an important road safety problem. Interactions between different road user types, such as between vehicles and vulnerable road users, are a particular concern. It has been suggested driver-centric road design plays a role in crashes. A multi-road user evaluation of three novel intersection designs is described. The designs were generated using the Cognitive Work Analysis Design Toolkit, underpinned by sociotechnical systems theory. The desktop evaluation involved drivers, motorcyclists, cyclists and pedestrians rating the design concepts against alignment with design goals, sociotechnical systems theory and usability, and providing feedback on the positive and negative aspects. Two concepts received more positive ratings and feedback in comparison to a concept that provided more user autonomy. The evaluation results also highlight clear differences in needs across road user groups. The design and evaluation process demonstrates how sociotechnical systems values and principles can be applied in the design of public spaces. Practitioner Summary: This study involved a participatory evaluation of novel road intersection designs, based on sociotechnical systems theory. The results identified important differences in needs and preferences across road user groups and demonstrate the value of sociotechnical systems theory and user participation in road transport design and evaluation processes. Abbreviations: CWA-DT: cognitive work analysis design toolkit; WDA: work domain analysis; SUS: system usability scale.

A New mHealth App for Monitoring and Awareness of Healthy Eating: Development and User Evaluation by Spanish Users.

Modern-day society has moved towards a more sedentary lifestyle. Advances in technology and changes in habits in our daily lives have led a large part of the population towards a spiralling sedentary lifestyle and obesity. The main objective of this work is to develop and subsequently assess a mobile app, named DietApp, that provides advice about obtaining a healthy diet according to age, clinical history and physical condition. DietApp has been developed for iOS and Android systems, and a survey comprising 7 simple questions enabled the app to be evaluated on a user level by taking into account aspects such as its usefulness and ease of use. DietApp was assessed by 150 Spanish individuals between 18 and 69 years of age, and 84% of them thought it was easy to use. 80% of users also considered the dietary suggestions provided by the app to be very useful while 62% were of the opinion that it is very useful in general. All of them would recommend the app to other users. During the six months when the app was used, any dietary excess or shortcomings were corrected in 72% of those interviewed. A mobile app has been created that is easy to use and attractive, providing personalised suggestions according to illness that are useful for the individual.

Evaluation of RPE-Select: A Web-Based Respiratory Protective Equipment Selector Tool.

This article describes the evaluation of an open-access web-based respiratory protective equipment selector tool (RPE-Select, accessible at http://www.healthyworkinglives.com/rpe-selector). This tool is based on the principles of the COSHH-Essentials (C-E) control banding (CB) tool, which was developed for the exposure risk management of hazardous chemicals in the workplace by small and medium sized enterprises (SMEs) and general practice H&S professionals. RPE-Select can be used for identifying adequate and suitable RPE for dusts, fibres, mist (solvent, water, and oil based), sprays, volatile solids, fumes, gases, vapours, and actual or potential oxygen deficiency. It can be applied for substances and products with safety data sheets as well as for a large number of commonly encountered process-generated substances (PGS), such as poultry house dusts or welding fume. Potential international usability has been built-in by using the Hazard Statements developed for the Globally Harmonised System (GHS) and providing recommended RPE in picture form as well as with a written specification. Illustration helps to compensate for the variabilities in assigned protection factors across the world. RPE-Select uses easily understandable descriptions/explanations and an interactive stepwise flow for providing input/answers at each step. The output of the selection process is a report summarising the user input data and a selection of RPE, including types of filters where applicable, from which the user can select the appropriate one for each wearer. In addition, each report includes 'Dos' and 'Don'ts' for the recommended RPE. RPE-Select outcomes, based on up to 20 hypothetical use scenarios, were evaluated in comparison with other available RPE selection processes and tools, and by 32 independent users with a broad range of familiarities with industrial use scenarios in general and respiratory protection in particular. For scenarios involving substances having safety data sheets, 87% of RPE-Select outcomes resulted in a 'safe' RPE selection, while 98% 'safe' outcomes were achieved for scenarios involving process-generated substances. Reasons for the outliers were examined. User comments and opinions on the mechanics and usability of RPE-Select are also presented.

User evaluation of patient counselling, combining nurse consultation and eHealth in hand eczema.

BACKGROUND AND OBJECTIVE: This study reports the findings from a user evaluation of a counselling programme for hand eczema patients in which face-to-face encounters were supplemented with user access to a new website. PATIENTS AND METHODS: Patients treated for hand eczema in two different settings were included consecutively. Website utilization was examined by use of the transaction log. Comparisons were made between participants who used the website and those who did not. The patients' perspectives were explored by the use of interviews. RESULTS: Among potential website users (n = 140), 88 patients (63%) had an average of 5.1 site visits. At follow-up, the website users had improved more in quality of life (p = 0.014), current burden of disease (p = 0.053), and itching (p = 0.042). The website users reported more changes in habits than did the non-website users (p = 0.024). No differences in clinical severity of hand eczema were found. The interviewees were generally satisfied with the counselling and the website. The strict log-on procedures were considered to be an obstacle to using the site. The consecutive inclusion of participants was considered to be a barrier to engagement in the dialogue forum. CONCLUSIONS: The website users benefited from the website, although this was not substantiated by clinical measurements. The trial design partly hampered website utilization. An initial feasibility study could have been warranted.

Evaluation of a novel Conjunctive Exploratory Navigation Interface for consumer health information: a crowdsourced comparative study.

BACKGROUND: Numerous consumer health information websites have been developed to provide consumers access to health information. However, lookup search is insufficient for consumers to take full advantage of these rich public information resources. Exploratory search is considered a promising complementary mechanism, but its efficacy has never before been rigorously evaluated for consumer health information retrieval interfaces. OBJECTIVE: This study aims to (1) introduce a novel Conjunctive Exploratory Navigation Interface (CENI) for supporting effective consumer health information retrieval and navigation, and (2) evaluate the effectiveness of CENI through a search-interface comparative evaluation using crowdsourcing with Amazon Mechanical Turk (AMT). METHODS: We collected over 60,000 consumer health questions from NetWellness, one of the first consumer health websites to provide high-quality health information. We designed and developed a novel conjunctive exploratory navigation interface to explore NetWellness health questions with health topics as dynamic and searchable menus. To investigate the effectiveness of CENI, we developed a second interface with keyword-based search only. A crowdsourcing comparative study was carefully designed to compare three search modes of interest: (A) the topic-navigation-based CENI, (B) the keyword-based lookup interface, and (C) either the most commonly available lookup search interface with Google, or the resident advanced search offered by NetWellness. To compare the effectiveness of the three search modes, 9 search tasks were designed with relevant health questions from NetWellness. Each task included a rating of difficulty level and questions for validating the quality of answers. Ninety anonymous and unique AMT workers were recruited as participants. RESULTS: Repeated-measures ANOVA analysis of the data showed the search modes A, B, and C had statistically significant differences among their levels of difficulty (P<.001). Wilcoxon signed-rank test (one-tailed) between A and B showed that A was significantly easier than B (P<.001). Paired t tests (one-tailed) between A and C showed A was significantly easier than C (P<.001). Participant responses on the preferred search modes showed that 47.8% (43/90) participants preferred A, 25.6% (23/90) preferred B, 24.4% (22/90) preferred C. Participant comments on the preferred search modes indicated that CENI was easy to use, provided better organization of health questions by topics, allowed users to narrow down to the most relevant contents quickly, and supported the exploratory navigation by non-experts or those unsure how to initiate their search. CONCLUSIONS: We presented a novel conjunctive exploratory navigation interface for consumer health information retrieval and navigation. Crowdsourcing permitted a carefully designed comparative search-interface evaluation to be completed in a timely and cost-effective manner with a relatively large number of participants recruited anonymously. Accounting for possible biases, our study has shown for the first time with crowdsourcing that the combination of exploratory navigation and lookup search is more effective than lookup search alone.

Cultural applicability and desirability of 'Broodles': The first serious game intervention for siblings of children with disabilities.

Objective: Serious games can serve as easily accessible interventions to support siblings of children with disabilities, who are at risk of developing mental health problems. The Dutch serious game 'Broodles' was developed for siblings aged 6-9 years. The current study aims to assess the cultural applicability, desirability, feasibility, and acceptability of 'Broodles' in Norway. Methods: Norwegian siblings (N = 16) aged 6-13 years and parents (N = 12) of children with intellectual disabilities assessed the game. Their feedback data from interviews and questionnaires were sorted using a model of engagement factors in serious games. Results: At pre-use, participants showed interest in the game, and after initial use the participants were overall positive about the format, content and objectives, including validation of emotions and recognition. The participants had suggestions for improved engagement and feasibility. Conclusion: The game was found to be culturally applicable, desirable and acceptable, although Norwegian translation is necessary for further evaluation. Recommendations to enhance engagement were provided, including suggestions to play the game with parents or in a group. Innovation: This initial assessment of the serious game Broodles in a non-Dutch setting shows promise for an innovative way of supporting siblings of children with disabilities.

A multimodal virtual vision platform as a next-generation vision system for a surgical robot.

Robot-assisted surgery platforms are utilized globally thanks to their stereoscopic vision systems and enhanced functional assistance. However, the necessity of ergonomic improvement for their use by surgeons has been increased. In surgical robots, issues with chronic fatigue exist owing to the fixed posture of the conventional stereo viewer (SV) vision system. A head-mounted display was adopted to alleviate the inconvenience, and a virtual vision platform (VVP) is proposed in this study. The VVP can provide various critical data, including medical images, vital signs, and patient records, in three-dimensional virtual reality space so that users can access medical information simultaneously. An availability of the VVP was investigated based on various user evaluations by surgeons and novices, who executed the given tasks and answered questionnaires. The performances of the SV and VVP were not significantly different; however, the craniovertebral angle of the VVP was 16.35° higher on average than that of the SV. Survey results regarding the VVP were positive; participants indicated that the optimal number of displays was six, preferring the 2 × 3 array. Reflecting the tendencies, the VVP can be a neoconceptual candidate to be customized for medical use, which opens a new prospect in a next-generation surgical robot.

A shared robot control system combining augmented reality and motor imagery brain-computer interfaces with eye tracking.

Objective: Brain-computer interface (BCI) control systems monitor neural activity to detect the user's intentions, enabling device control through mental imagery. Despite their potential, decoding neural activity in real-world conditions poses significant challenges, making BCIs currently impractical compared to traditional interaction methods. This study introduces a novel motor imagery (MI) BCI control strategy for operating a physically assistive robotic arm, addressing the difficulties of MI decoding from electroencephalogram (EEG) signals, which are inherently non-stationary and vary across individuals. Approach: A proof-of-concept BCI control system was developed using commercially available hardware, integrating MI with eye tracking in an augmented reality (AR) user interface to facilitate a shared control approach. This system proposes actions based on the user's gaze, enabling selection through imagined movements. A user study was conducted to evaluate the system's usability, focusing on its effectiveness and efficiency. Main results:Participants performed tasks that simulated everyday activities with the robotic arm, demonstrating the shared control system's feasibility and practicality in real-world scenarios. Despite low online decoding performance (mean accuracy: 0.52 9, F1: 0.29, Cohen's Kappa: 0.12), participants achieved a mean success rate of 0.83 in the final phase of the user study when given 15 minutes to complete the evaluation tasks. The success rate dropped below 0.5 when a 5-minute cutoff time was selected. Significance: These results indicate that integrating AR and eye tracking can significantly enhance the usability of BCI systems, despite the complexities of MI-EEG decoding. While efficiency is still low, the effectiveness of our approach was verified. This suggests that BCI systems have the potential to become a viable interaction modality for everyday applications in the future.

Evaluating the real-world usability of BCI control systems with augmented reality: a user study protocol.

Brain-computer interfaces (BCI) enable users to control devices through their brain activity. Motor imagery (MI), the neural activity resulting from an individual imagining performing a movement, is a common control paradigm. This study introduces a user-centric evaluation protocol for assessing the performance and user experience of an MI-based BCI control system utilizing augmented reality. Augmented reality is employed to enhance user interaction by displaying environment-aware actions, and guiding users on the necessary imagined movements for specific device commands. One of the major gaps in existing research is the lack of comprehensive evaluation methodologies, particularly in real-world conditions. To address this gap, our protocol combines quantitative and qualitative assessments across three phases. In the initial phase, the BCI prototype's technical robustness is validated. Subsequently, the second phase involves a performance assessment of the control system. The third phase introduces a comparative analysis between the prototype and an alternative approach, incorporating detailed user experience evaluations through questionnaires and comparisons with non-BCI control methods. Participants engage in various tasks, such as object sorting, picking and placing, and playing a board game using the BCI control system. The evaluation procedure is designed for versatility, intending applicability beyond the specific use case presented. Its adaptability enables easy customization to meet the specific user requirements of the investigated BCI control application. This user-centric evaluation protocol offers a comprehensive framework for iterative improvements to the BCI prototype, ensuring technical validation, performance assessment, and user experience evaluation in a systematic and user-focused manner.

Mobile Apps for Vaccination Services: Content Analysis and Quality Assessment.

BACKGROUND: Vaccination services are increasingly in demand by the public, and mobile apps are an effective tool to meet that demand. However, the characteristics and quality of these apps are unknown. OBJECTIVE: Commonly used vaccination service apps on the market were surveyed with regard to quality, service content, and user experience to evaluate and guide users. METHODS: The Qimai Data mobile app data analytics platform was used to search for common vaccination service apps by keyword, and the WeChat and Alipay platforms were searched for apps. The apps included in the study were independently evaluated by two reviewers using the Mobile Application Rating Scale, and the service content and user experience of the apps were analyzed. The intragroup correlation coefficient between raters was used to measure interrater reliability. RESULTS: In the app stores of the four major Android platforms and the iOS app store, 1092 and 207 apps were found, respectively; 189 WeChat applets and 30 Alipay applets were also found. A total of 29 apps was ultimately included in this study according to the inclusion criteria, including 21 independent apps, 4 WeChat applets, and 4 Alipay applets. Significant differences were found between independent apps and applets in terms of the quality score (t449.57=-5.301; P<.001) and the subjective quality score (z=-4.753; P<.001). No significant differences were found between iOS and Android platforms in terms of the quality score (t1404=-2.55; P=.80) and the subjective quality score (z=-0.137; P=.89). There was good intragroup consistency among the raters. CONCLUSIONS: In this study, independent apps and nonindependent apps that rely on social and payment platforms for implementation were included in the vaccination services category. The overall quality of these apps was acceptable. Nonindependent running apps were found to have slightly lower scores and showed room for improvement, and scores for the participatory apps were found to be generally low overall.

User evaluation of a mobile education application for the management of metabolic syndrome among cancer survivors.

OBJECTIVE: This study aimed to develop a smartphone education application for managing metabolic syndrome among cancer survivors and obtain user evaluation based on quantitative and qualitative data. METHODS: Ten cancer survivors and 10 oncology nurse specialists responded to a structured usability evaluation tool (Mobile Application Rating Scale: MARS). Quantitative data analysis was performed through descriptive statistics using SPSS version 25.0. We conducted semi-structured interviews of the cancer survivors and oncology nurse specialists. Qualitative data of interview responses were coded as the app's strengths and weaknesses, information, motivation, and behavioral change. RESULTS: The overall usability evaluation score of the app was 3.66 ± 0.39 for cancer survivors and 3.79 ± 0.20 for oncology nurse specialists. Both cancer survivors and oncology nurse specialists scored the area of functionality as the highest and engagement as the lowest. Additionally, the qualitative usability evaluation suggested that the app should be visually improved by including figures and tables to enhance readability and providing videos and more specific guidelines to directly elicit behavioral change. CONCLUSIONS: Metabolic syndrome in cancer survivors can be effectively managed by using the educational application developed in this study by improving the shortcomings of the app for cancer survivors.

Corrigendum: Riemannian geometry-based metrics to measure and reinforce user performance changes during brain-computer interface user training.

[This corrects the article DOI: 10.3389/fncom.2023.1108889.].

Riemannian geometry-based metrics to measure and reinforce user performance changes during brain-computer interface user training.

Despite growing interest and research into brain-computer interfaces (BCI), their usage remains limited outside of research laboratories. One reason for this is BCI inefficiency, the phenomenon where a significant number of potential users are unable to produce machine-discernible brain signal patterns to control the devices. To reduce the prevalence of BCI inefficiency, some have advocated for novel user-training protocols that enable users to more effectively modulate their neural activity. Important considerations for the design of these protocols are the assessment measures that are used for evaluating user performance and for providing feedback that guides skill acquisition. Herein, we present three trial-wise adaptations (running, sliding window and weighted average) of Riemannian geometry-based user-performance metrics (classDistinct reflecting the degree of class separability and classStability reflecting the level of within-class consistency) to enable feedback to the user following each individual trial. We evaluated these metrics, along with conventional classifier feedback, using simulated and previously recorded sensorimotor rhythm-BCI data to assess their correlation with and discrimination of broader trends in user performance. Analysis revealed that the sliding window and weighted average variants of our proposed trial-wise Riemannian geometry-based metrics more accurately reflected performance changes during BCI sessions compared to conventional classifier output. The results indicate the metrics are a viable method for evaluating and tracking user performance changes during BCI-user training and, therefore, further investigation into how these metrics may be presented to users during training is warranted.

Comparing Social Robot Embodiment for Child Musical Education.

The present research focuses in the comparison of two social robot models running the same Human-Robot Interaction (HRI) applications targeting the context of music education for children aged 9-11, with the objective of underlying the design choices favored by the target audience on the running tasks. The Guitar Tuner consists of two main functionalities: tuning process and performance evaluation, which we implemented using the NAO and Zenbo robots. User evaluation included 20 children and assessed their perceived robot embodiment preferences (e.g., shape, robot motion, displays, and emotional expressivity) and perceived usability aspects. The evaluation used an experimental remote protocol supporting collecting online feedback with users during the COVID-19 pandemic. Empirical results supported performing quantitative and qualitative evaluations of the HRI application and highlighting the perceived differences of robot embodiment features. The discussions center on improving a future version of the HRI application, plus children's considerations about their preferred robot embodiment features during the observation sessions. Finally, we propose recommendations for robot embodiment design for children and learning based on this case study and discuss protocol limitations during the social distancing context, that we believe as a valid alternative to move forward with experimental designs, particularly in robotics, becoming a great contribution to other researchers facing similar hurdles.

A Scholarly Knowledge Graph-Powered Dashboard: Implementation and User Evaluation.

Scholarly knowledge graphs provide researchers with a novel modality of information retrieval, and their wider use in academia is beneficial for the digitalization of published works and the development of scholarly communication. To increase the acceptance of scholarly knowledge graphs, we present a dashboard, which visualizes the research contributions on an educational science topic in the frame of the Open Research Knowledge Graph (ORKG). As dashboards are created at the intersection of computer science, graphic design, and human-technology interaction, we used these three perspectives to develop a multi-relational visualization tool aimed at improving the user experience. According to preliminary results of the user evaluation survey, the dashboard was perceived as more appealing than the baseline ORKG-powered interface. Our findings can be used for the development of scholarly knowledge graph-powered dashboards in different domains, thus facilitating acceptance of these novel instruments by research communities and increasing versatility in scholarly communication.

Gaze-driven placement of items for proactive visual exploration.

Recent advances in digital signage technology have improved the ability to visually select specific items within a group. Although this is due to the ability to dynamically update the display of items, the corresponding layout schemes remain a subject of research. This paper explores the sophisticated layout of items by respecting the underlying context of searching for favorite items. Our study begins by formulating the static placement of items as an optimization problem that incorporates aesthetic layout criteria as constraints. This is further extended to accommodate the dynamic placement of items for more proactive visual exploration based on the ongoing search context. Our animated layout is driven by analyzing the distribution of eye gaze through an eye-tracking device, by which we infer how the most attractive items lead to the finally wanted ones. We create a planar layout of items as a context map to establish association rules to dynamically replace existing items with new ones. For this purpose, we extract the set of important topics from a set of annotated texts associated with the items using matrix factorization. We also conduct user studies to evaluate the validity of the design criteria incorporated into both static and dynamic placement of items. After discussing the pros and cons of the proposed approach and possible themes for future research, we conclude this paper.