Optical Micro/Nanofiber Enabled Multiaxial Force Sensor for Tactile Visualization and Human-Machine Interface.

Tactile sensors with capability of multiaxial force perception play a vital role in robotics and human-machine interfaces. Flexible optical waveguide sensors have been an emerging paradigm in tactile sensing due to their high sensitivity, fast response, and antielectromagnetic interference. Herein, a flexible multiaxial force sensor enabled by U-shaped optical micro/nanofibers (MNFs) is reported. The MNF is embedded within an elastomer film topped with a dome-shaped protrusion. When the protrusion is subjected to vector forces, the embedded MNF undergoes anisotropic deformations, yielding time-resolved variations in light transmission. Detection of both normal and shear forces is achieved with sensitivities reaching 50.7 dB N-1 (14% kPa-1) and 82.2 dB N-1 (21% kPa-1), respectively. Notably, the structural asymmetry of the MNF induces asymmetrical optical modes, granting the sensor directional responses to four-directional shear forces. As proof-of-concept applications, tactile visualizations for texture and relief pattern recognition are realized with a spatial resolution of 160 µm. Moreover, a dual U-shaped MNF configuration is demonstrated as a human-machine interface for cursor manipulation. This work represents a step towards advanced multiaxial tactile sensing.

EMG and SSVEP-based bimodal estimation of elbow angle trajectory.

Detecting intentions and estimating movement trajectories in a human-machine interface (HMI) using electromyogram (EMG) signals is particularly challenging, especially for individuals with movement impairments. Therefore, incorporating additional information from other biological sources, potential discrete information in the movement, and the EMG signal can be practical. This study combined EMG and target information to enhance estimation performance during reaching movements. EMG activity of the shoulder and arm muscles, elbow angle, and the electroencephalogram signals of ten healthy subjects were recorded while they reached blinking targets. The reaching target was recognized by steady-state visual evoked potential (SSVEP). The selected target's final angle and EMG were then mapped to the elbow angle trajectory. The proposed bimodal structure, which integrates EMG and final elbow angle information, outperformed the EMG-based decoder. Even under conditions of higher fatigue, the proposed structure provided better performance than the EMG decoder. Including additional information about the recognized reaching target in the trajectory model improved the estimation of the reaching profile. Consequently, this study's findings suggest that bimodal decoders are highly beneficial for enhancing assistive robotic devices and prostheses, especially for real-time upper limb rehabilitation.

Investigating Implicit and Explicit Communication of Highly Automated Vehicles in Japan: How do Light-band eHMIs affect Pedestrians' Willingness to Cross, Trust and Perceived Safety?

In the near future, pedestrians will face highly automated vehicles on the roads. Highly automated vehicles (HAVs) should have safety-enhancing communication tools to guarantee traffic safety, e.g., vehicle kinematics and external human-machine interfaces (eHMIs). Pedestrians, as highly vulnerable road users, depend on communication with HAVs. Miscommunication between pedestrians and HAVs could quickly result in accidents, and this, in turn, could cause severe impairments for pedestrians. Light-band eHMIs have the potential to enhance traffic safety. However, eHMIs have been less explored in Japan so far. As a first-time approach, this experimental online study shed light on the effect of a light-band eHMI on Japanese pedestrians (N=99). In short video sequences, the participants interacted with two differently sized HAVs equipped with light-band eHMI. We investigated the effect of vehicle size (small vs. large), eHMI status (no eHMI vs. static eHMI vs. dynamic eHMI), and vehicle kinematics (yielding vs. non-yielding) on pedestrians' willingness to cross, trust, and perceived safety. To investigate possible side effects of eHMIs, we also included experimental conditions in which the eHMI mismatched the vehicle's kinematics. Results revealed that Japanese were more willing to cross the street and indicated higher trust- and safety ratings when they received information about the vehicle's intention and automation status (dynamic eHMI) compared to when they received no information (no eHMI) or only about the vehicle automation status (static eHMI). Surprisingly, Japanese participants tended to rely on the eHMI when there was mismatching information between eHMI and vehicle kinematics. Overall, we concluded that light-band eHMIs could contribute to a safe future interaction between pedestrians and HAVs in Japan under the requirement that the eHMI is in accordance with vehicle kinematics.

Design of a Gaze-Controlled Interactive Art System for the Elderly to Enjoy Life.

The impact of global population aging on older adults' health and emotional well-being is examined in this study, emphasizing innovative technological solutions to address their diverse needs. Changes in physical and mental functions due to aging, along with emotional challenges that necessitate attention, are highlighted. Gaze estimation and interactive art are utilized to develop an interactive system tailored for elderly users, where interaction is simplified through eye movements to reduce technological barriers and provide a soothing art experience. By employing multi-sensory stimulation, the system aims to evoke positive emotions and facilitate meaningful activities, promoting active aging. Named "Natural Rhythm through Eyes", it allows for users to interact with nature-themed environments via eye movements. User feedback via questionnaires and expert interviews was collected during public demonstrations in elderly settings to validate the system's effectiveness in providing usability, pleasure, and interactive experience for the elderly. Key findings include the following: (1) Enhanced usability of the gaze estimation interface for elderly users. (2) Increased enjoyment and engagement through nature-themed interactive art. (3) Positive influence on active aging through the integration of gaze estimation and interactive art. These findings underscore technology's potential to enhance well-being and quality of life for older adults navigating aging challenges.

Hello, is it me you're Stopping for? The Effect of external Human Machine Interface Familiarity on Pedestrians' Crossing Behaviour in an Ambiguous Situation.

OBJECTIVE: We investigated how different deceleration intentions (i.e. an automated vehicle either decelerated for leading traffic or yielded for pedestrians) and a novel (Slow Pulsing Light Band - SPLB) or familiar (Flashing Headlights - FH) external Human Machine Interface (eHMI) informed pedestrians' crossing behaviour. BACKGROUND: The introduction of SAE Level 4 Automated Vehicles (AVs) has recently fuelled interest in new forms of explicit communication via eHMIs, to improve the interaction between AVs and surrounding road users. Before implementing these eHMIs, it is necessary to understand how pedestrians use them to inform their crossing decisions. METHOD: Thirty participants took part in the study using a Head-Mounted Display. The independent variables were deceleration intentions and eHMI design. The percentage of crossings, collision frequency and crossing initiation time across trials were measured. RESULTS: Pedestrians were able to identify the intentions of a decelerating vehicle, using implicit cues, with more crossings made when the approaching vehicles were yielding to them. They were also more likely to cross when a familiar eHMI was presented, compared to a novel one or no eHMI, regardless of the vehicle's intention. Finally, participants learned to take a more cautious approach as trials progressed, and not to base their decisions solely on the eHMI. CONCLUSION: A familiar eHMI led to early crossings regardless of the vehicle's intention but also led to a higher collision frequency than a novel eHMI. APPLICATION: To achieve safe and acceptable interactions with AVs, it is important to provide eHMIs that are congruent with road users' expectations.

Peer-to-Peer Ultra-Wideband Localization for Hands-Free Control of a Human-Guided Smart Stroller.

We propose a hands-free control system for a human-guided smart stroller. The proposed method uses real-time peer-to-peer localization technology of the human and stroller to realize an intuitive hands-free control system based on the relative position between the human and the stroller. The control method is also based on functional and mechanical safety to ensure the safety of the stroller's occupant (child) and the pilot (parent) during locomotion. In this paper, first, we present a preliminary investigation of the humans' preference for the relative position in the context of hands-free guided strollers. Then, we present the control method and a prototype implemented with an electric wheelchair and UWB sensors for localization. We present an experimental evaluation of the proposed method with 14 persons walking with the developed prototype to investigate the usability and soundness of the proposed method compared to a remote joystick and manual operation. The evaluation experiments were conducted in an indoor environment and revealed that the proposed method matches the performance of joystick control but does not perform as well as manual operation. Notably, for female participants, the proposed method significantly surpasses joystick performance and achieves parity with manual operation, which shows its efficacy and potential for a smart stroller. Also, the results revealed that the proposed method significantly decreased the user's physical load compared to the manual operation. We present discussions on the controllability, usability, task load, and safety features of the proposed method, and conclude this work with a summary assessment.

Development of a human machine interface for robotically assisted surgery optimized for laparoscopic workflows.

INTRODUCTION: In robotic-assisted surgery (RAS), the input device is the primary site for the flow of information between the user and the robot. Most RAS systems remove the surgeon's console from the sterile surgical site. Beneficial for performing lengthy procedures with complex systems, this ultimately lacks the flexibility that comes with the surgeon being able to remain at the sterile site. METHODS: A prototype of an input device for RAS is constructed. The focus lies on intuitive control for surgeons and a seamless integration into the surgical workflow within the sterile environment. The kinematic design is translated from the kinematics of laparoscopic surgery. The input device uses three degrees of freedom from a flexible instrument as input. The prototype's performance is compared to that of a commercially available device in an evaluation. Metrics are used to evaluate the surgeons' performance with the respective input device in a virtual environment implemented for the evaluation. RESULTS: The evaluation of the two input devices shows statistically significant differences in the performance metrics. With the proposed prototype, the surgeons perform the tasks faster, more precisely, and with fewer errors. CONCLUSION: The prototype is an efficient and intuitive input device for surgeons with laparoscopic experience. The placement in the sterile working area allows for seamless integration into the surgical workflow and can potentially enable new robotic approaches.

Interactive Deformable Colored Sound Display Achieved with Electrostrictive Fluoropolymer and Halide Perovskite.

The association of color and sound helps human cognition through a synergetic effect like intersensory facilitation. Although soft human-machine interfaces (HMIs) providing unisensory expression have been widely developed, achieving synchronized optic and acoustic expression in one device system has been relatively less explored. It is because their operating principles are different in terms of materials, and implementation has mainly been attempted through structural approaches. Here, a deformable sound display is developed that generates multiple colored lights with large sound at low input voltage. The device is based on alternating-current electroluminescence (ACEL) covered with perovskite composite films. A sound wave is created by a polymer matrix of the ACEL, while simultaneously, various colors are produced by the perovskite films and the blue electroluminescence (EL) emitted from the phosphors in the ACEL. By patterning different colored perovskite films onto the ACELs, associating the color and the sound is successfully demonstrated by a piano keyboard and a wearable interactive device.

[ADAS and automation-a relevant contribution to maintaining mobility of older drivers?] / Fahrerassistenzsysteme und automatisiertes Fahren ­ ein relevanter Beitrag zum Erhalt der Mobilität im Alter?

Driving is the most important and safest form of mobility for the majority of senior citizens. However, physical and mental performance gradually decline with age, which can lead to more problems, critical situations or even accidents. Vehicle technology innovations such as advanced driver assistance systems (ADAS) have the potential to increase the road safety of older people and maintain their individual mobility for as long as possible.This overview article aims to identify ADAS that have the greatest potential to reduce the number of accidents involving older drivers. For this purpose, the accident and damage occurrence as well as the driving behaviour and compensation strategies of older people are examined in more detail. Suitable ADAS should compensate for typical driver errors, reduce information deficiencies and have a high level of acceptance. For older drivers, emergency braking, parking assistance, navigation, intersection assistance and distance speed control systems as well as systems for detecting blind spots and obstacles appear to be particularly suitable.Some of the disadvantages of ADAS are the lack of market penetration, acceptance problems and interface designs that have not yet been optimally adapted to the needs of older users. For older drivers in particular, it appears to be a priority to develop coherent and integrated solutions in the sense of cooperative assistance instead of pushing ahead with high and full automation with many system limits and exceptions, which can place high demands on attention, for example if the vehicle has to be taken over in a critical situation.

MXene-Deposited Melamine Foam-Based Iontronic Pressure Sensors for Wearable Electronics and Smart Numpads.

Iontronic pressure sensors hold significant potential to emerge as vital components in the field of flexible and wearable electronics, addressing a variety of applications spanning wearable technology, health monitoring systems, and human-machine interactions. This study introduces a novel iontronic pressure sensor structure based on a seamlessly deposited Ti3C2Tx MXene layer onto highly porous melamine foam as parallel plate electrodes and an ionically conductive electrolyte of 1-ethyl-3-methylimidazolium bis(trifluoromethylsulfonyl)imide/thermoplastic polyurethane coupled with carbon cloth as current collecting layers for improved sensitivity and high mechanical stability of more than 7000 cycles. MXene-deposited melamine foam-based iontronic pressure sensors (MIPS) showed a high sensitivity of 5.067 kPa-1 in the range of 45-60 kPa and a fast response/recovery time of 28/18 ms, respectively. The high sensitivity, high mechanical stability, and fast response/recovery time of the designed sensor make them highly promising candidates for real-time body motion monitoring. Moreover, sensors are employed as a smart numpad for integration into advanced ATM security systems utilizing machine learning algorithms. This research marks a significant advance in iontronic pressure sensor technology, offering promising avenues for application in wearable electronics and security systems.

Virtual/augmented reality-based human-machine interface and interaction modes in airport control towers.

The concept of an innovative human-machine interface and interaction modes based on virtual and augmented reality technologies for airport control towers has been developed with the aim of increasing the human performances and situational awareness of air traffic control operators. By presenting digital information through see-through head-mounted displays superimposed over the out-of-the-tower view, the proposed interface should stimulate controllers to operate in a head-up position and, therefore, reduce the number of switches between a head-up and a head-down position even in low visibility conditions. This paper introduces the developed interface and describes the exercises conducted to validate the technical solutions developed, focusing on the simulation platform and exploited technologies, to demonstrate how virtual and augmented reality, along with additional features such as adaptive human-machine interface, multimodal interaction and attention guidance, enable a more natural and effective interaction in the control tower. The results of the human-in-the-loop real-time validation exercises show that the prototype concept is feasible from both an operational and technical perspective, the solution proves to support the air traffic controllers in working in a head-up position more than head-down even with low-visibility operational scenarios, and to lower the time to react in critical or alerting situations with a positive impact on the human performances of the user. While showcasing promising results, this study also identifies certain limitations and opportunities for refinement, aimed at further optimising the efficacy and usability of the proposed interface.

Neuromorphic Computing-Assisted Triboelectric Capacitive-Coupled Tactile Sensor Array for Wireless Mixed Reality Interaction.

Flexible tactile sensors show promise for artificial intelligence applications due to their biological adaptability and rapid signal perception. Triboelectric sensors enable active dynamic tactile sensing, while integrating static pressure sensing and real-time multichannel signal transmission is key for further development. Here, we propose an integrated structure combining a capacitive sensor for static spatiotemporal mapping and a triboelectric sensor for dynamic tactile recognition. A liquid metal-based flexible dual-mode triboelectric-capacitive-coupled tactile sensor (TCTS) array of 4 × 4 pixels achieves a spatial resolution of 7 mm, exhibiting a pressure detection limit of 0.8 Pa and a fast response of 6 ms. Furthermore, neuromorphic computing using the MXene-based synaptic transistor achieves 100% recognition accuracy of handwritten numbers/letters within 90 epochs based on dynamic triboelectric signals collected by the TCTS array, and cross-spatial information communication from the perceived multichannel tactile data is realized in the mixed reality space. The results illuminate considerable application possibilities of dual-mode tactile sensing technology in human-machine interfaces and advanced robotics.

Ultrahigh Sensitivity for Thermographic Human-Machine Interface via Precious Metals Atomic Layer Deposition on V-MXene: Computational and Experimental Exploration.

Global healthcare based on the Internet of Things system is rapidly transforming to measure precise physiological body parameters without visiting hospitals at remote patients and associated symptoms monitoring. 2D materials and the prevailing mood of current ever-expanding MXene-based sensing devices motivate to introduce first the novel iridium (Ir) precious metal incorporated vanadium (V)-MXene via industrially favored emerging atomic layer deposition (ALD) techniques. The current work contributes a precise control and delicate balance of Ir single atomic forms or clusters on the V-MXene to constitute a unique precious metal-MXene embedded heterostructure (Ir-ALD@V-MXene) in practical real-time sensing healthcare applications to thermography with human-machine interface for the first time. Ir-ALD@V-MXene delivers an ultrahigh durability and sensing performance of 2.4% °C-1 than pristine V-MXene (0.42% °C-1), outperforming several conventionally used MXenes, graphene, underscoring the importance of the Ir-ALD innovative process. Aberration-corrected advanced ultra-high-resolution transmission/scanning transmission electron microscopy confirms the presence of Ir atomic clusters on well-aligned 2D-layered V-MXene structure and their advanced heterostructure formation (Ir-ALD@V-MXene), enhanced sensing mechanism is investigated using density functional theory (DFT) computations. A rational design empowering the Ir-ALD process on least explored V-MXene can potentially unfold further precious metals ALD-process developments for next-generation wearable personal healthcare devices.

Selecting the Appropriate Speed for Rotational Elements in Human-Machine Interfaces: A Quantitative Study.

The motion of rotation, which served as a dynamic symbol within human-computer interfaces, has garnered extensive attention in interface and graphic design. This study aimed to establish speed benchmarks for interface design by exploring visual system preferences for the perception of both simple and complex rotating icons within the velocity range of 5-1800 degrees per second. The research conducted two experiments with 12 participants to examine the observers' just noticeable difference in speed (JNDS) and perceived speed for rotational icons. Experiment one measured the JNDS over eight-speed levels using a constant stimulus method, achieving a range of 14.9-29%. Building on this, experiment two proposed a sequence of speeds within the given range and used a rating scale method to assess observers ' subjective perception of the speed series' rapidity. The findings indicated that speed increases impacted the ability to differentiate between speeds; key points for categorizing low, medium, and high speeds were identified at 10, 180, and 720 degrees/s, respectively. Shape complexity was found to modulate the visual system's perception of actual speed, such that at rotation speeds above 180 degrees/s, complex icons appeared to rotate faster than simpler ones. Most importantly, the study applied quantitative methods and metrology to interface design, offering a more scientific approach to the design workflow.

Intelligent Human-Computer Interaction: Combined Wrist and Forearm Myoelectric Signals for Handwriting Recognition.

Recent studies have highlighted the possibility of using surface electromyographic (EMG) signals to develop human-computer interfaces that are also able to recognize complex motor tasks involving the hand as the handwriting of digits. However, the automatic recognition of words from EMG information has not yet been studied. The aim of this study is to investigate the feasibility of using combined forearm and wrist EMG probes for solving the handwriting recognition problem of 30 words with consolidated machine-learning techniques and aggregating state-of-the-art features extracted in the time and frequency domains. Six healthy subjects, three females and three males aged between 25 and 40 years, were recruited for the study. Two tests in pattern recognition were conducted to assess the possibility of classifying fine hand movements through EMG signals. The first test was designed to assess the feasibility of using consolidated myoelectric control technology with shallow machine-learning methods in the field of handwriting detection. The second test was implemented to assess if specific feature extraction schemes can guarantee high performances with limited complexity of the processing pipeline. Among support vector machine, linear discriminant analysis, and K-nearest neighbours (KNN), the last one showed the best classification performances in the 30-word classification problem, with a mean accuracy of 95% and 85% when using all the features and a specific feature set known as TDAR, respectively. The obtained results confirmed the validity of using combined wrist and forearm EMG data for intelligent handwriting recognition through pattern recognition approaches in real scenarios.

The effect of human-machine interface modality, specificity, and timing on driver performance and behavior while using vehicle automation.

The effectiveness of the human-machine interface (HMI) in a driving automation system during takeover situations is based, in part, on its design. Past research has indicated that modality, specificity, and timing of the HMI have an impact on driver behavior. The objective of this study was to examine the effectiveness of two HMIs, which vary by modality, specificity, and timing, on drivers' takeover time, performance, and eye glance behavior. Drivers' behavior was examined in a driving simulator study with different levels of automation, varying traffic conditions, and while completing a non-driving related task. Results indicated that HMI type had a statistically significant effect on velocity and off-road eye glances such that those who were exposed to an HMI that gave multimodal warnings with greater specificity exhibited better performance. There were no effects of HMI on acceleration, lane position, or other eye glance metrics (e.g., on road glance duration). Future work should disentangle HMI design further to determine exactly which aspects of design yield between safety critical behavior.

EEG and EMG-based human-machine interface for navigation of mobility-related assistive wheelchair (MRA-W).

The control of human-machine interfaces (HMIs), such as motorized wheelchairs, has been widely investigated using biopotentials produced by electrochemical processes in the human body. However, many studies in this field sometimes overlook crucial factors like special users' needs, who often have inadequate muscle mass and strength, and paresis needed to operate a wheelchair. This study proposes a novel solution: an economical, universally compatible, and user-centric manual-to-powered wheelchair conversion kit. The powered wheelchair is operated using a hybrid control system integrating electroencephalogram (EEG) and electromyography (EMG), utilizing an LSTM network. It uses a low-cost electroencephalogram (EEG) headset and a wearable electromyography (EMG) electrode armband to solve these constraints. The proposed system comprised three crucial objectives: the development of an EEG-based user attentive detection system, an EMG-based navigation system, and a transform conventional wheelchair into a powered wheelchair. Human test subjects were utilized to evaluate the proposed system, and the study complied with accepted ethical guidelines. We selected four EEG features (p < 0.023) for the attentive detection system and six EMG features (p < 0.037) to detect navigation intentions. User attentive detection was achieved at 83.33 (±0.34) %, while the navigation intention system produced 86.67 (±0.52) % accuracy. The overall system was successful in reaching an accuracy rate of 85.0 (±0.19) % and a weighted average precision of 0.89. After the dataset was trained using an LSTM network, the overall accuracy produced was 97.3 (±0.5) %, higher than the accuracy produced by the Quadratic SVM classifier. By giving older and disabled people a more convenient way to use powered wheelchairs, this research helps to build ergonomic and cost-effective biopotential-based HMIs, enhancing their quality of life.

Hazard warning modalities and timing thresholds for older drivers with impaired vision.

PURPOSE: We examined collision warning systems with different modalities and timing thresholds, assessing their impact on responses to pedestrian hazards by drivers with impaired contrast sensitivity (ICS). METHODS: Seventeen ICS (70-84 y, median CS 1.35 log units) and 17 normal vision (NV: 68-73 y, median CS 1.95) participants completed 6 city drives in a simulator with 3 bimodal warnings: visual-auditory, visual-directional-tactile, and visual-non-directional-tactile. Each modality had one drive with early and one with late warnings, triggered at 3.5 s and 2 s time-to-collision, respectively. RESULTS: ICS participants triggered more early (43 vs 37 %) and late warnings (12 vs 6 %) than NV participants and had more collisions (3 vs 0 %). Early warnings reduced time to fixate hazards (late 1.9 vs early 1.2 s, p < 0.001), brake response times (2.8 vs 1.8 s, p < 0.001) and collision rates (1.2 vs 0.02 %). With late warnings, ICS participants took 0.7 s longer to brake than NV (p < 0.001) and had an 11 % collision rate (vs 0.7 % with early warnings). Non-directional-tactile warnings yielded the lowest collision rates for ICS participants (4 vs auditory 12 vs directional-tactile 15.2 %) in late warning scenarios. All ICS participants preferred early warnings. CONCLUSIONS: While early warnings improved hazard responses and reduced collisions for ICS participants, late warnings did not, resulting in high collision rates. In contrast, both early and late warnings were helpful for NV drivers. Non-directional-tactile warnings were the most effective in reducing collisions. The findings provide insights relevant to the development of hazard warnings tailored for drivers with impaired vision.