Automation function and malfunction: effects on human performance in accident handling tasks.

By conducting a mixed-design experiment using simplified accident handling tasks performed by two-person teams, this study examined the effects of automation function and condition (before, during, and after malfunction) on human performance. Five different and non-overlapping functions related to human information processing model were considered and their malfunctions were set in a first-failure way. The results showed that while the automation malfunction impaired task performance, the performance degradation for information analysis was more severe than response planning. Contrary to other functions, the situation awareness for response planning and response implementation tended to increase during malfunctioning and decrease after. In addition, decreased task performance reduced trust in automation, and malfunctions in earlier stages of information processing resulted in lower trust. Suggestions provided for the design and training related to automation emphasise the importance of high-level cognitive support and the benefit of involving automation error handling in training.

The effects of automation function and malfunction on human performance are important for design and training. The experimental results in this study revealed the significance of high-level cognitive support. Also, introducing automation error handling in training can be helpful in improving situation awareness of the teams.

Cross-Subject Lifelong Learning for Continuous Estimation From Surface Electromyographic Signal.

The employment of surface electromyographic (sEMG) signals in the estimation of hand kinematics represents a promising non-invasive methodology for the advancement of human-machine interfaces. However, the limitations of existing subject-specific methods are obvious as they confine the application to individual models that are custom-tailored for specific subjects, thereby reducing the potential for broader applicability. In addition, current cross-subject methods are challenged in their ability to simultaneously cater to the needs of both new and existing users effectively. To overcome these challenges, we propose the Cross-Subject Lifelong Network (CSLN). CSLN incorporates a novel lifelong learning approach, maintaining the patterns of sEMG signals across a varied user population and across different temporal scales. Our method enhances the generalization of acquired patterns, making it applicable to various individuals and temporal contexts. Our experimental investigations, encompassing both joint and sequential training approaches, demonstrate that the CSLN model not only attains enhanced performance in cross-subject scenarios but also effectively addresses the issue of catastrophic forgetting, thereby augmenting training efficacy.

Optimizing Human-Robot Teaming Performance through Q-Learning-Based Task Load Adjustment and Physiological Data Analysis.

The transition to Industry 4.0 and 5.0 underscores the need for integrating humans into manufacturing processes, shifting the focus towards customization and personalization rather than traditional mass production. However, human performance during task execution may vary. To ensure high human-robot teaming (HRT) performance, it is crucial to predict performance without negatively affecting task execution. Therefore, to predict performance indirectly, significant factors affecting human performance, such as engagement and task load (i.e., amount of cognitive, physical, and/or sensory resources required to perform a particular task), must be considered. Hence, we propose a framework to predict and maximize the HRT performance. For the prediction of task performance during the development phase, our methodology employs features extracted from physiological data as inputs. The labels for these predictions-categorized as accurate performance or inaccurate performance due to high/low task load-are meticulously crafted using a combination of the NASA TLX questionnaire, records of human performance in quality control tasks, and the application of Q-Learning to derive task-specific weights for the task load indices. This structured approach enables the deployment of our model to exclusively rely on physiological data for predicting performance, thereby achieving an accuracy rate of 95.45% in forecasting HRT performance. To maintain optimized HRT performance, this study further introduces a method of dynamically adjusting the robot's speed in the case of low performance. This strategic adjustment is designed to effectively balance the task load, thereby enhancing the efficiency of human-robot collaboration.

Evaluating initial usability of a hand augmentation device across a large and diverse sample.

The advancement of motor augmentation and the broader domain of human-machine interaction rely on a seamless integration with users' physical and cognitive capabilities. These considerations may markedly fluctuate among individuals on the basis of their age, form, and abilities. There is a need to develop a standard for considering these diversity needs and preferences to guide technological development, and large-scale testing can provide us with evidence for such considerations. Public engagement events provide an important opportunity to build a bidirectional discourse with potential users for the codevelopment of inclusive and accessible technologies. We exhibited the Third Thumb, a hand augmentation device, at a public engagement event and tested participants from the general public, who are often not involved in such early technological development of wearable robotic technology. We focused on wearability (fit and control), ability to successfully operate the device, and ability levels across diversity factors relevant for physical technologies (gender, handedness, and age). Our inclusive design was successful in 99.3% of our diverse sample of 596 individuals tested (age range from 3 to 96 years). Ninety-eight percent of participants were further able to successfully manipulate objects using the extra thumb during the first minute of use, with no significant influences of gender, handedness, or affinity for hobbies involving the hands. Performance was generally poorer among younger children (aged ≤11 years). Although older and younger adults performed the task comparably, we identified age costs with the older adults. Our findings offer tangible demonstration of the initial usability of the Third Thumb for a broad demographic.

Perception of experience influences altruism and perception of agency influences trust in human-machine interactions.

As robots become increasingly integrated into social economic interactions, it becomes crucial to understand how people perceive a robot's mind. It has been argued that minds are perceived along two dimensions: experience, i.e., the ability to feel, and agency, i.e., the ability to act and take responsibility for one's actions. However, the influence of these perceived dimensions on human-machine interactions, particularly those involving altruism and trust, remains unknown. We hypothesize that the perception of experience influences altruism, while the perception of agency influences trust. To test these hypotheses, we pair participants with bot partners in a dictator game (to measure altruism) and a trust game (to measure trust) while varying the bots' perceived experience and agency, either by manipulating the degree to which the bot resembles humans, or by manipulating the description of the bots' ability to feel and exercise self-control. The results demonstrate that the money transferred in the dictator game is influenced by the perceived experience, while the money transferred in the trust game is influenced by the perceived agency, thereby confirming our hypotheses. More broadly, our findings support the specificity of the mind hypothesis: Perceptions of different dimensions of the mind lead to different kinds of social behavior.

Human-cobot collaboration's impact on success, time completion, errors, workload, gestures and acceptability during an assembly task.

The 5.0 industry promotes collaborative robots (cobots). This research studies the impacts of cobot collaboration using an experimental setup. 120 participants realized a simple and a complex assembly task. 50% collaborated with another human (H/H) and 50% with a cobot (H/C). The workload and the acceptability of the cobotic collaboration were measured. Working with a cobot decreases the effect of the task complexity on the human workload and on the output quality. However, it increases the time completion and the number of gestures (while decreasing their frequency). The H/C couples have a higher chance of success but they take more time and more gestures to realize the task. The results of this research could help developers and stakeholders to understand the impacts of implementing a cobot in production chains.

Impacts of information quantity and display formats on driving behaviors in a connected vehicle environment.

The emerging connected vehicle (CV) technologies facilitate the development of integrated advanced driver assistance systems (ADASs), with which various functions are coordinated in a comprehensive framework. However, challenges arise in enabling drivers to perceive important information with minimal distractions when multiple messages are simultaneously provided by integrated ADASs. To this end, this study introduces three types of human-machine interfaces (HMIs) for an integrated ADAS: 1) three messages using a visual display only, 2) four messages using a visual display only, and 3) three messages using visual plus auditory displays. Meanwhile, the differences in driving performance across three HMI types are examined to investigate the impacts of information quantity and display formats on driving behaviors. Additionally, variations in drivers' responses to the three HMI types are examined. Driving behaviors of 51 drivers with respect to three HMI types are investigated in eight field testing scenarios. These scenarios include warnings for rear-end collision, lateral collision, forward collision, lane-change, and curve speed, as well as notifications for emergency events downstream, the specified speed limit, and car-following behaviors. Results indicate that, compared to a visual display only, presenting three messages through visual and auditory displays enhances driving performance in four typical scenarios. Compared to the presentation of three messages, a visual display offering four messages improves driving performance in rear-end collision warning scenarios but diminishes the performance in lane-change scenarios. Additionally, the relationship between information quantity and display formats shown on HMIs and driving performance can be moderated by drivers' gender, occupation, driving experience, annual driving distance, and safety attitudes. Findings are indicative to designers in automotive industries in developing HMIs for future CVs.

The effect of human-machine interface modality, specificity, and timing on driver performance and behavior while using vehicle automation.

The effectiveness of the human-machine interface (HMI) in a driving automation system during takeover situations is based, in part, on its design. Past research has indicated that modality, specificity, and timing of the HMI have an impact on driver behavior. The objective of this study was to examine the effectiveness of two HMIs, which vary by modality, specificity, and timing, on drivers' takeover time, performance, and eye glance behavior. Drivers' behavior was examined in a driving simulator study with different levels of automation, varying traffic conditions, and while completing a non-driving related task. Results indicated that HMI type had a statistically significant effect on velocity and off-road eye glances such that those who were exposed to an HMI that gave multimodal warnings with greater specificity exhibited better performance. There were no effects of HMI on acceleration, lane position, or other eye glance metrics (e.g., on road glance duration). Future work should disentangle HMI design further to determine exactly which aspects of design yield between safety critical behavior.

Deciding when to cross in front of an autonomous vehicle: How child and adult pedestrians respond to eHMI timing and vehicle kinematics.

How autonomous vehicles (AVs) communicate their intentions to vulnerable road users (e.g., pedestrians) is a concern given the rapid growth and adoption of this technology. At present, little is known about how children respond to external Human Machine Interface (eHMI) signals from AVs. The current study examined how adults and children respond to the combination of explicit (eHMI signals) and implicit information (vehicle deceleration) to guide their road-crossing decisions. Children (8- to 12-year-olds) and adults made decisions about when to cross in front of a driverless car in an immersive virtual environment. The car sometimes stopped, either abruptly or gradually (manipulated within subjects), to allow participants to cross. When yielding, the car communicated its intent via a dome light that changed from red to green and varied in its timing onset (manipulated between subjects): early eHMI onset, late eHMI onset, or control (no eHMI). As expected, we found that both children and adults waited longer to enter the roadway when vehicles decelerated abruptly than gradually. However, adults responded to the early eHMI signal by crossing sooner when the cars decelerated either gradually or abruptly compared to the control condition. Children were heavily influenced by the late eHMI signal, crossing later when the eHMI signal appeared late and the vehicle decelerated either gradually or abruptly compared to the control condition. Unlike adults, children in the control condition behaved similarly to children in the early eHMI condition by crossing before the yielding vehicle came to a stop. Together, these findings suggest that early eHMI onset may lead to riskier behavior (initiating crossing well before a gradually decelerating vehicle comes to a stop), whereas late eHMI onset may lead to safer behavior (waiting for the eHMI signal to appear before initiating crossing). Without an eHMI signal, children show a concerning overreliance on gradual vehicle deceleration to judge yielding intent.

Frequency-encoded eye tracking smart contact lens for human-machine interaction.

Eye tracking techniques enable high-efficient, natural, and effortless human-machine interaction by detecting users' eye movements and decoding their attention and intentions. Here, a miniature, imperceptible, and biocompatible smart contact lens is proposed for in situ eye tracking and wireless eye-machine interaction. Employing the frequency encoding strategy, the chip-free and battery-free lens successes in detecting eye movement and closure. Using a time-sequential eye tracking algorithm, the lens has a great angular accuracy of <0.5°, which is even less than the vision range of central fovea. Multiple eye-machine interaction applications, such as eye-drawing, Gluttonous Snake game, web interaction, pan-tilt-zoom camera control, and robot vehicle control, are demonstrated on the eye movement model and in vivo rabbit. Furthermore, comprehensive biocompatibility tests are implemented, demonstrating low cytotoxicity and low eye irritation. Thus, the contact lens is expected to enrich approaches of eye tracking techniques and promote the development of human-machine interaction technology.

[Design and support performance evaluation of medical multi-position auxiliary support exoskeleton mechanism].

Aiming at the status of muscle and joint damage caused on surgeons keeping surgical posture for a long time, this paper designs a medical multi-position auxiliary support exoskeleton with multi-joint mechanism by analyzing the surgical postures and conducting conformational studies on different joints respectively. Then by establishing a human-machine static model, this study obtains the joint torque and joint force before and after the human body wears the exoskeleton, and calibrates the strength of the exoskeleton with finite element analysis software. The results show that the maximum stress of the exoskeleton is less than the material strength requirements, the overall deformation is small, and the structural strength of the exoskeleton meets the use requirements. Finally, in this study, subjects were selected to participate in the plantar pressure test and biomechanical simulation with the man-machine static model, and the results were analyzed in terms of plantar pressure, joint torque and joint force, muscle force and overall muscle metabolism to assess the exoskeleton support performance. The results show that the exoskeleton has better support for the whole body and can reduce the musculoskeletal burden. The exoskeleton mechanism in this study better matches the actual working needs of surgeons and provides a new paradigm for the design of medical support exoskeleton mechanism.

Operator selection for human-automation teaming: The role of manual task skill in predicting automation failure intervention.

Humans working in modern work systems are increasingly required to supervise task automation. We examined whether manual aircraft conflict detection skill predicted participants' ability to respond to conflict detection automation failures in simulated air traffic control. In a conflict discrimination task (to assess manual skill), participants determined whether pairs of aircraft were in conflict or not by judging their relative-arrival time at common intersection points. Then in a simulated air traffic control task, participants supervised automation which either partially or fully detected and resolved conflicts on their behalf. Automation supervision required participants to detect when automation may have failed and effectively intervene. When automation failed, participants who had better manual conflict detection skill were faster and more accurate to intervene. However, a substantial proportion of variance in failure intervention was not explained by manual conflict detection skill, potentially reflecting that future research should consider other cognitive skills underlying automation supervision.

Imposing Motion Variability for Ergonomic Human-Robot Collaboration.

OCCUPATIONAL APPLICATIONS"Overassistive" robots can adversely impact long-term human-robot collaboration in the workplace, leading to risks of worker complacency, reduced workforce skill sets, and diminished situational awareness. Ergonomics practitioners should thus be cautious about solely targeting widely adopted metrics for improving human-robot collaboration, such as user trust and comfort. By contrast, introducing variability and adaptation into a collaborative robot's behavior could prove vital in preventing the negative consequences of overreliance and overtrust in an autonomous partner. This work reported here explored how instilling variability into physical human-robot collaboration can have a measurably positive effect on ergonomics in a repetitive task. A review of principles related to this notion of "stimulating" robot behavior is also provided to further inform ergonomics practitioners of existing human-robot collaboration frameworks.

Background: Collaborative robots, or cobots, are becoming ubiquitous in occupational settings due to benefits that include improved worker safety and increased productivity. Existing research on human-robot collaboration in industry has made progress in enhancing workers' psychophysical states, by optimizing measures of ergonomics risk factors, such as human posture, comfort, and cognitive workload. However, short-term objectives for robotic assistance may conflict with the worker's long-term preferences, needs, and overall wellbeing.Purpose: To investigate the ergonomic advantages and disadvantages of employing a collaborative robotics framework that intentionally imposes variability in the robot's behavior to stimulate the human partner's psychophysical state.Methods: A review of "overassistance" within human-robot collaboration and methods of addressing this phenomenon via adaptive automation. In adaptive approaches, the robot assistance may even challenge the user to better achieve a long-term objective while partially conflicting with their short-term task goals. Common themes across these approaches were extracted to motivate and support the proposed idea of stimulating robot behavior in physical human-robot collaboration.Results: Experimental evidence to justify stimulating robot behavior is presented through a human-robot handover study. A robot handover policy that regularly injects variability into the object transfer location led to significantly larger dynamics in the torso rotations and center of mass of human receivers compared to an "overassistive" policy that constrains receiver motion. Crucially, the stimulating handover policy also generated improvements in widely used ergonomics risk indicators of human posture.Conclusions: Our findings underscore the potential ergonomic benefits of a cobot's actions imposing variability in a user's responsive behavior, rather than indirectly restricting human behavior by optimizing the immediate task objective. Therefore, a transition from cobot policies that optimize instantaneous measures of ergonomics to those that continuously engage users could hold promise for human-robot collaboration in occupational settings characterized by repeated interactions.

The role of social support in human-automation interaction.

This theoretical article examines the concept of social support in the context of human-automation interaction, outlining several critical issues. We identified several factors that we expect to influence the consequences of social support and to what extent it is perceived as appropriate (e.g. provider possibilities, recipient expectations), notably regarding potential threats to self-esteem. We emphasise the importance of performance (including extra-role performance) as a potential outcome, whereas previous research has primarily concentrated on health and well-being. We discuss to what extent automation may provide different types of social support (e.g. emotional, instrumental), and how it differs from human support. Finally, we propose a taxonomy of automated support, arguing that source of support is not a binary concept. We conclude that more empirical work is needed to examine the multiple effects of social support for core performance indicators and extra-role performance and emphasise that there are ethical questions involved.

This theoretical article examines the role of automated social support given the increasing ability of automated systems. It concludes that it seems likely that automated systems may be perceived as supportive if they conform to pertinent criteria for design. However, empirical studies are needed to assess the impact of the complex interplay of humans and automation being involved together in the design and provision of social support.

Teleoperator-Robot-Human Interaction in Manufacturing: Perspectives from Industry, Robot Manufacturers, and Researchers.

OCCUPATIONAL APPLICATIONSIndustrial robots have become an important aspect in modern industry. In the context of human-robot collaboration, enabling teleoperated robots to work in close proximity to local/onsite humans can provide new opportunities to improve human engagement in a distributed workplace. Interviews with industry stakeholders highlighted several potential benefits of such teleoperator-robot-human collaboration (tRHC), including the application of tRHC to tasks requiring both expertise and manual dexterity (e.g., maintenance and highly skilled tasks in sectors including construction, manufacturing, and healthcare), as well as opportunities to expand job accessibility for individuals with disabilities and older individuals. However, interviewees also indicated potential challenges of tRHC, particularly related to human perception (e.g., perceiving remote environments), safety, and trust. Given these challenges, and the current limited information on the practical value and implementation of tRHC, we propose several future research directions, with a focus on human factors and ergonomics, to help realize the potential benefits of tRHC.

Background The increasing prevalence of robots in industrial environments is attributed in part to advancements in collaborative robot technologies, enabling robots to work in close proximity to humans. Simultaneously, the rise of teleoperation, involving remote robot control, poses unique opportunities and challenges for human-robot collaboration (HRC) in diverse and distributed workspaces.Purpose There is not yet a comprehensive understanding of HRC in teleoperation, specifically focusing on collaborations involving the teleoperator, the robot, and the local or onsite workers in industrial settings, here referred to as teleoperator-robot-human collaboration (tRHC). We aimed to identify opportunities, challenges, and potential applications of tRHC through insights provided from industry stakeholders, thereby supporting effective future industrial implementations.Methods Thirteen stakeholders in robotics, specializing in different domains (i.e., safety, robot manufacturing, aerospace/automotive manufacturing, and supply chains), completed semi-structured interviews that focused on exploring diverse aspects relevant to tRHC. The interviews were then transcribed and thematic analysis was applied to group responses into broader categories, which were further compared across stakeholder industries.Results We identified three main categories and 13 themes from the interviews. These categories include Benefits, Concerns, and Technical Challenges. Interviewees highlighted accessibility, ergonomics, flexibility, safety, time & cost saving, and trust as benefits of tRHC. Concerns raised encompassed safety, standards, trust, and workplace optimization. Technical challenges consisted of critical issues such as communication time delays, the need for high dexterity in robot manipulators, the importance of establishing shared situational awareness among all agents, and the potential of augmented and virtual reality in providing immersive control interfaces.Conclusions Despite important challenges, tRHC could offer unique benefits, facilitating seamless collaboration among the teleoperator, teleoperated robot(s), and onsite workers across physical and geographic boundaries. To realize such benefits and address the challenges, we propose several research directions to further explore and develop tRHC capabilities.

A Literature Review on Safety Perception and Trust during Human-Robot Interaction with Autonomous Mobile Robots That Apply to Industrial Environments.

Occupational ApplicationsAutonomous mobile robots are used in manufacturing and warehousing industries, to transport material across the facility and deliver parts to work cells. Human workers might encounter or interact with these robots in aisle ways or at their workstation. It is important to consider factors that impact worker safety and trust when implementing autonomous mobile robots in the workplace. This paper reviews prior research that aims to improve the safety of human-robot interaction with autonomous mobile robots and identifies needs for future research. Researchers used a variety of questionnaires and behavioral assessment methods to measure perceived safety. Factors such as robot appearance, approach speed, and approach direction, significantly affect perceived safety. Additionally, projection of signals on the floor, turn signals, and haptic communication devices, can improve the predictability and overall safety of robot navigation.

Introduction: Autonomous mobile robots are rapidly emerging in the workplace, which potentially creates new hazards for human workers that interact with them.Purpose: We aimed to systematically review previous research on human-robot interaction with autonomous mobile robots that apply to industrial environments, and to identify research needs to improve worker safety and trust.Methods: We completed a systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses methodology. We focused on articles that contained experiments with human participants and that included findings associated with improving safety and/or trust of workers who interact with mobile robots in industrial environments. We identified 50 articles that fit inclusion/exclusion criteria for the review.Results: Almost all of the reported experiments were conducted in a controlled laboratory setting. There were 27 different types of autonomous mobile robots. Only two studies involved industrial mobile robots that were commercially available and could be implemented in an industrial environment. Most studies used questionnaires, with the most common topic relating to participant perceptions of various robot traits, while few directly evaluated perceived safety and trust using questionnaires. Behavioral and physiological assessment methods were used in 70% and 8% of the studies, respectively. Separation distance between the participant and robot was the most common behavioral assessment method. A variety of robot characteristics were found to have a significant effect on human perception of safety and other similar concepts.Conclusions: Future research requires rigorous reporting of participant demographics and experience level with robots. We found that 34% and 44% of references failed to report the mean age of their participant sample and their experience with robots, respectively. Among several gaps that we identified in the literature were a lack of field experiments, sparse research involving multiple mobile robots, and limited use of industrial mobile robots in experiments with human participants.

Measuring cognitive workload in the nuclear control room: a review.

Despite the substantial literature and human factors guidance, evaluators report challenges in selecting cognitive workload measures for the evaluation of complex human-technology systems. A review of 32 articles found that self-report measures and secondary tasks were systematically sensitive to human-system interface conditions and correlated with physiological measures. Therefore, including a self-report measure of cognitive workload is recommended when evaluating human-system interfaces. Physiological measures were mainly used in method studies, and future research must demonstrate the utility of these measures for human-system evaluation in complex work settings. However, indexes of physiological measures showed promise for cognitive workload assessment. The review revealed a limited focus on the measurement of excessive cognitive workload, although this is a key topic in nuclear process control. To support human-system evaluation of adequate cognitive workload, future research on behavioural measures may be useful in the identification and analysis of underload and overload.

This review provides background for the selection of cognitive workload measures for the evaluation of complex human­technology systems and identifies future research needs for applied cognitive workload assessment.

A novel unsupervised dynamic feature domain adaptation strategy for cross-individual myoelectric gesture recognition.

Objective.Surface electromyography pattern recognition (sEMG-PR) is considered as a promising control method for human-machine interaction systems. However, the performance of a trained classifier would greatly degrade for novel users since sEMG signals are user-dependent and largely affected by a number of individual factors such as the quantity of subcutaneous fat and the skin impedance.Approach.To solve this issue, we proposed a novel unsupervised cross-individual motion recognition method that aligned sEMG features from different individuals by self-adaptive dimensional dynamic distribution adaptation (SD-DDA) in this study. In the method, both the distances of marginal and conditional distributions between source and target features were minimized through automatically selecting the optimal feature domain dimension by using a small amount of unlabeled target data.Main results.The effectiveness of the proposed method was tested on four different feature sets, and results showed that the average classification accuracy was improved by above 10% on our collected dataset with the best accuracy reached 90.4%. Compared to six kinds of classic transfer learning methods, the proposed method showed an outstanding performance with improvements of 3.2%-13.8%. Additionally, the proposed method achieved an approximate 9% improvement on a publicly available dataset.Significance.These results suggested that the proposed SD-DDA method is feasible for cross-individual motion intention recognition, which would provide help for the application of sEMG-PR based system.

Automation complacency on the road.

Given that automation complacency, a hitherto controversial concept, is already used to blame and punish human drivers in current accident investigations and courts, it is essential to map complacency research in driving automation and determine whether current research can support its legitimate usage in these practical fields. Here, we reviewed its status quo in the domain and conducted a thematic analysis. We then discussed five fundamental challenges that might undermine its scientific legitimation: conceptual confusion exists in whether it is an individual versus systems problem; uncertainties exist in current evidence of complacency; valid measures specific to complacency are lacking; short-term laboratory experiments cannot address the long-term nature of complacency and thus their findings may lack external validity; and no effective interventions directly target complacency prevention. The Human Factors/Ergonomics community has a responsibility to minimise its usage and defend human drivers who rely on automation that is far from perfect.Practitioner summary: Human drivers are accused of complacency and overreliance on driving automation in accident investigations and courts. Our review work shows that current academic research in the driving automation domain cannot support its legitimate usage in these practical fields. Its misuse will create a new form of consumer harms.