Communication Strategies in Human-Autonomy Teams During Technological Failures.

OBJECTIVE: This study examines low-, medium-, and high-performing Human-Autonomy Teams' (HATs') communication strategies during various technological failures that impact routine communication strategies to adapt to the task environment. BACKGROUND: Teams must adapt their communication strategies during dynamic tasks, where more successful teams make more substantial adaptations. Adaptations in communication strategies may explain how successful HATs overcome technological failures. Further, technological failures of variable severity may alter communication strategies of HATs at different performance levels in their attempts to overcome each failure. METHOD: HATs in a Remotely Piloted Aircraft System-Synthetic Task Environment (RPAS-STE), involving three team members, were tasked with photographing targets. Each triad had two randomly assigned participants in navigator and photographer roles, teaming with an experimenter who simulated an AI pilot in a Wizard of Oz paradigm. Teams encountered two different technological failures, automation and autonomy, where autonomy failures were more challenging to overcome. RESULTS: High-performing HATs calibrated their communication strategy to the complexity of the different failures better than medium- and low-performing teams. Further, HATs adjusted their communication strategies over time. Finally, only the most severe failures required teams to increase the efficiency of their communication. CONCLUSION: HAT effectiveness under degraded conditions depends on the type of communication strategies enacted by the team. Previous findings from studies of all-human teams apply here; however, novel results suggest information requests are particularly important to HAT success during failures. APPLICATION: Understanding the communication strategies of HATs under degraded conditions can inform training protocols to help HATs overcome failures.

Human factors considerations for the context-aware design of adaptive autonomous teammates.

Despite the gains in performance that AI can bring to human-AI teams, they also present them with new challenges, such as the decline in human ability to respond to AI failures as the AI becomes more autonomous. This challenge is particularly dangerous in human-AI teams, where the AI holds a unique role in the team's success. Thus, it is imperative that researchers find solutions for designing AI team-mates that consider their human team-mates' needs in their adaptation logic. This study explores adaptive autonomy as a solution to overcoming these challenges. We conducted twelve contextual inquiries with professionals in two teaming contexts in order to understand how human teammate perceptions can be used to determine optimal autonomy levels for AI team-mates. The results of this study will enable the human factors community to develop AI team-mates that can enhance their team's performance while avoiding the potentially devastating impacts of their failures.

As AI becomes more autonomous, the human ability to detect and respond to their failures decreases as they become less a part of the AI's decision-making loop. This contextual inquiry study shows how human factors are affected by and should influence the design of adaptive AI team-mates in different teaming contexts.

The Impact of Training on Human-Autonomy Team Communications and Trust Calibration.

OBJECTIVE: This work examines two human-autonomy team (HAT) training approaches that target communication and trust calibration to improve team effectiveness under degraded conditions. BACKGROUND: Human-autonomy teaming presents challenges to teamwork, some of which may be addressed through training. Factors vital to HAT performance include communication and calibrated trust. METHOD: Thirty teams of three, including one confederate acting as an autonomous agent, received either entrainment-based coordination training, trust calibration training, or control training before executing a series of missions operating a simulated remotely piloted aircraft. Automation and autonomy failures simulating degraded conditions were injected during missions, and measures of team communication, trust, and task efficiency were collected. RESULTS: Teams receiving coordination training had higher communication anticipation ratios, took photos of targets faster, and overcame more autonomy failures. Although autonomy failures were introduced in all conditions, teams receiving the calibration training reported that their overall trust in the agent was more robust over time. However, they did not perform better than the control condition. CONCLUSIONS: Training based on entrainment of communications, wherein introduction of timely information exchange through one team member has lasting effects throughout the team, was positively associated with improvements in HAT communications and performance under degraded conditions. Training that emphasized the shortcomings of the autonomous agent appeared to calibrate expectations and maintain trust. APPLICATIONS: Team training that includes an autonomous agent that models effective information exchange may positively impact team communication and coordination. Training that emphasizes the limitations of an autonomous agent may help calibrate trust.

Towards Ethical AI: Empirically Investigating Dimensions of AI Ethics, Trust Repair, and Performance in Human-AI Teaming.

OBJECTIVE: Determining the efficacy of two trust repair strategies (apology and denial) for trust violations of an ethical nature by an autonomous teammate. BACKGROUND: While ethics in human-AI interaction is extensively studied, little research has investigated how decisions with ethical implications impact trust and performance within human-AI teams and their subsequent repair. METHOD: Forty teams of two participants and one autonomous teammate completed three team missions within a synthetic task environment. The autonomous teammate made an ethical or unethical action during each mission, followed by an apology or denial. Measures of individual team trust, autonomous teammate trust, human teammate trust, perceived autonomous teammate ethicality, and team performance were taken. RESULTS: Teams with unethical autonomous teammates had significantly lower trust in the team and trust in the autonomous teammate. Unethical autonomous teammates were also perceived as substantially more unethical. Neither trust repair strategy effectively restored trust after an ethical violation, and autonomous teammate ethicality was not related to the team score, but unethical autonomous teammates did have shorter times. CONCLUSION: Ethical violations significantly harm trust in the overall team and autonomous teammate but do not negatively impact team score. However, current trust repair strategies like apologies and denials appear ineffective in restoring trust after this type of violation. APPLICATION: This research highlights the need to develop trust repair strategies specific to human-AI teams and trust violations of an ethical nature.

Teaming With a Synthetic Teammate: Insights into Human-Autonomy Teaming.

Objective Three different team configurations are compared with the goal of better understanding human-autonomy teaming (HAT). Background Although an extensive literature on human-automation interaction exists, much less is known about HAT in which humans and autonomous agents interact as coordinated units. Further research must be conducted to better understand how all-human teams compare to HAT. Methods In an unmanned aerial system (UAS) context, a comparison was made among three types of three-member teams: (1) synthetic teams in which the pilot role is assigned to a synthetic teammate, (2) control teams in which the pilot was an inexperienced human, and (3) experimenter teams in which an experimenter served as an experienced pilot. Ten of each type of team participated. Measures of team performance, target processing efficiency, team situation awareness, and team verbal behaviors were analyzed. Results Synthetic teams performed as well at the mission level as control (all human) teams but processed targets less efficiently. Experimenter teams performed better across all other measures compared to control and synthetic teams. Conclusion Though there is potential for a synthetic agent to function as a full-fledged teammate, further advances in autonomy are needed to improve team-level dynamics in HAT teams. Application This research contributes to our understanding of how to make autonomy a good team player.

Adapting to the human: A systematic review of a decade of human factors research on adaptive autonomy.

This systematic review provides an understanding of existing human factors research on adaptive autonomy, its design, its impacts, and its definition. We conducted a search on adaptive autonomy and additional relevant search terms in four databases, which produced an initial 245 articles. The application of inclusion and exclusion criteria produced a total of 60 articles for in-depth review. Through a collaborative coding process and analysis, we extracted triggers for and types of autonomy adaptations, as well as human factors dependent variables that have been studied in previous adaptive autonomy research. Based on this analysis, we present a definition of adaptive autonomy for use in human factors artificial intelligence research, as well as a comprehensive review of existing research contributions, notable research gaps, and the application of adaptive autonomy.

Introduction to the Emerging Cognitive Science of Distributed Human-Autonomy Teams.

Teams are a fundamental aspect of life-from sports to business, to defense, to science, to education. While the cognitive sciences tend to focus on information processing within individuals, others have argued that teams are also capable of demonstrating cognitive capacities similar to humans, such as skill acquisition and forgetting (cf., Cooke, Gorman, Myers, & Duran, 2013; Fiore et al., 2010). As artificially intelligent and autonomous systems improve in their ability to learn, reason, interact, and coordinate with human teammates combined with the observation that teams can express cognitive capacities typically seen in individuals, a cognitive science of teams is emerging. Consequently, new questions are being asked about teams regarding teamness, trust, the introduction and effects of autonomous systems on teams, and how best to measure team behavior and phenomena. In this topic, four facets of human-autonomy team cognition are introduced with leaders in the field providing in-depth articles associated with one or more of the facets: (1) defining teams; (2) how trust is established, maintained, and repaired when broken; (3) autonomous systems operating as teammates; and (4) metrics for evaluating team cognition across communication, coordination, and performance.

Collaborative augmented reality in higher education: A systematic review of effectiveness, outcomes, and challenges.

This article reports a systematic literature review that examined past research exploring the effectiveness of collaborative Augmented Reality (AR) enabled instruction, in higher education contexts. To be included, an article should consist of an experimental study investigating the use of collaborative AR for learning in higher education settings. An initial search was conducted on five databases that resulted in a total of 2537 articles, of which 20 were finalized for this review. The main findings suggest that AR-enabled collaborative learning benefits students' overall learning outcomes and provides a positive collaboration experience in higher education settings. Further research is needed to determine the interaction elements, collaboration mechanisms, and information representation through AR that would potentially enhance student learning outcomes. This article concludes by discussing the implications of these findings, identifying challenges and strategies for developing effective collaborative AR-enabled learning content.

Investigating Team Coordination in Baseball Using a Novel Joint Decision Making Paradigm.

A novel joint decision making paradigm for assessing team coordination was developed and tested using baseball infielders. Balls launched onto an infield at different trajectories were filmed using four video cameras that were each placed at one of the typical positions of the four infielders. Each participant viewed temporally occluded videos for one of the four positions and were asked to say either "ball" if they would attempt to field it or the name of the bag that they would cover. The evaluation of two experienced coaches was used to assign a group coordination score for each trajectory and group decision times were calculated. Thirty groups of 4 current college baseball players were: (i) teammates (players from same team/view from own position), (ii) non-teammates (players from different teams/view from own position), or (iii) scrambled teammates (players from same team/view not from own position). Teammates performed significantly better (i.e., faster and more coordinated decisions) than the other two groups, whereas scrambled teammates performed significantly better than non-teammates. These findings suggest that team coordination is achieved through both experience with one's teammates' responses to particular events (e.g., a ball hit up the middle) and one's own general action capabilities (e.g., running speed). The sensitivity of our joint decision making paradigm to group makeup provides support for its use as a method for studying team coordination.

Identification of the Emplacement of Improvised Explosive Devices by Experienced Mission Payload Operators.

Improvised Explosive Devices (IEDs) have become one of the deadliest threats to military personnel, resulting in over 50% of American combat casualties in Iraq and Afghanistan. Identification of IED emplacement is conducted by mission payload operators (MPOs). Yet, experienced MPOs are limited in number, making MPO training a critical intervention. In this article, we implement a Cognitive Engineering Based on Expert Skill methodology to better understand how experienced MPOs identify the emplacement of IEDs for the purposes of improving training. First, expert knowledge was elicited through interviews and questionnaires to identify the types of perceptual cues used and how these cues are cognitively processed. Results indicate that there are many different static and dynamic cues that interact with each other over time and space. Using data from the interviews and questionnaires, an empirically grounded framework is presented that explains the cognitive process of IED emplacement detection. Using the overall findings and the framework, IED emplacement training scenarios were developed and built into a simulation.

Team Cognition As a Means to Improve Care Delivery in Critically Ill Patients With Cancer After Hematopoietic Cell Transplantation.

Hematopoietic cell transplantation (HCT) is an important and complex treatment modality for a variety of hematologic malignancies and some solid tumors. Although outcomes of patients who have undergone HCT and require care in intensive care units (ICUs) have improved over time, mortality rates remain high and there are significant associated costs. Lack of a team-based approach to care, especially during critical illness, is detrimental to patient autonomy and satisfaction, and to team morale, ultimately leading to poor quality of care. In this manuscript, we describe the case of a patient who had undergone HCT and was in the ICU setting, where inconsistent team interaction among the various stakeholders delivering care resulted in a lack of shared goals and poor outcomes. Team cognition is cognitive processing at the team level through interactions among team members and is reflected in dynamic communication and coordination behaviors. Although the patient received multidisciplinary care as needed in a medically complicated case, a lack of team cognition and, particularly, inconsistent communication among the dynamic teams caring for the patient, led to mixed messages being delivered with high-cost implications for the health-care system and the family. This article highlights concepts and recommendations that begin a necessary in-depth assessment of implications for clinical care and initiate a research agenda that examines the effects of team cognition on HCT teams, and, more generally, critical care of the patient with cancer.