Communication Strategies in Human-Autonomy Teams During Technological Failures.

OBJECTIVE: This study examines low-, medium-, and high-performing Human-Autonomy Teams' (HATs') communication strategies during various technological failures that impact routine communication strategies to adapt to the task environment. BACKGROUND: Teams must adapt their communication strategies during dynamic tasks, where more successful teams make more substantial adaptations. Adaptations in communication strategies may explain how successful HATs overcome technological failures. Further, technological failures of variable severity may alter communication strategies of HATs at different performance levels in their attempts to overcome each failure. METHOD: HATs in a Remotely Piloted Aircraft System-Synthetic Task Environment (RPAS-STE), involving three team members, were tasked with photographing targets. Each triad had two randomly assigned participants in navigator and photographer roles, teaming with an experimenter who simulated an AI pilot in a Wizard of Oz paradigm. Teams encountered two different technological failures, automation and autonomy, where autonomy failures were more challenging to overcome. RESULTS: High-performing HATs calibrated their communication strategy to the complexity of the different failures better than medium- and low-performing teams. Further, HATs adjusted their communication strategies over time. Finally, only the most severe failures required teams to increase the efficiency of their communication. CONCLUSION: HAT effectiveness under degraded conditions depends on the type of communication strategies enacted by the team. Previous findings from studies of all-human teams apply here; however, novel results suggest information requests are particularly important to HAT success during failures. APPLICATION: Understanding the communication strategies of HATs under degraded conditions can inform training protocols to help HATs overcome failures.

The Impact of Training on Human-Autonomy Team Communications and Trust Calibration.

OBJECTIVE: This work examines two human-autonomy team (HAT) training approaches that target communication and trust calibration to improve team effectiveness under degraded conditions. BACKGROUND: Human-autonomy teaming presents challenges to teamwork, some of which may be addressed through training. Factors vital to HAT performance include communication and calibrated trust. METHOD: Thirty teams of three, including one confederate acting as an autonomous agent, received either entrainment-based coordination training, trust calibration training, or control training before executing a series of missions operating a simulated remotely piloted aircraft. Automation and autonomy failures simulating degraded conditions were injected during missions, and measures of team communication, trust, and task efficiency were collected. RESULTS: Teams receiving coordination training had higher communication anticipation ratios, took photos of targets faster, and overcame more autonomy failures. Although autonomy failures were introduced in all conditions, teams receiving the calibration training reported that their overall trust in the agent was more robust over time. However, they did not perform better than the control condition. CONCLUSIONS: Training based on entrainment of communications, wherein introduction of timely information exchange through one team member has lasting effects throughout the team, was positively associated with improvements in HAT communications and performance under degraded conditions. Training that emphasized the shortcomings of the autonomous agent appeared to calibrate expectations and maintain trust. APPLICATIONS: Team training that includes an autonomous agent that models effective information exchange may positively impact team communication and coordination. Training that emphasizes the limitations of an autonomous agent may help calibrate trust.

From Teams to Teamness: Future Directions in the Science of Team Cognition.

OBJECTIVE: We review the current state-of-the-art in team cognition research, but more importantly describe the limitations of existing theories, laboratory paradigms, and measures considering the increasing complexities of modern teams and the study of team cognition. BACKGROUND: Research on, and applications of, team cognition has led to theories, data, and measures over the last several decades. METHOD: This article is based on research questions generated in a spring 2022 seminar on team cognition at Arizona State University led by the first author. RESULTS: Future research directions are proposed for extending the conceptualization of teams and team cognition by examining dimensions of teamness; extending laboratory paradigms to attain more realistic teaming, including nonhuman teammates; and advancing measures of team cognition in a direction such that data can be collected unobtrusively, in real time, and automatically. CONCLUSION: The future of team cognition is one of the new discoveries, new research paradigms, and new measures. APPLICATION: Extending the concepts of teams and team cognition can also extend the potential applications of these concepts.

State of science: models and methods for understanding and enhancing teams and teamwork in complex sociotechnical systems.

This state of the science review brings together the disparate literature of effective strategies for enhancing and accelerating team performance. The review evaluates and synthesises models and proposes recommended avenues for future research. The two major models of the Input-Mediator-Output-Input (IMOI) framework and the Big Five dimensions of teamwork were reviewed and both will need significant development for application to future teams comprising non-human agents. Research suggests that a multi-method approach is appropriate for team measurements, such as the integration of methods from self-report, observer ratings, event-based measurement and automated recordings. Simulations are recommended as the most effective team-based training interventions. The impact of new technology and autonomous agents is discussed with respect to the changing nature of teamwork. In particular, whether existing teamwork models and measures are suitable to support the design, operation and evaluation of human-nonhuman teams of the future. Practitioner summary: This review recommends a multi-method approach to the measurement and evaluation of teamwork. Team models will need to be adapted to describe interaction with non-human agents, which is what the future is most likely to hold. The most effective team training interventions use simulation-based approaches.

Evaluating sociotechnical dynamics in a simulated remotely-piloted aircraft system: a layered dynamics approach.

As coordination mechanisms change and technology failures occur, a sociotechnical system must reorganise itself across human and technological layers to maintain effectiveness. We present a study examining reorganisation across communication, controls and vehicle layers of a remotely-piloted aircraft system (RPAS) using a layered dynamics approach. Team members (pilot; navigator; photographer) performed 5 simulated RPAS missions using different operator configurations, including all-human and human-autonomy teams. Reorganization (operationally defined using entropy) time series measured the changing system reorganisation profiles under different operator configurations and following autonomy failures. Correlations between these reorganisation profiles and team effectiveness scores describe the manner in which the system had to be coordinated to maintain effectiveness under these changing conditions. Four unplanned autonomy failures were analysed to visualise system reorganisation following a technology failure. With its objective and real-time modelling and measurement capabilities, layered dynamics complements existing systems thinking tools for understanding sociotechnical complexity and enhancing system effectiveness. Practitioner summary: A layered dynamics approach for understanding how a sociotechnical system dynamically reorganises itself is presented. The layered dynamics of RPAS were analysed under different operator configurations and following autonomy failures. Layered dynamics complements existing system-thinking tools for modelling sociotechnical system complexity and effectiveness. Abbreviation: RPAS: remotely-piloted aircraft system; HIS: human-systems integration; EAST: event analysis of systemic teamwork; H1: hypothesis 1; H2: hypothesis 2; H3: hypothesis 3; CERTT-STE: cognitive engineering research on team tasks--synthetic task environment; AVO: air vehicle operator; PLO: payload operator; DEMPC: data exploitation, mission planning, and communications; ACT-R: adaptive control of thought-rational; sec: seconds; ANOVA: analysis of variance.

Information-Pooling Bias in Collaborative Security Incident Correlation Analysis.

OBJECTIVE: Incident correlation is a vital step in the cybersecurity threat detection process. This article presents research on the effect of group-level information-pooling bias on collaborative incident correlation analysis in a synthetic task environment. BACKGROUND: Past research has shown that uneven information distribution biases people to share information that is known to most team members and prevents them from sharing any unique information available with them. The effect of such biases on security team collaborations are largely unknown. METHOD: Thirty 3-person teams performed two threat detection missions involving information sharing and correlating security incidents. Incidents were predistributed to each person in the team based on the hidden profile paradigm. Participant teams, randomly assigned to three experimental groups, used different collaboration aids during Mission 2. RESULTS: Communication analysis revealed that participant teams were 3 times more likely to discuss security incidents commonly known to the majority. Unaided team collaboration was inefficient in finding associations between security incidents uniquely available to each member of the team. Visualizations that augment perceptual processing and recognition memory were found to mitigate the bias. CONCLUSION: The data suggest that (a) security analyst teams, when conducting collaborative correlation analysis, could be inefficient in pooling unique information from their peers; (b) employing off-the-shelf collaboration tools in cybersecurity defense environments is inadequate; and (c) collaborative security visualization tools developed considering the human cognitive limitations of security analysts is necessary. APPLICATION: Potential applications of this research include development of team training procedures and collaboration tool development for security analysts.

Teaming With a Synthetic Teammate: Insights into Human-Autonomy Teaming.

Objective Three different team configurations are compared with the goal of better understanding human-autonomy teaming (HAT). Background Although an extensive literature on human-automation interaction exists, much less is known about HAT in which humans and autonomous agents interact as coordinated units. Further research must be conducted to better understand how all-human teams compare to HAT. Methods In an unmanned aerial system (UAS) context, a comparison was made among three types of three-member teams: (1) synthetic teams in which the pilot role is assigned to a synthetic teammate, (2) control teams in which the pilot was an inexperienced human, and (3) experimenter teams in which an experimenter served as an experienced pilot. Ten of each type of team participated. Measures of team performance, target processing efficiency, team situation awareness, and team verbal behaviors were analyzed. Results Synthetic teams performed as well at the mission level as control (all human) teams but processed targets less efficiently. Experimenter teams performed better across all other measures compared to control and synthetic teams. Conclusion Though there is potential for a synthetic agent to function as a full-fledged teammate, further advances in autonomy are needed to improve team-level dynamics in HAT teams. Application This research contributes to our understanding of how to make autonomy a good team player.

Defining the methodological challenges and opportunities for an effective science of sociotechnical systems and safety.

An important part of the application of sociotechnical systems theory (STS) is the development of methods, tools and techniques to assess human factors and ergonomics workplace requirements. We focus in this paper on describing and evaluating current STS methods for workplace safety, as well as outlining a set of six case studies covering the application of these methods to a range of safety contexts. We also describe an evaluation of the methods in terms of ratings of their ability to address a set of theoretical and practical questions (e.g. the degree to which methods capture static/dynamic aspects of tasks and interactions between system levels). The outcomes from the evaluation highlight a set of gaps relating to the coverage and applicability of current methods for STS and safety (e.g. coverage of external influences on system functioning; method usability). The final sections of the paper describe a set of future challenges, as well as some practical suggestions for tackling these. PRACTITIONER SUMMARY: We provide an up-to-date review of STS methods, a set of case studies illustrating their use and an evaluation of their strengths and weaknesses. The paper concludes with a 'roadmap' for future work.

Introduction to the Emerging Cognitive Science of Distributed Human-Autonomy Teams.

Teams are a fundamental aspect of life-from sports to business, to defense, to science, to education. While the cognitive sciences tend to focus on information processing within individuals, others have argued that teams are also capable of demonstrating cognitive capacities similar to humans, such as skill acquisition and forgetting (cf., Cooke, Gorman, Myers, & Duran, 2013; Fiore et al., 2010). As artificially intelligent and autonomous systems improve in their ability to learn, reason, interact, and coordinate with human teammates combined with the observation that teams can express cognitive capacities typically seen in individuals, a cognitive science of teams is emerging. Consequently, new questions are being asked about teams regarding teamness, trust, the introduction and effects of autonomous systems on teams, and how best to measure team behavior and phenomena. In this topic, four facets of human-autonomy team cognition are introduced with leaders in the field providing in-depth articles associated with one or more of the facets: (1) defining teams; (2) how trust is established, maintained, and repaired when broken; (3) autonomous systems operating as teammates; and (4) metrics for evaluating team cognition across communication, coordination, and performance.

Establishing Human Observer Criterion in Evaluating Artificial Social Intelligence Agents in a Search and Rescue Task.

Artificial social intelligence (ASI) agents have great potential to aid the success of individuals, human-human teams, and human-artificial intelligence teams. To develop helpful ASI agents, we created an urban search and rescue task environment in Minecraft to evaluate ASI agents' ability to infer participants' knowledge training conditions and predict participants' next victim type to be rescued. We evaluated ASI agents' capabilities in three ways: (a) comparison to ground truth-the actual knowledge training condition and participant actions; (b) comparison among different ASI agents; and (c) comparison to a human observer criterion, whose accuracy served as a reference point. The human observers and the ASI agents used video data and timestamped event messages from the testbed, respectively, to make inferences about the same participants and topic (knowledge training condition) and the same instances of participant actions (rescue of victims). Overall, ASI agents performed better than human observers in inferring knowledge training conditions and predicting actions. Refining the human criterion can guide the design and evaluation of ASI agents for complex task environments and team composition.

Dynamical analysis in real time: detecting perturbations to team communication.

Dynamical systems methods characterise patterns of change over time. Typically, such methods are applied only after data collection is complete. However, brief disturbances - perturbations - can occur as a process unfolds and can result in undesirable outcomes if not acted on. The application of dynamics in real time would be useful for detecting these sudden changes. Real-time analysis was accomplished by updating dynamical estimates simultaneously across different window sizes. We calculated the largest Lyapunov exponent, a measure of dynamical stability, to detect a perturbation to team communication in a simulated uninhabited air vehicle (UAV) reconnaissance mission. The perturbation consisted of information demands from a confederate that occurred unexpectedly during performance of a UAV mission. We demonstrate the use of real-time methods in detecting that perturbation as it occurred. In application, this technique would have enabled real-time intervention. Extensions of the real-time dynamical method to other domains of psychological inquiry are discussed. PRACTITIONER SUMMARY: A real-time dynamical analysis method that was developed to detect unexpected perturbations in team communication is described. The use of the method is demonstrated on perturbed communication from a three-person uninhabited air vehicle command-and-control team. The generalisability of the method is considered with respect to physiological and motor coordination dynamics.

Team coordination dynamics.

Team coordination consists of both the dynamics of team member interaction and the environmental dynamics to which a team is subjected. Focusing on dynamics, an approach is developed that contrasts with traditional aggregate-static concepts of team coordination as characterized by the shared mental model approach. A team coordination order parameter was developed to capture momentary fluctuations in coordination. Team coordination was observed in three-person uninhabited air vehicle teams across two experimental sessions. The dynamics of the order parameter were observed under changes of a team familiarity control parameter. Team members returned for the second session to either the same (Intact) or different (Mixed) team. 'Roadblock' perturbations, or novel changes in the task environment, were introduced in order to probe the stability of team coordination. Nonlinear dynamic methods revealed differences that a traditional approach did not: Intact and Mixed team coordination dynamics looked very different; Mixed teams were more stable than Intact teams and explored the space of solutions without the need for correction. Stability was positively correlated with the number of roadblock perturbations that were overcome successfully. The novel and non-intuitive contribution of a dynamical analysis was that Mixed teams, who did not have a long history working together, were more adaptive. Team coordination dynamics carries new implications for traditional problems such as training adaptive teams.

Bridging psychology and engineering to make technology work for people.

Engineering grand challenges and big ideas not only demand innovative engineering solutions, but also typically involve and affect human thought, behavior, and quality of life. To solve these types of complex problems, multidisciplinary teams must bring together experts in engineering and psychological science, yet fusing these distinct areas can be difficult. This article describes how Human Systems Engineering (HSE) researchers have confronted such challenges at the interface of humans and technological systems. Two narrative cases are reported-computer game-based cognitive assessments and medical device reprocessing-and lessons learned are shared. The article then discusses 2 strategies currently being explored to enact such lessons and enhance these kinds of multidisciplinary engineering teams: a "top-down" administrative approach that supports team formation and productivity through a university research center, and a "bottom-up" engineering education approach that prepares students to work at the intersection of psychology and engineering. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Team cognition in experienced command-and-control teams.

Team cognition in experienced command-and-control teams is examined in an UAV (Uninhabited Aerial Vehicle) simulation. Five 3-person teams with experience working together in a command-and-control setting were compared to 10 inexperienced teams. Each team participated in five 40-min missions of a simulation in which interdependent team members control a UAV to take reconnaissance photos. Experienced teams exceeded performance of inexperienced teams, suggesting transfer of previous command-and-control experience. Compared to inexperienced teams, experienced teams had fewer errors on process-related training knowledge, superior team process ratings, and communications containing fewer coordination-related utterances. These findings support the view that team cognition emerges through the interactions of team members, that interactions distinguish high-performing teams from average teams, and that these interactions transfer across different tasks.

Investigating Team Coordination in Baseball Using a Novel Joint Decision Making Paradigm.

A novel joint decision making paradigm for assessing team coordination was developed and tested using baseball infielders. Balls launched onto an infield at different trajectories were filmed using four video cameras that were each placed at one of the typical positions of the four infielders. Each participant viewed temporally occluded videos for one of the four positions and were asked to say either "ball" if they would attempt to field it or the name of the bag that they would cover. The evaluation of two experienced coaches was used to assign a group coordination score for each trajectory and group decision times were calculated. Thirty groups of 4 current college baseball players were: (i) teammates (players from same team/view from own position), (ii) non-teammates (players from different teams/view from own position), or (iii) scrambled teammates (players from same team/view not from own position). Teammates performed significantly better (i.e., faster and more coordinated decisions) than the other two groups, whereas scrambled teammates performed significantly better than non-teammates. These findings suggest that team coordination is achieved through both experience with one's teammates' responses to particular events (e.g., a ball hit up the middle) and one's own general action capabilities (e.g., running speed). The sensitivity of our joint decision making paradigm to group makeup provides support for its use as a method for studying team coordination.

Identification of the Emplacement of Improvised Explosive Devices by Experienced Mission Payload Operators.

Improvised Explosive Devices (IEDs) have become one of the deadliest threats to military personnel, resulting in over 50% of American combat casualties in Iraq and Afghanistan. Identification of IED emplacement is conducted by mission payload operators (MPOs). Yet, experienced MPOs are limited in number, making MPO training a critical intervention. In this article, we implement a Cognitive Engineering Based on Expert Skill methodology to better understand how experienced MPOs identify the emplacement of IEDs for the purposes of improving training. First, expert knowledge was elicited through interviews and questionnaires to identify the types of perceptual cues used and how these cues are cognitively processed. Results indicate that there are many different static and dynamic cues that interact with each other over time and space. Using data from the interviews and questionnaires, an empirically grounded framework is presented that explains the cognitive process of IED emplacement detection. Using the overall findings and the framework, IED emplacement training scenarios were developed and built into a simulation.

Team Cognition As a Means to Improve Care Delivery in Critically Ill Patients With Cancer After Hematopoietic Cell Transplantation.

Hematopoietic cell transplantation (HCT) is an important and complex treatment modality for a variety of hematologic malignancies and some solid tumors. Although outcomes of patients who have undergone HCT and require care in intensive care units (ICUs) have improved over time, mortality rates remain high and there are significant associated costs. Lack of a team-based approach to care, especially during critical illness, is detrimental to patient autonomy and satisfaction, and to team morale, ultimately leading to poor quality of care. In this manuscript, we describe the case of a patient who had undergone HCT and was in the ICU setting, where inconsistent team interaction among the various stakeholders delivering care resulted in a lack of shared goals and poor outcomes. Team cognition is cognitive processing at the team level through interactions among team members and is reflected in dynamic communication and coordination behaviors. Although the patient received multidisciplinary care as needed in a medically complicated case, a lack of team cognition and, particularly, inconsistent communication among the dynamic teams caring for the patient, led to mixed messages being delivered with high-cost implications for the health-care system and the family. This article highlights concepts and recommendations that begin a necessary in-depth assessment of implications for clinical care and initiate a research agenda that examines the effects of team cognition on HCT teams, and, more generally, critical care of the patient with cancer.

Interactive team cognition.

Cognition in work teams has been predominantly understood and explained in terms of shared cognition with a focus on the similarity of static knowledge structures across individual team members. Inspired by the current zeitgeist in cognitive science, as well as by empirical data and pragmatic concerns, we offer an alternative theory of team cognition. Interactive Team Cognition (ITC) theory posits that (1) team cognition is an activity, not a property or a product; (2) team cognition should be measured and studied at the team level; and (3) team cognition is inextricably tied to context. There are implications of ITC for theory building, modeling, measurement, and applications that make teams more effective performers.

Measuring patterns in team interaction sequences using a discrete recurrence approach.

OBJECTIVE: Recurrence-based measures of communication determinism and pattern information are described and validated using previously collected team interaction data. BACKGROUND: Team coordination dynamics has revealed that"mixing" team membership can lead to flexible interaction processes, but keeping a team "intact" can lead to rigid interaction processes. We hypothesized that communication of intact teams would have greater determinism and higher pattern information compared to that of mixed teams. METHOD: Determinism and pattern information were measured from three-person Uninhabited Air Vehicle team communication sequences over a series of 40-minute missions. Because team members communicated using push-to-talk buttons, communication sequences were automatically generated during each mission. RESULTS: The Composition x Mission determinism effect was significant. Intact teams' determinism increased over missions, whereas mixed teams' determinism did not change. Intact teams had significantly higher maximum pattern information than mixed teams. CONCLUSION: Results from these new communication analysis methods converge with content-based methods and support our hypotheses. APPLICATION: Because they are not content based, and because they are automatic and fast, these new methods may be amenable to real-time communication pattern analysis.