Modelling human navigation and decision dynamics in a first-person herding task.

This study investigated whether dynamical perceptual-motor primitives (DPMPs) could also be used to capture human navigation in a first-person herding task. To achieve this aim, human participants played a first-person herding game, in which they were required to corral virtual cows, called targets, into a specified containment zone. In addition to recording and modelling participants' movement trajectories during gameplay, participants' target-selection decisions (i.e. the order in which participants corralled targets) were recorded and modelled. The results revealed that a simple DPMP navigation model could effectively reproduce the movement trajectories of participants and that almost 80% of the participants' target-selection decisions could be captured by a simple heuristic policy. Importantly, when this policy was coupled to the DPMP navigation model, the resulting system could successfully simulate and predict the behavioural dynamics (movement trajectories and target-selection decisions) of participants in novel multi-target contexts. Implications of the findings for understanding complex human perceptual-motor behaviour and the development of artificial agents for robust human-machine interaction are discussed.

Autistic young people adaptively use gaze to facilitate joint attention during multi-gestural dyadic interactions.

LAY ABSTRACT: Autistic people have been said to have 'problems' with joint attention, that is, looking where someone else is looking. Past studies of joint attention have used tasks that require autistic people to continuously look at and respond to eye-gaze cues. But joint attention can also be done using other social cues, like pointing. This study looked at whether autistic and non-autistic young people use another person's eye gaze during joint attention in a task that did not require them to look at their partner's face. In the task, each participant worked together with their partner to find a computer-generated object in virtual reality. Sometimes the participant had to help guide their partner to the object, and other times, they followed their partner's lead. Participants were told to point to guide one another but were not told to use eye gaze. Both autistic and non-autistic participants often looked at their partner's face during joint attention interactions and were faster to respond to their partner's hand-pointing when the partner also looked at the object before pointing. This shows that autistic people can and do use information from another person's eyes, even when they don't have to. It is possible that, by not forcing autistic young people to look at their partner's face and eyes, they were better able to gather information from their partner's face when needed, without being overwhelmed. This shows how important it is to design tasks that provide autistic people with opportunities to show what they can do.

Behavioral dynamics of conversation, (mis)communication and coordination in noisy environments.

During conversations people coordinate simultaneous channels of verbal and nonverbal information to hear and be heard. But the presence of background noise levels such as those found in cafes and restaurants can be a barrier to conversational success. Here, we used speech and motion-tracking to reveal the reciprocal processes people use to communicate in noisy environments. Conversations between twenty-two pairs of typical-hearing adults were elicited under different conditions of background noise, while standing or sitting around a table. With the onset of background noise, pairs rapidly adjusted their interpersonal distance and speech level, with the degree of initial change dependent on noise level and talker configuration. Following this transient phase, pairs settled into a sustaining phase in which reciprocal speech and movement-based coordination processes synergistically maintained effective communication, again with the magnitude of stability of these coordination processes covarying with noise level and talker configuration. Finally, as communication breakdowns increased at high noise levels, pairs exhibited resetting behaviors to help restore communication-decreasing interpersonal distance and/or increasing speech levels in response to communication breakdowns. Approximately 78 dB SPL defined a threshold where behavioral processes were no longer sufficient for maintaining effective conversation and communication breakdowns rapidly increased.

Initial phase and frequency modulations of pumping a playground swing.

The playground swing is a dynamic, coupled oscillator system consisting of the swing as an object and a human as the swinger. Here, we propose a model for capturing the effect of the initial phase of natural upper body motion on the continuous pumping of a swing and validate this model from the motion data of ten participants pumping swings of three different swing chain lengths. Our model predicts that the swing pumps the most if the phase of maximum lean back, which we call the initial phase, occurs when the swing is at a vertical (midpoint) position and moving forward when the amplitude is small. As the amplitude grows, the optimal initial phase gradually shifts towards an earlier phase of the cycle, the back extreme of the swing's trajectory. As predicted by our model, all participants shifted the initial phase of their upper body movements earlier as swing amplitude increased. This indicated that swingers adjust both the frequency and initial phase of their upper body movements to successfully pump a playground swing.

Dynamical biomarkers in teams and other multiagent systems.

Effective team behavior in high-performance environments such as in sport and the military requires individual team members to efficiently perceive the unfolding task events, predict the actions and action intents of the other team members, and plan and execute their own actions to simultaneously accomplish individual and collective goals. To enhance team performance through effective cooperation, it is crucial to measure the situation awareness and dynamics of each team member and how they collectively impact the team's functioning. Further, to be practically useful for real-life settings, such measures must be easily obtainable from existing sensors. This paper presents several methodologies that can be used on positional and movement acceleration data of team members to quantify and/or predict team performance, assess situation awareness, and to help identify task-relevant information to support individual decision-making. Given the limited reporting of these methods within military cohorts, these methodologies are described using examples from team sports and teams training in virtual environments, with discussion as to how they can be applied to real-world military teams.

Predicting and understanding human action decisions during skillful joint-action using supervised machine learning and explainable-AI.

This study investigated the utility of supervised machine learning (SML) and explainable artificial intelligence (AI) techniques for modeling and understanding human decision-making during multiagent task performance. Long short-term memory (LSTM) networks were trained to predict the target selection decisions of expert and novice players completing a multiagent herding task. The results revealed that the trained LSTM models could not only accurately predict the target selection decisions of expert and novice players but that these predictions could be made at timescales that preceded a player's conscious intent. Importantly, the models were also expertise specific, in that models trained to predict the target selection decisions of experts could not accurately predict the target selection decisions of novices (and vice versa). To understand what differentiated expert and novice target selection decisions, we employed the explainable-AI technique, SHapley Additive explanation (SHAP), to identify what informational features (variables) most influenced modelpredictions. The SHAP analysis revealed that experts were more reliant on information about target direction of heading and the location of coherders (i.e., other players) compared to novices. The implications and assumptions underlying the use of SML and explainable-AI techniques for investigating and understanding human decision-making are discussed.

Conversation dynamics in a multiplayer video game with knowledge asymmetry.

Despite the challenges associated with virtually mediated communication, remote collaboration is a defining characteristic of online multiplayer gaming communities. Inspired by the teamwork exhibited by players in first-person shooter games, this study investigated the verbal and behavioral coordination of four-player teams playing a cooperative online video game. The game, Desert Herding, involved teams consisting of three ground players and one drone operator tasked to locate, corral, and contain evasive robot agents scattered across a large desert environment. Ground players could move throughout the environment, while the drone operator's role was akin to that of a "spectator" with a bird's-eye view, with access to veridical information of the locations of teammates and the to-be-corralled agents. Categorical recurrence quantification analysis (catRQA) was used to measure the communication dynamics of teams as they completed the task. Demands on coordination were manipulated by varying the ground players' ability to observe the environment with the use of game "fog." Results show that catRQA was sensitive to changes to task visibility, with reductions in task visibility reorganizing how participants conversed during the game to maintain team situation awareness. The results are discussed in the context of future work that can address how team coordination can be augmented with the inclusion of artificial agents, as synthetic teammates.

Assessing Team Effectiveness by How Players Structure Their Search in a First-Person Multiplayer Video Game.

People working as a team can achieve more than when working alone due to a team's ability to parallelize the completion of tasks. In collaborative search tasks, this necessitates the formation of effective division of labor strategies to minimize redundancies in search. For such strategies to be developed, team members need to perceive the task's relevant components and how they evolve over time, as well as an understanding of what others will do so that they can structure their own behavior to contribute to the team's goal. This study explored whether the capacity for team members to coordinate effectively can be related to how participants structure their search behaviors in an online multiplayer collaborative search task. Our results demonstrated that the structure of search behavior, quantified using detrended fluctuation analysis, was sensitive to contextual factors that limit a participant's ability to gather information. Further, increases in the persistence of movement fluctuations during search behavior were found as teams developed more effective coordinative strategies and were associated with better task performance.

Validation of deep learning-based markerless 3D pose estimation.

Deep learning-based approaches to markerless 3D pose estimation are being adopted by researchers in psychology and neuroscience at an unprecedented rate. Yet many of these tools remain unvalidated. Here, we report on the validation of one increasingly popular tool (DeepLabCut) against simultaneous measurements obtained from a reference measurement system (Fastrak) with well-known performance characteristics. Our results confirm close (mm range) agreement between the two, indicating that under specific circumstances deep learning-based approaches can match more traditional motion tracking methods. Although more work needs to be done to determine their specific performance characteristics and limitations, this study should help build confidence within the research community using these new tools.

Task dynamics define the contextual emergence of human corralling behaviors.

Social animals have the remarkable ability to organize into collectives to achieve goals unobtainable to individual members. Equally striking is the observation that despite differences in perceptual-motor capabilities, different animals often exhibit qualitatively similar collective states of organization and coordination. Such qualitative similarities can be seen in corralling behaviors involving the encirclement of prey that are observed, for example, during collaborative hunting amongst several apex predator species living in disparate environments. Similar encirclement behaviors are also displayed by human participants in a collaborative problem-solving task involving the herding and containment of evasive artificial agents. Inspired by the functional similarities in this behavior across humans and non-human systems, this paper investigated whether the containment strategies displayed by humans emerge as a function of the task's underlying dynamics, which shape patterns of goal-directed corralling more generally. This hypothesis was tested by comparing the strategies naïve human dyads adopt during the containment of a set of evasive artificial agents across two disparate task contexts. Despite the different movement types (manual manipulation or locomotion) required in the different task contexts, the behaviors that humans display can be predicted as emergent properties of the same underlying task-dynamic model.

Navigational Behavior of Humans and Deep Reinforcement Learning Agents.

Rapid advances in the field of Deep Reinforcement Learning (DRL) over the past several years have led to artificial agents (AAs) capable of producing behavior that meets or exceeds human-level performance in a wide variety of tasks. However, research on DRL frequently lacks adequate discussion of the low-level dynamics of the behavior itself and instead focuses on meta-level or global-level performance metrics. In doing so, the current literature lacks perspective on the qualitative nature of AA behavior, leaving questions regarding the spatiotemporal patterning of their behavior largely unanswered. The current study explored the degree to which the navigation and route selection trajectories of DRL agents (i.e., AAs trained using DRL) through simple obstacle ridden virtual environments were equivalent (and/or different) from those produced by human agents. The second and related aim was to determine whether a task-dynamical model of human route navigation could not only be used to capture both human and DRL navigational behavior, but also to help identify whether any observed differences in the navigational trajectories of humans and DRL agents were a function of differences in the dynamical environmental couplings.

Gaze facilitates responsivity during hand coordinated joint attention.

The coordination of attention between individuals is a fundamental part of everyday human social interaction. Previous work has focused on the role of gaze information for guiding responses during joint attention episodes. However, in many contexts, hand gestures such as pointing provide another valuable source of information about the locus of attention. The current study developed a novel virtual reality paradigm to investigate the extent to which initiator gaze information is used by responders to guide joint attention responses in the presence of more visually salient and spatially precise pointing gestures. Dyads were instructed to use pointing gestures to complete a cooperative joint attention task in a virtual environment. Eye and hand tracking enabled real-time interaction and provided objective measures of gaze and pointing behaviours. Initiators displayed gaze behaviours that were spatially congruent with the subsequent pointing gestures. Responders overtly attended to the initiator's gaze during the joint attention episode. However, both these initiator and responder behaviours were highly variable across individuals. Critically, when responders did overtly attend to their partner's face, their saccadic reaction times were faster when the initiator's gaze was also congruent with the pointing gesture, and thus predictive of the joint attention location. These results indicate that humans attend to and process gaze information to facilitate joint attention responsivity, even in contexts where gaze information is implicit to the task and joint attention is explicitly cued by more spatially precise and visually salient pointing gestures.

Conversational distance adaptation in noise and its effect on signal-to-noise ratio in realistic listening environments.

Everyday environments impose acoustical conditions on speech communication that require interlocutors to adapt their behavior to be able to hear and to be heard. Past research has focused mainly on the adaptation of speech level, while few studies investigated how interlocutors adapt their conversational distance as a function of noise level. Similarly, no study tested the interaction between distance and speech level adaptation in noise. In the present study, participant pairs held natural conversations while binaurally listening to identical noise recordings of different realistic environments (range of 53-92 dB sound pressure level), using acoustically transparent headphones. Conversations were in standing or sitting (at a table) conditions. Interlocutor distances were tracked using wireless motion-capture equipment, which allowed subjects to move closer or farther from each other. The results show that talkers adapt their voices mainly according to the noise conditions and much less according to distance. Distance adaptation was highest in the standing condition. Consequently, mainly in the loudest environments, listeners were able to improve the signal-to-noise ratio (SNR) at the receiver location in the standing condition compared to the sitting condition, which became less negative. Analytical approximations are provided for the conversational distance as well as the receiver-related speech and SNR.

Synchronous vs. non-synchronous imitation: Using dance to explore interpersonal coordination during observational learning.

Observational learning can enhance the acquisition and performance quality of complex motor skills. While an extensive body of research has focused on the benefits of synchronous (i.e., concurrent physical practice) and non-synchronous (i.e., delayed physical practice) observational learning strategies, the question remains as to whether these approaches differentially influence performance outcomes. Accordingly, we investigate the differential outcomes of synchronous and non-synchronous observational training contexts using a novel dance sequence. Using multidimensional cross-recurrence quantification analysis, movement time-series were recorded for novice dancers who either synchronised with (n = 22) or observed and then imitated (n = 20) an expert dancer. Participants performed a 16-count choreographed dance sequence for 20 trials assisted by the expert, followed by one final, unassisted performance trial. Although end-state performance did not significantly differ between synchronous and non-synchronous learners, a significant decline in performance quality from imitation to independent replication was shown for synchronous learners. A non-significant positive trend in performance accuracy was shown for non-synchronous learners. For all participants, better imitative performance across training trials led to better end-state performance, but only for the accuracy (and not timing) of movement reproduction. Collectively, the results suggest that synchronous learners came to rely on a real-time mapping process between visual input from the expert and their own visual and proprioceptive intrinsic feedback, to the detriment of learning. Thus, the act of synchronising alone does not ensure an appropriate training context for advanced sequence learning.

Assessing abstract thought and its relation to language with a new nonverbal paradigm: Evidence from aphasia.

In recent years, language has been shown to play a number of important cognitive roles over and above the communication of thoughts. One hypothesis gaining support is that language facilitates thought about abstract categories, such as democracy or prediction. To test this proposal, a novel set of semantic memory task trials, designed for assessing abstract thought non-linguistically, were normed for levels of abstractness. The trials were rated as more or less abstract to the degree that answering them required the participant to abstract away from both perceptual features and common setting associations corresponding to the target image. The normed materials were then used with a population of people with aphasia to assess the relationship of abstract thought to language. While the language-impaired group with aphasia showed lower overall accuracy and longer response times than controls in general, of special note is that their response times were significantly longer as a function of a trial's degree of abstractness. Further, the aphasia group's response times in reporting their degree of confidence (a separate, metacognitive measure) were negatively correlated with their language production abilities, with lower language scores predicting longer metacognitive response times. These results provide some support for the hypothesis that language is an important aid to abstract thought and to metacognition about abstract thought.

Hopf Bifurcations in Complex Multiagent Activity: The Signature of Discrete to Rhythmic Behavioral Transitions.

Most human actions are composed of two fundamental movement types, discrete and rhythmic movements. These movement types, or primitives, are analogous to the two elemental behaviors of nonlinear dynamical systems, namely, fixed-point and limit cycle behavior, respectively. Furthermore, there is now a growing body of research demonstrating how various human actions and behaviors can be effectively modeled and understood using a small set of low-dimensional, fixed-point and limit cycle dynamical systems (differential equations). Here, we provide an overview of these dynamical motorprimitives and detail recent research demonstrating how these dynamical primitives can be used to model the task dynamics of complex multiagent behavior. More specifically, we review how a task-dynamic model of multiagent shepherding behavior, composed of rudimentary fixed-point and limit cycle dynamical primitives, can not only effectively model the behavior of cooperating human co-actors, but also reveals how the discovery and intentional use of optimal behavioral coordination during task learning is marked by a spontaneous, self-organized transition between fixed-point and limit cycle dynamics (i.e., via a Hopf bifurcation).

Co-actors Exhibit Similarity in Their Structure of Behavioural Variation That Remains Stable Across Range of Naturalistic Activities.

Human behaviour, along with any natural/biological behaviour, has varying degrees of intrinsic 'noise' or variability. Many studies have shown that the structure or patterning of this variability is sensitive to changes in task and constraint. Furthermore, two or more humans interacting together often begin to exhibit similar structures of behavioural variability (i.e., the patterning of their behavioural fluctuations becomes aligned or matched) independent of any moment-to-moment synchronization (termed complexity matching). However, much of the previous work has focused on a subset of simple or contrived behaviours within the context of highly controlled laboratory tasks. In the current study, individuals and pairs performed five self-paced (unsupervised), semi-structured activities around a university campus. Empatica E4 wristbands and iPhones were used to record the participants' behavioural activity via accelerometers and GPS. The results revealed that the structure of variability in naturalistic human behaviour co-varies with the task-goal constraints, and that the patterning of the behavioural fluctuations exhibited by co-acting individuals does become aligned during the performance of everyday activities. The results also revealed that the degree of complexity matching that occurred between pairs remained invariant across activity type, indicating that this measure could be employed as a robust, task-independent index of interpersonal behaviour.

Affect variability and predictability: Using recurrence quantification analysis to better understand how the dynamics of affect relate to health.

Changes in affect over time have been associated with health outcomes. However, previously utilized measurement methods focus on variability of affect (e.g., standard deviation, root mean squared successive difference) and ignore the more complex temporal patterns of affect over time. These patterns may be an important feature in understanding how the dynamics of affect relate to health. Recurrence quantification analysis (RQA) may help alleviate this problem by assessing temporal characteristics unassessed by past methods. RQA metrics, such as determinism and recurrence, can provide a measure of the predictability of affect over time, indexing how often patterns within affective experiences repeat. In Study 1, we first contrasted RQA metrics with commonly used measures of variability to demonstrate that RQA can further differentiate among patterns of affect. In Study 2, we analyzed the associations between these new metrics and health, namely, depressive and somatic symptoms. We found that RQA metrics predicted health above and beyond mean levels and variability of affect over time. The most desirable health outcomes were observed in people who had high mean positive affect, low mean negative affect, low affect variability, and high affect predictability. These studies are the first to demonstrate the utility of RQA for determining how temporal patterns in affective experiences are important for health outcomes. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Feedback delays can enhance anticipatory synchronization in human-machine interaction.

Research investigating the dynamics of coupled physical systems has demonstrated that small feedback delays can allow a dynamic response system to anticipate chaotic behavior. This counterintuitive phenomenon, termed anticipatory synchronization, has been observed in coupled electrical circuits, laser semi-conductors, and artificial neurons. Recent research indicates that the same process might also support the ability of humans to anticipate the occurrence of chaotic behavior in other individuals. Motivated by this latter work, the current study examined whether the process of feedback delay induced anticipatory synchronization could be employed to develop an interactive artificial agent capable of anticipating chaotic human movement. Results revealed that incorporating such delays within the movement-control dynamics of an artificial agent not only enhances an artificial agent's ability to anticipate chaotic human behavior, but to synchronize with such behavior in a manner similar to natural human-human anticipatory synchronization. The implication of these findings for the development of human-machine interaction systems is discussed.

Effects of Task Constraint on Action Dynamics.

The actualization of action possibilities (i.e., affordances) can often be accomplished in numerous ways. For instance, an individual could walk over to a rubbish bin to drop an item in or throw the piece of rubbish into the bin from some distance away. The aim of the current study was to investigate the action dynamics that emerge from such under-constrained task or action spaces using an object transportation task. Participants were instructed to transport balls between a starting location and a large wooden box located 9 meters away. The temporal interval between the sequential presentation of balls was manipulated as a control parameter and was expected to influence the distance participants moved prior to throwing or dropping the ball into the target box. A two-parameter state space derived from the Cusp Catastrophe Model was employed to illustrate how behavioral variability emerged as a consequence of the under-constrained task context. Two follow-up experiments demonstrated direct correspondence between model predictions and observed action dynamics as a function of increasing task constraints. Implications for modelling, the theory of affordances, and empirical studies more generally are discussed.