Human social motor solutions for human-machine interaction in dynamical task contexts.

Multiagent activity is commonplace in everyday life and can improve the behavioral efficiency of task performance and learning. Thus, augmenting social contexts with the use of interactive virtual and robotic agents is of great interest across health, sport, and industry domains. However, the effectiveness of human-machine interaction (HMI) to effectively train humans for future social encounters depends on the ability of artificial agents to respond to human coactors in a natural, human-like manner. One way to achieve effective HMI is by developing dynamical models utilizing dynamical motor primitives (DMPs) of human multiagent coordination that not only capture the behavioral dynamics of successful human performance but also, provide a tractable control architecture for computerized agents. Previous research has demonstrated how DMPs can successfully capture human-like dynamics of simple nonsocial, single-actor movements. However, it is unclear whether DMPs can be used to model more complex multiagent task scenarios. This study tested this human-centered approach to HMI using a complex dyadic shepherding task, in which pairs of coacting agents had to work together to corral and contain small herds of virtual sheep. Human-human and human-artificial agent dyads were tested across two different task contexts. The results revealed (i) that the performance of human-human dyads was equivalent to those composed of a human and the artificial agent and (ii) that, using a "Turing-like" methodology, most participants in the HMI condition were unaware that they were working alongside an artificial agent, further validating the isomorphism of human and artificial agent behavior.

Virtual auditory aperture passability.

Two experiments investigated (1) the ability of individuals to perceive the passability of apertures that are constructed using two virtual sounds sources and (2) the nature of the perceptual information that is used when determining passability in such a way. In the first experiment, participants judged whether they could successfully walk between two sound sources, heard through headphones, without turning their shoulders. We hypothesized that judgements would be accurate and driven by the detection of a proposed informational variable that relates head rotation, forward locomotion and aperture width. To test this hypothesis, we used motion tracking and a gain manipulation to alter apparent head rotation relative to virtual sound source positions and evaluated the effect on performance. Participants were able to accurately judge aperture passability based only on acoustic information. However, the gain manipulation did not show a significant influence on perceptual reports. The unexpected significant influence of lateral head movement on perceptual accuracy, however, does suggest that an alternative informational variable, based on lateral movement, may have been used. In the second experiment, a group of participants with wide shoulders was compared to a group with narrow shoulders on a similar task. Significant differences in minimally acceptable aperture width were found between the wide and narrow groups. When these aperture widths were scaled to the participants' shoulder widths, however, the differences were no longer present. These findings are consistent with previous studies investigating perception of passability and offer promising applications of virtual reality technology in the study of auditory perceptual abilities.

Felt heaviness is used to perceive the affordance for throwing but rotational inertia does not affect either.

Bingham et al. discovered a perceptible affordance property, composed of a relation between object weight and size, used to select optimal objects for long-distance throwing. Subsequent research confirmed this finding, but disconfirmed a hypothesis formulated by Bingham et al. about the information used to perceive the affordance. Following this, Zhu and Bingham investigated the possibility that optimal objects for throwing are selected as having a particular felt heaviness. The results supported this hypothesis. Perceived heaviness exhibits the size-weight illusion: to be perceived as equally heavy, larger objects must weigh more than smaller ones. Amazeen and Turvey showed that heaviness perception is determined by rotational inertia. We investigated whether rotational inertia would determine both perceived heaviness and throw-ability when spherical objects were held in the hand and wielded about the wrist. We found again that a particular judged heaviness corresponded to judged throw-ability. However, rotational inertia was found to have no effect on either judgment, suggesting that rotational inertia does not determine perceived heaviness of spherical objects held in the hand, as it did for the weighted-rod-type objects used by Amazeen and Turvey.

Joint action in a cooperative precision task: nested processes of intrapersonal and interpersonal coordination.

The authors determined the effects of changes in task demands on interpersonal and intrapersonal coordination. Participants performed a joint task in which one participant held a stick to which a circle was attached at the top (holding role), while the other held a pointer through the circle without touching its borders (pointing role). Experiment 1 investigated whether interpersonal and intrapersonal coordination varied depending on task difficulty. Results showed that interpersonal and intrapersonal coordination increased in degree and stability with increments in task difficulty. Experiment 2 explored the effects of individual constraints by increasing the balance demands of the task (one or both members of the pair stood in a less stable tandem stance). Results showed that interpersonal coordination increased in degree and stability as joint task demands increased and that coupling strength varied depending on joint and individual task constraints. In all, results suggest that interpersonal and intrapersonal coordination are affected by the nature of the task performed and the constraints it places on joint and individual performance.

Grasping Embodiment: Haptic Feedback for Artificial Limbs.

Upper-limb prostheses are subject to high rates of abandonment. Prosthesis abandonment is related to a reduced sense of embodiment, the sense of self-location, agency, and ownership that humans feel in relation to their bodies and body parts. If a prosthesis does not evoke a sense of embodiment, users are less likely to view them as useful and integrated with their bodies. Currently, visual feedback is the only option for most prosthesis users to account for their augmented activities. However, for activities of daily living, such as grasping actions, haptic feedback is critically important and may improve sense of embodiment. Therefore, we are investigating how converting natural haptic feedback from the prosthetic fingertips into vibrotactile feedback administered to another location on the body may allow participants to experience haptic feedback and if and how this experience affects embodiment. While we found no differences between our experimental manipulations of feedback type, we found evidence that embodiment was not negatively impacted when switching from natural feedback to proximal vibrotactile feedback. Proximal vibrotactile feedback should be further studied and considered when designing prostheses.

Task dynamics define the contextual emergence of human corralling behaviors.

Social animals have the remarkable ability to organize into collectives to achieve goals unobtainable to individual members. Equally striking is the observation that despite differences in perceptual-motor capabilities, different animals often exhibit qualitatively similar collective states of organization and coordination. Such qualitative similarities can be seen in corralling behaviors involving the encirclement of prey that are observed, for example, during collaborative hunting amongst several apex predator species living in disparate environments. Similar encirclement behaviors are also displayed by human participants in a collaborative problem-solving task involving the herding and containment of evasive artificial agents. Inspired by the functional similarities in this behavior across humans and non-human systems, this paper investigated whether the containment strategies displayed by humans emerge as a function of the task's underlying dynamics, which shape patterns of goal-directed corralling more generally. This hypothesis was tested by comparing the strategies naïve human dyads adopt during the containment of a set of evasive artificial agents across two disparate task contexts. Despite the different movement types (manual manipulation or locomotion) required in the different task contexts, the behaviors that humans display can be predicted as emergent properties of the same underlying task-dynamic model.

Feedback delays can enhance anticipatory synchronization in human-machine interaction.

Research investigating the dynamics of coupled physical systems has demonstrated that small feedback delays can allow a dynamic response system to anticipate chaotic behavior. This counterintuitive phenomenon, termed anticipatory synchronization, has been observed in coupled electrical circuits, laser semi-conductors, and artificial neurons. Recent research indicates that the same process might also support the ability of humans to anticipate the occurrence of chaotic behavior in other individuals. Motivated by this latter work, the current study examined whether the process of feedback delay induced anticipatory synchronization could be employed to develop an interactive artificial agent capable of anticipating chaotic human movement. Results revealed that incorporating such delays within the movement-control dynamics of an artificial agent not only enhances an artificial agent's ability to anticipate chaotic human behavior, but to synchronize with such behavior in a manner similar to natural human-human anticipatory synchronization. The implication of these findings for the development of human-machine interaction systems is discussed.

Perception of another person's maximum reach-with-jump height from walking kinematics.

Humans can perceive affordances (possibilities for action) for themselves and others, including the maximum overhead height reachable by jumping (reach-with-jump height, RWJ). While observers can accurately perceive maximum RWJ for another person without previously seeing the person jump, estimates improve after viewing the person walk, suggesting there is structure in walking kinematics that is informative about the ability to produce vertical force for jumping. We used principal component analysis (PCA) to identify patterns in human walking kinematics that specify another person's maximum RWJ ability, and to determine whether athletes are more sensitive than non-athletes to these patterns. Kinematic data during treadmill walking were collected and submitted to PCA to obtain loading values for the kinematic time series variables on the first principal component. Kinematic data were also used to create point-light (PL) displays, in which the movement kinematics of PL walkers were manipulated using the obtained PCA loading values to determine how changes in body-segment movements impacted perception of maximum RWJ height. While manipulating individual segmental loadings in the PL displays did not substantially affect RWJ estimates, PL displays created by replacing the PCA loadings of a high-jumper with those of a low-jumper, and vice versa, resulted in corresponding reversals of participants' RWJ estimates, suggesting that the global structure of walking kinematics carries information about another's maximum RWJ height. Athletes exhibited greater sensitivity than controls to the kinematic manipulations, indicating that they are better attuned to useful kinematic information as a result of their sport experience.

An information-based approach to action understanding.

It has been proposed that the ability to make sense of other agents' behavior relies on the activation of internal mechanisms that map action perception onto action execution. In this study we explored the constraints on this ability introduced by eyeheight-scaled information in the optic array. Short and tall participants provided maximum overhead reaching judgments for themselves and another participant. Perceptual information was manipulated by changing the participants' optically specified eyeheight. Observers were modestly accurate in perceiving maximum overhead reach for themselves and for another actor whose action capabilities differed. Perceived maximum overhead reach increased for both self- and other-judgments when the perceiver's eyeheight was increased. The results suggest an important role of perceptual information that has gone unrecognized in existing accounts of action understanding and prediction.

Tuning in to another person's action capabilities: perceiving maximal jumping-reach height from walking kinematics.

Three experiments investigated the ability to perceive the maximum height to which another actor could jump to reach an object. Experiment 1 determined the accuracy of estimates for another actor's maximal reach-with-jump height and compared these estimates to estimates of the actor's standing maximal reaching height and to estimates of the perceiver's own maximal reaching and reach-with-jump height. Perception of another actor's maximum reach-with-jump height was less accurate than the other estimates, but still accurate to within 8% error. The actor's reach-with-jump height was modified in Experiment 2 by attaching weights around the actor's ankles. Perceivers, who were explicitly aware of the manipulation, adjusted their maximum reach-with-jump estimates for the actor accordingly. In Experiment 3, perceivers were not explicitly aware of the weight manipulation, but provided significantly lower maximum reach-with-jump estimates after watching the actor walk while wearing the weights compared to estimates obtained after watching the actor walk while not wearing the weights. The results suggest that the actor's walking pattern was informative about the actor's capacity to produce a different action, jumping to reach an object.

Articulatory constraints on interpersonal postural coordination.

Cooperative conversation has been shown to foster interpersonal postural coordination. The authors investigated whether such coordination is mediated by the influence of articulation on postural sway. In Experiment 1, talkers produced words in synchrony or in alternation, as the authors varied speaking rate and word similarity. Greater shared postural activity was found for the faster speaking rate. In Experiment 2, the authors demonstrated that shared postural activity also increases when individuals speak the same words or speak words that have similar stress patterns. However, this increase in shared postural activity is present only when participants' data are compared with those of their partner, who was present during the task, but not when compared with the data of a member of a different pair speaking the same word sequences as those of the original partner. The authors' findings suggest that interpersonal postural coordination observed during conversation is mediated by convergent speaking patterns.

Rotational inertia and multimodal heaviness perception.

Perceived heaviness of wielded objects has been shown to be a function of the objects' rotational inertia--the objects' resistance to rotational acceleration. Studies have also demonstrated that if virtual objects rotate faster than the actual wielded object (i.e., a rotational gain is applied to virtual object motion), the wielded object is perceived as systematically lighter. The present research determined whether combining those inertial and visual manipulations would influence heaviness perception in a manner consistent with an inertial model of multimodal heaviness perception. Rotational inertia and optical rotational gain of wielded objects were manipulated to specify inertia multimodally. Both visual and haptic manipulations significantly influenced perceived heaviness. The results suggest that rotational inertia is detected multimodally and that multimodal heaviness perception conforms to an inertial model.

Rotational kinematics influence multimodal perception of heaviness.

Perceived heaviness has been shown to be specific to an object's rotational inertia (I), its resistance to rotational acceleration. According to the kinematic specification of dynamics (KSD) principle, we hypothesized that I is optically specified by rotational kinematics. Using virtual depictions of wielded objects, we investigated whether the visually detected rotational kinematics of wielded objects would influence perceived heaviness in a manner consistent with the inertial model of heaviness perception. We scaled the virtual object's movement so that it rotated more or less than its wielded counterpart, specifying lower and higher I, respectively. Perceived heaviness was inversely related to the rotational scaling factor, consistent with a KSD interpretation of the inertial model.

Postural responses to specific types of working memory tasks.

Standing participants performed working memory tasks that varied along three dimensions: (1) type of information presented (verbal or visual); (2) the primary cognitive process engaged (encoding or rehearsal); and (3) interference that targeted the working memory components (phonological loop and visual sketchpad) believed primarily active during rehearsal. Postural sway variability decreased during rehearsal and increased during encoding. The effects during encoding, but not during rehearsal, differed for verbal versus visual material. Changes in cognitive activity were also associated with changes in the temporal patterns of postural sway. The results suggest postural control is sensitive to the type of cognitive task performed.

Dual-task influences on retrieval from semantic memory and coordination dynamics.

Bimanual 1:1 rhythmic coordination was performed while retrieving words from a specified category. The effects of divided attention (DA) on coordination were indexed by changes in mean relative phase and recurrence measures of shared activity between the two limbs. Effects of DA on memory were indexed by deficits in exemplars retrieved relative to the baseline. Shifts in relative phase were found, accompanied by a recall deficit for DA during the retrieval task. DA also reduced the degree of shared activity between left and right rhythmic motions. Our discussion focuses on DA-induced parameter changes in retrieval and coordination dynamics, as well as on the hypothesis that stability is the general factor mediating dual-task performance.

Functional Task Constraints Foster Enhanced Postural Control in Children With Cerebral Palsy.

BACKGROUND: Postural instability is a classical characteristic of cerebral palsy (CP), but it has not been examined during functional play activity. Recent work has demonstrated that when motor tasks are made functionally more relevant, performance improves, even in children with movement pathology. It is possible that in a disease state, the underlying control mechanisms that are associated with healthy physiology must be elicited. OBJECTIVE: The study objective was to explore the utility of the functional play task methodology as a more rich and interpretable approach to the quantification of postural instability in children with CP. DESIGN: Postural stability measures obtained from a cross-sectional cohort of children with CP (n=30) were compared with stability measures taken from children with typical development (n=30) during a single measurement period. METHODS: Postural stability data were obtained with a portable force platform system. Postural sway was quantified during a precision manual functional play task. A baseline condition (no task) also was included. Postural sway variability and postural sway regularity were analyzed with analyses of variance. RESULTS: There was an apparent difference in postural control (greater irregularity, greater sway variability) during quiet stance between children with CP and peers with typical development; this difference was mitigated during the performance of the precision functional play task. LIMITATIONS: A small and nonprobability sample of convenience may limit the findings of this study. CONCLUSIONS: The findings illustrate flexibility and adaptability in the postural control system despite the pathological features associated with CP.

Encoding and retrieval during bimanual rhythmic coordination.

In 2 experiments, bimanual 1:1 rhythmic coordination was performed concurrently with encoding or retrieval of word lists. Effects of divided attention (DA) on coordination were indexed by changes in mean relative phase and recurrence measures of shared activity between the 2 limbs. Effects of DA on memory were indexed by deficits in recall relative to baseline. For DA at both encoding and retrieval, the equilibrium values of relative phase were shifted and the degree of shared activity between left and right rhythmic motions was reduced. Recall was reduced, however, only for DA at encoding. The results corroborate and extend those obtained with more conventional secondary tasks (e.g., visual reaction time), suggesting attention dissimilarities between episodic encoding and retrieval.

Concurrent cognitive task modulates coordination dynamics.

Does a concurrent cognitive task affect the dynamics of bimanual rhythmic coordination? In-phase coordination was performed under manipulations of phase detuning and movement frequency and either singly or in combination with an arithmetic task. Predicted direction-specific shifts in stable relative phase from 0° due to detuning and movement frequency were amplified by the cognitive task. Nonlinear cross-recurrence analysis suggested that this cognitive influence on the locations of the stable points or attractors of coordination entailed a magnification of attractor noise without a reduction in attractor strength. An approximation to these findings was achieved through parameter changes in a motion equation in relative phase. Results are discussed in terms of dual-task performance as limited resources, dynamics rather than chronometrics, and reparameterization rather than degradation.

Children with cerebral palsy effectively modulate postural control to perform a supra-postural task.

The purpose of this study was to determine whether signatures of adaptive postural control remain present in children with cerebral palsy (CP) when they performed a supra-postural task (i.e., a task performed above and beyond the control of posture) requiring them to balance a marble inside a tube held in the hands. Measures of center of pressure (COP) dynamics (how regular or predictable were the COP data as quantified by the sample entropy metric) and variability (as quantified by the COP standard deviation) were obtained from a sample of children with CP (n=30) and compared to the same measures taken from typically developing (TD) children. Children with CP demonstrated an apparent inefficiency in postural control (greater irregularity, greater sway variability) relative to TD peers during a quiet-stance (no supra-postural task) condition (p<.05). During supra-postural task performance, those differences were attenuated, though they remained statistically different (p<.05). The findings illustrate flexibility and adaptability in the postural control system, despite the pathological features associated with CP.

Harmony from chaos? Perceptual-motor delays enhance behavioral anticipation in social interaction.

Effective interpersonal coordination is fundamental to robust social interaction, and the ability to anticipate a coactor's behavior is essential for achieving this coordination. However, coordination research has focused on the behavioral synchrony that occurs between the simple periodic movements of coactors and, thus, little is known about the anticipation that occurs during complex, everyday interaction. Research on the dynamics of coupled neurons, human motor control, electrical circuits, and laser semiconductors universally demonstrates that small temporal feedback delays are necessary for the anticipation of chaotic events. We therefore investigated whether similar feedback delays would promote anticipatory behavior during social interaction. Results revealed that coactors were not only able to anticipate others' chaotic movements when experiencing small perceptual-motor delays, but also exhibited movement patterns of equivalent complexity. This suggests that such delays, including those within the human nervous system, may enhance, rather than hinder, the anticipatory processes that underlie successful social interaction.