When performing actions with robots, attribution of intentionality affects the sense of joint agency.

Sense of joint agency (SoJA) is the sense of control experienced by humans when acting with others to bring about changes in the shared environment. SoJA is proposed to arise from the sensorimotor predictive processes underlying action control and monitoring. Because SoJA is a ubiquitous phenomenon occurring when we perform actions with other humans, it is of great interest and importance to understand whether-and under what conditions-SoJA occurs in collaborative tasks with humanoid robots. In this study, using behavioral measures and neural responses measured by electroencephalography (EEG), we aimed to evaluate whether SoJA occurs in joint action with the humanoid robot iCub and whether its emergence is influenced by the perceived intentionality of the robot. Behavioral results show that participants experienced SoJA with the robot partner when it was presented as an intentional agent but not when it was presented as a mechanical artifact. EEG results show that the mechanism that influences the emergence of SoJA in the condition when the robot is presented as an intentional agent is the ability to form similarly accurate predictions about the sensory consequences of our own and others' actions, leading to similar modulatory activity over sensory processing. Together, our results shed light on the joint sensorimotor processing mechanisms underlying the emergence of SoJA in human-robot interaction and underscore the importance of attribution of intentionality to the robot in human-robot collaboration.

Tools and methods to study and replicate experiments addressing human social cognition in interactive scenarios.

In the last decade, scientists investigating human social cognition have started bringing traditional laboratory paradigms more "into the wild" to examine how socio-cognitive mechanisms of the human brain work in real-life settings. As this implies transferring 2D observational paradigms to 3D interactive environments, there is a risk of compromising experimental control. In this context, we propose a methodological approach which uses humanoid robots as proxies of social interaction partners and embeds them in experimental protocols that adapt classical paradigms of cognitive psychology to interactive scenarios. This allows for a relatively high degree of "naturalness" of interaction and excellent experimental control at the same time. Here, we present two case studies where our methods and tools were applied and replicated across two different laboratories, namely the Italian Institute of Technology in Genova (Italy) and the Agency for Science, Technology and Research in Singapore. In the first case study, we present a replication of an interactive version of a gaze-cueing paradigm reported in Kompatsiari et al. (J Exp Psychol Gen 151(1):121-136, 2022). The second case study presents a replication of a "shared experience" paradigm reported in Marchesi et al. (Technol Mind Behav 3(3):11, 2022). As both studies replicate results across labs and different cultures, we argue that our methods allow for reliable and replicable setups, even though the protocols are complex and involve social interaction. We conclude that our approach can be of benefit to the research field of social cognition and grant higher replicability, for example, in cross-cultural comparisons of social cognition mechanisms.

Cultural differences in joint attention and engagement in mutual gaze with a robot face.

Joint attention is a pivotal mechanism underlying human ability to interact with one another. The fundamental nature of joint attention in the context of social cognition has led researchers to develop tasks that address this mechanism and operationalize it in a laboratory setting, in the form of a gaze cueing paradigm. In the present study, we addressed the question of whether engaging in joint attention with a robot face is culture-specific. We adapted a classical gaze-cueing paradigm such that a robot avatar cued participants' gaze subsequent to either engaging participants in eye contact or not. Our critical question of interest was whether the gaze cueing effect (GCE) is stable across different cultures, especially if cognitive resources to exert top-down control are reduced. To achieve the latter, we introduced a mathematical stress task orthogonally to the gaze cueing protocol. Results showed larger GCE in the Singapore sample, relative to the Italian sample, independent of gaze type (eye contact vs. no eye contact) or amount of experienced stress, which translates to available cognitive resources. Moreover, since after each block, participants rated how engaged they felt with the robot avatar during the task, we observed that Italian participants rated as more engaging the avatar during the eye contact blocks, relative to no eye contact while Singaporean participants did not show any difference in engagement relative to the gaze. We discuss the results in terms of cultural differences in robot-induced joint attention, and engagement in eye contact, as well as the dissociation between implicit and explicit measures related to processing of gaze.

Modulatory Effects of Communicative Gaze on Attentional Orienting Are Driven by Dorsomedial Prefrontal Cortex but Not Right Temporoparietal Junction.

Communicative gaze (e.g., mutual or averted) has been shown to affect attentional orienting. However, no study to date has clearly separated the neural basis of the pure social component that modulates attentional orienting in response to communicative gaze from other processes that might be a combination of attentional and social effects. We used TMS to isolate the purely social effects of communicative gaze on attentional orienting. Participants completed a gaze-cueing task with a humanoid robot who engaged either in mutual or in averted gaze before shifting its gaze. Before the task, participants received either sham stimulation (baseline), stimulation of right TPJ (rTPJ), or dorsomedial prefrontal cortex (dmPFC). Results showed, as expected, that communicative gaze affected attentional orienting in baseline condition. This effect was not evident for rTPJ stimulation. Interestingly, stimulation to rTPJ also canceled out attentional orienting altogether. On the other hand, dmPFC stimulation eliminated the socially driven difference in attention orienting between the two gaze conditions while maintaining the basic general attentional orienting effect. Thus, our results allowed for separation of the pure social effect of communicative gaze on attentional orienting from other processes that are a combination of social and generic attentional components.

The impact of facial expression and communicative gaze of a humanoid robot on individual Sense of Agency.

Sense of Agency (SoA) is the feeling of control over one's actions and their outcomes. A well-established implicit measure of SoA is the temporal interval estimation paradigm, in which participants estimate the time interval between a voluntary action and its sensory consequence. In the present study, we aimed to investigate whether the valence of action outcome modulated implicit SoA. The valence was manipulated through interaction partner's (i) positive/negative facial expression, or (ii) type of gaze (gaze contact or averted gaze). The interaction partner was the humanoid robot iCub. In Experiment 1, participants estimated the time interval between the onset of their action (head movement towards the robot), and the robot's facial expression (happy vs. sad face). Experiment 2 was identical, but the outcome of participants' action was the type of robot's gaze (gaze contact vs. averted). In Experiment 3, we assessed-in a within-subject design-the combined effect of robot's type of facial expression and type of gaze. Results showed that, while the robot's facial expression did not affect participants' SoA (Experiment 1), the type of gaze affected SoA in both Experiment 2 and Experiment 3. Overall, our findings showed that the robot's gaze is a more potent factor than facial expression in modulating participants' implicit SoA.

Artificial scaffolding: Augmenting social cognition by means of robot technology.

The concept of scaffolding refers to the support that the environment provides in the acquisition and consolidation of new abilities. Technological advancements allow for support in the acquisition of cognitive capabilities, such as second language acquisition using simple smartphone applications There is, however, one domain of cognition that has been scarcely addressed in the context of technologically assisted scaffolding: social cognition. We explored the possibility of supporting the acquisition of social competencies of a group of children with autism spectrum disorder engaged in a rehabilitation program (age = 5.8 ± 1.14, 10 females, 33 males) by designing two robot-assisted training protocols tailored to Theory of Mind competencies. One protocol was performed with a humanoid robot and the other (control) with a non-anthropomorphic robot. We analyzed changes in NEPSY-II scores before and after the training using mixed effects models. Our results showed that activities with the humanoid significantly improved NEPSY-II scores on the ToM scale. We claim that the motor repertoire of humanoids makes them ideal platforms for artificial scaffolding of social skills in individuals with autism, as they can evoke similar social mechanisms to those elicited in human-human interaction, without providing the same social pressure that another human might exert.

To follow or not to follow your gaze: The interplay between strategic control and the eye contact effect on gaze-induced attention orienting.

Eye contact constitutes a strong communicative signal in human interactions and has been shown to modulate various cognitive processes and states. However, little is known about its impact on gaze-mediated attentional orienting in the context of its interplay with strategic top-down control. Here, we aimed at investigating how the social component of eye contact interacts with the top-down strategic control. To this end, we designed a gaze cuing paradigm with the iCub humanoid robot, in which iCub either established eye contact with the participants before averting its gaze or avoided their eyes. Across four experiments, we manipulated gaze cue validity to either elicit strategic top-down inhibitory activity (25% validity) or to allow for relaxing the control mechanisms (50% validity). Also, we manipulated the stimulus-onset-asynchrony (SOA) to examine the dynamics of the top-down modulatory effects. Our results showed that eye contact influenced the gaze cuing effect when the strategic control was not required, by prolonging the prioritized processing of the gazed-at locations. Thus, the effect was observed only when the measurement was taken after a sufficient amount of time (1,000 ms SOA). However, when inhibitory control was necessary (25% validity), the social component was not potent enough to exert influence over the gaze cuing effect independently. Overall, we propose that strategic top-down control is the primary driving force over the gaze cuing effect and that the social aspect plays a modulatory effect by prolonging prioritized processing of gazed-at locations. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Mutual gaze with a robot affects human neural activity and delays decision-making processes.

In most everyday life situations, the brain needs to engage not only in making decisions but also in anticipating and predicting the behavior of others. In such contexts, gaze can be highly informative about others' intentions, goals, and upcoming decisions. Here, we investigated whether a humanoid robot's gaze (mutual or averted) influences the way people strategically reason in a social decision-making context. Specifically, participants played a strategic game with the robot iCub while we measured their behavior and neural activity by means of electroencephalography (EEG). Participants were slower to respond when iCub established mutual gaze before their decision, relative to averted gaze. This was associated with a higher decision threshold in the drift diffusion model and accompanied by more synchronized EEG alpha activity. In addition, we found that participants reasoned about the robot's actions in both conditions. However, those who mostly experienced the averted gaze were more likely to adopt a self-oriented strategy, and their neural activity showed higher sensitivity to outcomes. Together, these findings suggest that robot gaze acts as a strong social signal for humans, modulating response times, decision threshold, neural synchronization, as well as choice strategies and sensitivity to outcomes. This has strong implications for all contexts involving human-robot interaction, from robotics to clinical applications.

Eye contact during joint attention with a humanoid robot modulates oscillatory brain activity.

Eye contact established by a human partner has been shown to affect various cognitive processes of the receiver. However, little is known about humans' responses to eye contact established by a humanoid robot. Here, we aimed at examining humans' oscillatory brain response to eye contact with a humanoid robot. Eye contact (or lack thereof) was embedded in a gaze-cueing task and preceded the phase of gaze-related attentional orienting. In addition to examining the effect of eye contact on the recipient, we also tested its impact on gaze-cueing effects (GCEs). Results showed that participants rated eye contact as more engaging and responded with higher desynchronization of alpha-band activity in left fronto-central and central electrode clusters when the robot established eye contact with them, compared to no eye contact condition. However, eye contact did not modulate GCEs. The results are interpreted in terms of the functional roles involved in alpha central rhythms (potentially interpretable also as mu rhythm), including joint attention and engagement in social interaction.

Examining joint attention with the use of humanoid robots-A new approach to study fundamental mechanisms of social cognition.

This article reviews methods to investigate joint attention and highlights the benefits of new methodological approaches that make use of the most recent technological developments, such as humanoid robots for studying social cognition. After reviewing classical approaches that address joint attention mechanisms with the use of controlled screen-based stimuli, we describe recent accounts that have proposed the need for more natural and interactive experimental protocols. Although the recent approaches allow for more ecological validity, they often face the challenges of experimental control in more natural social interaction protocols. In this context, we propose that the use of humanoid robots in interactive protocols is a particularly promising avenue for targeting the mechanisms of joint attention. Using humanoid robots to interact with humans in naturalistic experimental setups has the advantage of both excellent experimental control and ecological validity. In clinical applications, it offers new techniques for both diagnosis and therapy, especially for children with autism spectrum disorder. The review concludes with indications for future research, in the domains of healthcare applications and human-robot interaction in general.

On the role of eye contact in gaze cueing.

Most experimental protocols examining joint attention with the gaze cueing paradigm are "observational" and "offline", thereby not involving social interaction. We examined whether within a naturalistic online interaction, real-time eye contact influences the gaze cueing effect (GCE). We embedded gaze cueing in an interactive protocol with the iCub humanoid robot. This has the advantage of ecological validity combined with excellent experimental control. Critically, before averting the gaze, iCub either established eye contact or not, a manipulation enabled by an algorithm detecting position of the human eyes. For non-predictive gaze cueing procedure (Experiment 1), only the eye contact condition elicited GCE, while for counter-predictive procedure (Experiment 2), only the condition with no eye contact induced GCE. These results reveal an interactive effect of strategic (gaze validity) and social (eye contact) top-down components on the reflexive orienting of attention induced by gaze cues. More generally, we propose that naturalistic protocols with an embodied presence of an agent can cast a new light on mechanisms of social cognition.

Longitudinal study of a NoGo-P3 event-related potential component following mild traumatic brain injury in adults.

BACKGROUND: Event-related potentials have repeatedly revealed electrophysiological markers of cognitive dysfunction associated with Mild Traumatic Brain Injury (MTBI) and may represent a sensitive tool to guide cognitive rehabilitative interventions. We previously found patients with symptomatic MTBI characterized by smaller P300 (or P3) wave amplitudes in a NoGo-P3 subcomponent in the acute phase of the injury. The goal of this longitudinal study was to investigate whether this early NoGo-P3 subcomponent differs over time in symptomatic MTBI patients and healthy controls. METHODS: We included adults with a diagnosis of MTBI and individually matched healthy controls tested at 1 week, 3 months, and 1 year after the MTBI. Symptoms were assessed by the Rivermead Post-Concussion Symptoms Questionnaire. NoGo-P3 was collected by using a cued Go/NoGo task and the relevant subcomponent was extracted by independent component analysis. RESULTS: Among 53 adults with a diagnosis of MTBI and 53 controls, we included 35 with symptomatic MTBI and 35 matched healthy controls (18 females each group; mean age 34.06±13.15 and 34.26±12.98 years). Amplitudes for the early NoGo-P3 subcomponent were lower for symptomatic MTBI patients than controls (P<0.05) at 1 week post-injury. Furthermore, mixed ANOVA revealed a significant time by group interaction (P<0.05), so the effect of time differed for symptomatic MTBI patients and healthy controls. The amplitudes for MTBI patients normalized from 1 week to 3 months post-injury and were comparable to those of controls from 3 months to 1 year post-injury. However, amplitudes for 3 MTBI patients with particularly severe complaints 1 year post-injury did not normalize and were lower than those for the remaining MTBI sample (P<0.05). CONCLUSIONS: Selected event-related potentials can be used as a sensitive and objective tool to illustrate the cognitive consequences of and recovery after MTBI.

Test-retest reliability of ERP components: A short-term replication of a visual Go/NoGo task in ADHD subjects.

Event-related potentials (ERPs) have been widely used to investigate brain functioning in children with Attention Deficit Hyperactivity Disorder (ADHD) in both research and diagnostic settings. To ensure the efficiency of ERP techniques in ADHD diagnosis and in longitudinal observational studies, the test-retest reliability of the affected population must be validated. Thus, the present article assesses the short-term test-retest reliability of certain early and late ERPs (i.e., P1, N1, N2, P2, P3), as well as independent components (ICs) decomposed from the above mentioned ERPs (IC P3 Go, IC P3 NoGo early, IC P3 NoGo late) relevant to ADHD, through the Intraclass Correlation Coefficient (ICC). More specifically, we employ a cued visual Go/NoGo paradigm for recording ERPs from 22 children with ADHD (mean age 12.2), twice within 30 min. Amplitudes and latencies are calculated by the 'peak amplitude' method and by a variation of the fractional area. Results for amplitudes lie mostly within the 'good' and 'excellent' range for both measurement methods, while ICC for latencies is more variable ranging from 'poor' to 'excellent' results. Crucially, the ICs, which are associated with distinct functionally independent processes of the executive attentional system have shown a comparable test-retest reliability with the raw ERPs. Our results are consistent with other reliability studies of neurotypical population in the literature, and as such, consist initial evidence that ERPs could be reliable neurophysiological markers for the ADHD population.

Altered cognitive processes in the acute phase of mTBI: an analysis of independent components of event-related potentials.

Mild traumatic brain injuries (mTBI) generate acute disruptions of brain function and a subset of patients shows persisting cognitive, affective, and somatic symptoms. Deficits in the executive function domain are among the more frequent cognitive impairments reported by mTBI patients. By means of independent component analysis, event-related potential components from a visual cued go/nogo task, namely contingent negative variation (CNV) and NoGo-P3, were decomposed into distinct independent components that have been shown to be associated with the executive processes of energization, monitoring, and task setting. A group of symptomatic mTBI patients was compared with a group of controls matched for sex, age, and education. Patients showed reduced amplitudes in the late CNV as well as in the early NoGo-P3 subcomponents. Whereas the decreased CNVlate component indicates an impaired ability to generate representations of stimulus-response associations and to energize the maintenance of response patterns, the reduced P3NOGOearly component suggests a deficient ability to invest attentional effort in the initiation of response patterns in mTBI patients. Besides indicating the effects of mTBI on cognitive brain processing, the results may open up the possibility for assessing individual mTBI profiles and facilitate personalized rehabilitative measures.

Magnetic resonance dispersion imaging for localization of angiogenesis and cancer growth.

PURPOSE: Cancer angiogenesis can be imaged by using dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI). Pharmacokinetic modeling can be used to assess vascular perfusion and permeability, but the assessment of angiogenic changes in the microvascular architecture remains challenging. This article presents 2 models enabling the characterization of the microvascular architecture by DCE-MRI. THEORY: The microvascular architecture is reflected in the dispersion coefficient according to the convective dispersion equation. A solution of this equation, combined with the Tofts model, permits defining a dispersion model for magnetic resonance imaging. A reduced dispersion model is also presented. METHODS: The proposed models were evaluated for prostate cancer diagnosis. Dynamic contrast-enhanced magnetic resonance imaging was performed, and concentration-time curves were calculated in each voxel. The simultaneous generation of parametric maps related to permeability and dispersion was obtained through model fitting. A preliminary validation was carried out through comparison with the histology in 15 patients referred for radical prostatectomy. RESULTS: Cancer localization was accurate with both dispersion models, with an area under the receiver operating characteristic curve greater than 0.8. None of the compared parameters, aimed at assessing vascular permeability and perfusion, showed better results. CONCLUSIONS: A new DCE-MRI method is proposed to characterize the microvascular architecture through the assessment of intravascular dispersion, without the need for separate arterial-input-function estimation. The results are promising and encourage further research.