Self-administered questionnaires enhance emotion estimation of individuals with autism spectrum disorders in a robotic interview setting.

Background: Robots offer many unique opportunities for helping individuals with autism spectrum disorders (ASD). Determining the optimal motion of robots when interacting with individuals with ASD is important for achieving more natural human-robot interactions and for exploiting the full potential of robotic interventions. Most prior studies have used supervised machine learning (ML) of user behavioral data to enable robot perception of affective states (i.e., arousal and valence) and engagement. It has previously been suggested that including personal demographic information in the identification of individuals with ASD is important for developing an automated system to perceive individual affective states and engagement. In this study, we hypothesized that assessing self-administered questionnaire data would contribute to the development of an automated estimation of the affective state and engagement when individuals with ASD are interviewed by an Android robot, which will be linked to implementing long-term interventions and maintaining the motivation of participants. Methods: Participants sat across a table from an android robot that played the role of the interviewer. Each participant underwent a mock job interview. Twenty-five participants with ASD (males 22, females 3, average chronological age = 22.8, average IQ = 94.04) completed the experiment. We collected multimodal data (i.e., audio, motion, gaze, and self-administered questionnaire data) to train a model to correctly classify the state of individuals with ASD when interviewed by an android robot. We demonstrated the technical feasibility of using ML to enable robot perception of affect and engagement of individuals with ASD based on multimodal data. Results: For arousal and engagement, the area under the curve (AUC) values of the model estimates and expert coding were relatively high. Overall, the AUC values of arousal, valence, and engagement were improved by including self-administered questionnaire data in the classification. Discussion: These findings support the hypothesis that assessing self-administered questionnaire data contributes to the development of an automated estimation of an individual's affective state and engagement. Given the efficacy of including self-administered questionnaire data, future studies should confirm the effectiveness of such long-term intervention with a robot to maintain participants' motivation based on the proposed method of emotion estimation.

Therapeutic potential of robots for people who stutter: a preliminary study.

Introduction: Growing anecdotal evidence suggests the feasibility of robotic intervention for people who suffer from disorders related to state anxiety. Few studies have been conducted on utilizing robots for persons who stutter (PWS). The present study examines the feasibility of using a robot for speech therapy for PWS. Methods: We prepared four settings (i.e., interviews with unfamiliar persons, interviews with unfamiliar communication robots, reading sentences aloud with a tandem robot that can utter the same words as a user by repeating the user's voice after a short delay, and reading sentences aloud while being alone). We assessed the potential of the robots as both interlocutors and practice partners in training with delayed auditory feedback (DAF) for PWS. Moreover, we assessed the relationship between the trait of stuttering and the participants' affinity to the robots. Results: Eleven PWS participated in the study. Eight (72.7%) participants had fewer stuttering-related psychological symptoms when they communicated with robots than when they communicated with humans. Spearman's rank correlation analysis revealed that there was a significant negative correlation between the Modified Erickson Communication Attitude scale (S-24) and the difference between the scores for stuttering-related psychological symptoms pertaining to the communication robot and humans (p < 0.01). Six participants (54.5%) had fewer stuttering-related psychological symptoms when they read aloud with the tandem robot than when they read aloud alone. There were significant positive correlations between S-24 and the differences between the scores for stuttering-related psychological symptoms when reading aloud with the tandem robot and those when reading aloud alone (p < 0.01). Discussion: The communication robot and tandem utterance robot can sometimes be burdensome, although both robots were always easier to talk to for PWS in this preliminary study. The participants with positive speech-related attitudes were more inclined to decrease stuttering-related psychological symptoms when communicating with CommU than when communicating with humans. The participants whose speech-related attitudes were negative were more inclined to show a decrease in stuttering-related psychological symptoms when reading aloud with the tandem robot. Further studies are needed to provide more detailed information.

Development of depression assessment tools using humanoid robots -Can tele-operated robots talk with depressive persons like humans?

BACKGROUND: Depression is a common mental disorder and causes significant social loss. Early intervention for depression is important. Nonetheless, depressed patients tend to conceal their symptoms from others based on shame and stigma, thus hesitate to visit psychiatrists especially during early phase. We hypothesize that application of humanoid robots would be a novel solution. Depressed patients may feel more comfortable talking with such robots than humans. METHODS: We recruited 13 patients with major depressive disorder (MDD) and 27 healthy volunteers as controls. Participants took both tele-operated humanoid robot and human interviews to evaluate severity of depression using the Hamilton Depression Rating Scale (HDRS). In addition, participants completed a self-administered questionnaire asking about their impressions of the robot interview. RESULTS: Confidence interval and t-test analysis have revealed that the HDRS scores are equally reliable between robot and human interviews. No significant differences were observed between the two interviews regarding "nervousness about the interview" and "hesitancy to talk about depressed moods and suicidal ideation." Compared to human interviews, robot interviews yielded significantly lower scores on shame-related factors especially among patients with MDD. LIMITATION: Small sample size, and the evaluator is male only. CONCLUSIONS: This is the first report to show the reliability of tele-operated humanoid robot interviews for assessment of depression. Robot interviews are potentially equally reliable as human interviews. Robot interviews are suggested to be more appropriate in assessing shame-related suppressed emotions and hidden thoughts of depressed patients in clinical practice, which may reduce the stigma associated with depression.

Effect of having and switching multiple avatars on the operator's right to talk and receive social support.

People with communication difficulties encounter several challenges in their daily online interactions, such as a limited right to talk (RoT), insufficient social support (SS), and a low sense of being attended to (SoBA). Computer-mediated technologies are limited in addressing such problems owing to their limited capacity in transferring verbal and nonverbal cues between users. In this study, to address the limited RoT, low SS, and low SoBA challenges, we proposed a robotic video conference system with two teleoperated robot avatars. The proposed system was compared with another robotic video conference system that adopts only one teleoperated robot avatar. In the field experiment, 37 participants took part in two discussion sessions using each system type, where RoT, SS, and SoBA were adopted as the measured indices. The proposed system significantly increased the users' RoT and SS compared with other robotic video conference systems. This study contributes to the literature by demonstrating the effect exerted by the type of robotic video conference adopted on users' feelings about RoT, SS, and SoBA.

Social skills training using multiple humanoid robots for individuals with autism spectrum conditions.

Introduction: Social skills training (SST) is used to help individuals with autism spectrum conditions (ASC) better understand the perspectives of others and social interactions, develop empathy skills, and learn how to engage with others socially. However, many individuals with ASC cannot easily sustain high motivation and concentration during such an intervention when it is administered by humans. We developed a social skills training program using multiple humanoid robots (STUH), including an android robot, that aimed to enable individuals with ASC to become familiar with the perspectives of others and improve their sociability and empathy skills. The objective of the present study was to investigate the effectiveness of STUH for these individuals. Methods: In STUH, we prepared 50 social exercises that consisted of conversations and behavioral interactions between an android robot and a simple humanoid robot. We prepared another humanoid robot that featured a cartoon-like and mechanical design, which played the role of host. In the first half-session of STUH, participants worked on the exercise from the perspective of an outsider. In the second half-session of STUH, they simulated experience by using robots as their avatars. The intervention associated with STUH was conducted for five days in total. We conducted an analysis of variance (ANOVA) featuring the intervention time point as the independent variable to examine changes in each score on the sociability index items. Results: In total, 14 individuals with ASC participated in the study. The results of multiple comparison tests using the Bonferroni method indicated that all sociability index items improved between preintervention and follow-up. Our program enabled the participants to become familiar with the perspectives of others and improve their sociability. Discussion: Given the promising results of this study, future studies featuring long-term follow-up should be conducted to draw definitive conclusions about the efficacy of our training system.

A new group-based online job interview training program using computer graphics robots for individuals with autism spectrum disorders.

Introduction: Job interviews are a major barrier to employment for individuals with autism spectrum disorders (ASD). During the coronavirus pandemic, establishing online job interview training at home was indispensable. However, many hurdles prevent individuals with ASD from concentrating on online job interview training. To facilitate the acquisition of interview skills from home for individuals with ASD, we developed a group interview training program with a virtual conferencing system (GIT-VICS Program) that uses computer graphics (CG) robots. Methods: This study investigated the feasibility of the GIT-VICS Program in facilitating skill acquisition for face-to-face job interviews in pre-post measures. In the GIT-VICS Program, five participants were grouped and played the roles of interviewees (1), interviewers (2), and human resources (2). They alternately practiced each role in GIT-VICS Program sessions conducted over 8 or 9 days over three consecutive weeks. Before and after the GIT-VICS Program, the participants underwent a mock face-to-face job interview with two experienced human interviewers (MFH) to evaluate its effect. Results: Fourteen participants completed the trial procedures without experiencing any technological challenges or distress that would have led to the termination of the session. The GIT-VICS Program improved their job interview skills (verbal competence, nonverbal competence, and interview performance). Discussion: Given the promising results of this study and to draw clear conclusions about the efficacy of CG robots for mock online job interview training, future studies adding appropriate guidance for manner of job interview by experts are needed.

Influence of collaborative customer service by service robots and clerks in bakery stores.

In recent years, various service robots have been deployed in stores as recommendation systems. Previous studies have sought to increase the influence of these robots by enhancing their social acceptance and trust. However, when such service robots recommend a product to customers in real environments, the effect on the customers is influenced not only by the robot itself, but also by the social influence of the surrounding people such as store clerks. Therefore, leveraging the social influence of the clerks may increase the influence of the robots on the customers. Hence, we compared the influence of robots with and without collaborative customer service between the robots and clerks in two bakery stores. The experimental results showed that collaborative customer service increased the purchase rate of the recommended bread and improved the impressions of the robot and store experience of the customers. Because the results also showed that the workload required for the clerks to collaborate with the robot was not high, this study suggests that all stores with service robots may demonstrate high effectiveness in introducing collaborative customer service.

Sound masking by a low-pitch speech-shaped noise improves a social robot's talk in noisy environments.

Introduction: There has been a surge in the use of social robots for providing information, persuasion, and entertainment in noisy public spaces in recent years. Considering the well-documented negative effect of noise on human cognition, masking sounds have been introduced. Masking sounds work, in principle, by making the intrusive background speeches less intelligible, and hence, less distracting. However, this reduced distraction comes with the cost of increasing annoyance and reduced cognitive performance in the users of masking sounds. Methods: In a previous study, it was shown that reducing the fundamental frequency of the speech-shaped noise as a masking sound significantly contributes to its being less annoying and more efficient. In this study, the effectiveness of the proposed masking sound was tested on the performance of subjects listening to a lecture given by a social robot in a noisy cocktail party environment. Results: The results indicate that the presence of the masking sound significantly increased speech comprehension, perceived understandability, acoustic satisfaction, and sound privacy of the individuals listening to the robot in an adverse listening condition. Discussion: To the knowledge of the authors, no previous work has investigated the application of sound masking technology in human-robot interaction designs. The future directions of this trend are discussed.

Intervention with a humanoid robot avatar for individuals with social anxiety disorders comorbid with autism spectrum disorders.

For some individuals with social anxiety disorders (SAD) comorbid with autism spectrum disorders (ASD), it is difficult to speak in front of others. Herein, we report the case of a patient with SAD comorbid with ASD who could not speak in front of others until she used a humanoid robot as her avatar. During the intervention, her personality changed from shy to outgoing, which is explained by the Proteus effect. These case findings suggest that interventions with a robot avatar might improve the motivation for individuals with SAD comorbid with ASD who cannot speak in front of others to communicate.

A cross-cultural investigation of people's intuitive beliefs about the origins of cognition.

Nature vs. nurture is an enduring theme of studies of the mind. Past studies on American children and adults have revealed a preference for thinking that even fundamental cognitive abilities documented in human infants and non-human species are late-emerging and reliant on learning and nurture. However, little is known about the generalizability of this "intuitive empiricist" belief and what factors may help explain it. Adult participants (N = 600) reported their beliefs about the emergence of several fundamental cognitive abilities demonstrated by preverbal infants. Studies 1A-1C showed that adults from both Japan and the US similarly estimated an older age of onset for cognitive abilities in human children as compared to the findings of cognitive science and consistently attributed acquisition of these abilities to learning rather than innateness in humans, and they made these learning attributions more so for humans than for non-human species. Study 2 showed that participants' beliefs about biological evolution versus creationism were related to their age onset estimates for fundamental cognitive abilities, and their beliefs about the malleability of intelligence were related to participants' explanations of the origin of fundamental cognitive abilities. These findings suggest generalizable preferences for nurture over nature across both Eastern and Western cultures (Japan and the United States), which may be related to people's beliefs about human origins and the power of learning.

Effect of robot's vertical body movement on its perceived emotion: A preliminary study on vertical oscillation and transition.

The emotion expressions of social robots are some of the most important developments in recent studies on human-robot interactions (HRIs). Several research studies have been conducted to assess effective factors to improve the quality of emotion expression of the robots. In this study, we examined the effects of a robot's vertical oscillation and transition on the quality of its emotion expression, where the former indicates the periodic up/down movement of the body of the robot, while the latter indicates a one-time up or down movement. Short-term and long-term emotion expressions of the robot were studied independently for the four basic emotions described in the circumplex model of emotions: joy, anger, sadness, and relief. We designed an experiment with an adequate statistical power and minimum sample size of human subjects based on a priori power analysis. Human subjects were asked to evaluate the robot's emotion expressions by watching its video with/without vertical movement. The results of the experiment showed that for the long-term emotions, the speed of vertical oscillation corresponded to the degree of arousal of the emotion expression as noted in the circumplex model; this indicated that fast oscillations improved the emotion expression with a higher degree of arousal, such as joy and anger, while slow or no oscillations were more suited to emotions with a lower degree of arousal, such as sadness and relief. For the short-term emotions, the direction of the vertical transition corresponded to the degree of valence for most of the expressed emotions, while the speed of vertical oscillation reflected the degree of arousal. The findings of this study can be adopted in the development of conversational robots to enhance their emotion expression.

Behavioral Assessment of a Humanoid Robot When Attracting Pedestrians in a Mall.

Research is currently being conducted on the use of robots as human labor support technology. In particular, the service industry needs to allocate more manpower, and it will be important for robots to support people. This study focuses on using a humanoid robot as a social service robot to convey information in a shopping mall, and the types of robot behaviors were analyzed. In order to convey the information, two processes must occur. Pedestrians must stop in front of the robot, and the robot must continue the engagement with them. For the purpose of this study, three types of autonomous robot behaviors were analyzed and compared in these processes in the experiment: greeting, in-trouble, dancing behaviors. After interactions were attempted with 5,000+ pedestrians, this study revealed that the in-trouble behavior can make pedestrians stop more and stay longer. In addition, in order to evaluate the effectiveness of the robot in a real environment, the comparative results between three robot behaviors and human advertisers revealed that (1) the results of the greeting and dancing behavior are comparable to those of the humans, and (2) the performance of the in-trouble behavior in providing information tasks is higher than that of all human advertisers. These findings demonstrate that the performance of robots is comparable to that of humans in providing information tasks in a limited environment; therefore, it is expected that service robots as a labor support technology will be able to perform well in the real world.

Android Robot Promotes Disclosure of Negative Narratives by Individuals With Autism Spectrum Disorders.

Many individuals with autism spectrum disorders (ASD) demonstrate some challenges with personal narrative writing. Sentence completion tests (SCT) is a class of semi-structured projective techniques and encourage respondents to disclose their private narratives. Even in SCT, only providing beginning of sentences is inadequate to compensate atypicalities in their creativity and imagination, and self-disclosure is difficult for many individuals with ASD. It is reported that many individuals with ASD often achieve a higher degree of task engagement through interactions with robots and that robotic systems may be useful in eliciting and promoting social communication such as self-disclosure for some individuals with ASD. There is a possibility that exemplification by android robots in place of human interviewers can result in a higher degree of task engagement for individuals with ASD. The objective of this study was to investigate whether additional exemplifications by android robots in the SCT can prompt self-disclosure for individuals with ASD. We compared the difference in disclosure statements and subjective emotion in the testing paper of the SCT in additional exemplification by an android robot and a human interviewer. In addition, we assessed the disclosure statements and subjective emotions in the SCT, for which exemplifications were written on testing paper to make the comparison. Our quantitative data suggested that exemplification by android robot promoted more self-disclosure, especially about the negative topic compared to exemplification by a human interviewer and that written on test paper. In addition, the level of participant embarrassment in response to exemplification by the android robot seemed to be lower compared to that in the human interviewer condition. In the assessment and support for individuals with ASD, eliciting self-disclosure is a pressing issue. It is hoped that the appropriate use of robots will lead to a better understanding and support for their application.

Multiple Groups of Agents for Increased Movement Interference and Synchronization.

We examined the influence of groups of agents and the type of avatar on movement interference. In addition, we studied the synchronization of the subject with the agent. For that, we conducted experiments utilizing human subjects to examine the influence of one, two, or three agents, as well as human or robot avatars, and finally, the agent moving biologically or linearly. We found the main effect on movement interference was the number of agents; namely, three agents had significantly more influence on movement interference than one agent. These results suggest that the number of agents is more influential on movement interference than other avatar characteristics. For the synchronization, the main effect of the type of the agent was revealed, showing that the human agent kept more synchronization compared to the robotic agent. In this experiment, we introduced an additional paradigm on the interference which we called synchronization, discovering that a group of agents is able to influence this behavioral level as well.

Alignment of the attitude of teleoperators with that of a semi-autonomous android.

Studies on social robots that can communicate with humans are increasingly important. In particular, semi-aautonomous robots have shown potential for practical applications in which robot autonomy and human teleoperation are jointly used to accomplish difficult tasks. However, it is unknown how the attitude represented in the autonomous behavior of the robots affects teleoperators. Previous studies reported that when humans play a particular role, their attitudes align with that role. The teleoperators of semi-autonomous robots also play the role given to the robots and may assimilate their autonomous expression. We hypothesized that the attitude of teleoperators may align with that of robots through teleoperation. To verify this, we conducted an experiment with conditions under which a participant operated a part of the body of an android robot that autonomously expressed a preferential attitude toward a painting and a condition under which they did not. Experimental results demonstrated that the preferential attitude of participants who teleoperated the android aligned statistically significantly more with that of the robot in comparison to those who did not teleoperate it, thereby supporting our hypothesis. This finding is novel regarding attitude change in teleoperators of semi-autonomous robots and can support the implementation of effective human-robot collaboration systems.

Differences in the Optimal Motion of Android Robots for the Ease of Communications Among Individuals With Autism Spectrum Disorders.

Android robots are employed in various fields. Many individuals with autism spectrum disorders (ASD) have the motivation and aptitude for using such robots. Interactions with these robots are structured to resemble social situations in which certain social behaviors can occur and to simulate daily life. Considering that individuals with ASD have strong likes and dislikes, ensuring not only the optimal appearance but also the optimal motion of robots is important to achieve smooth interaction and to draw out the potential of robotic interventions. We investigated whether individuals with ASD found it easier to talk to an android robot with little motion (i.e., only opening and closing its mouth during speech) or an android robot with much motion (i.e., in addition to opening and closing its mouth during speech, moving its eyes from side to side and up and down, blinking, deeply breathing, and turning or moving its head or body at random). This was a crossover study in which a total of 25 participants with ASD experienced mock interviews conducted by an android robot with much spontaneous facial and bodily motion and an android robot with little motion. We compared demographic data between participants who answered that the android robot with much motion was easier to talk to than android robot with little motion and those who answered the opposite. In addition, we investigated how each type of demographic data was related to participants' feeling of comfort in an interview setting with an android robot. Fourteen participants indicated that the android robot with little motion was easier to talk to than the robot with much motion, whereas 11 participants answered the opposite. There were significant differences between these two groups in the sensory sensitivity score, which reflects the tendency to show a low neurological threshold. In addition, we found correlations between the sensation seeking score, which reflects the tendency to show a high neurological threshold, and self-report ratings of comfort in each condition. These results provide preliminary support for the importance of setting the motion of an android robot considering the sensory traits of ASD.

Expansion in speech time can restore comprehension in a simultaneously speaking bilingual robot.

Introduction: In this study, the development of a social robot, capable of giving speech simultaneously in more than one language was in mind. However, the negative effect of background noise on speech comprehension is well-documented in previous works. This deteriorating effect is more highlighted when the background noise has speech-like properties. Hence, the presence of speech as the background noise in a simultaneously speaking bilingual robot can be fatal for the speech comprehension of each person listening to the robot. Methods: To improve speech comprehension and consequently, user experience in the intended bilingual robot, the effect of time expansion on speech comprehension in a multi-talker speech scenario was investigated. Sentence recognition, speech comprehension, and subjective evaluation tasks were implemented in the study. Results: The obtained results suggest that a reduced speech rate, leading to an expansion in the speech time, in addition to increased pause duration in both the target and background speeches can lead to statistically significant improvement in both sentence recognition, and speech comprehension of participants. More interestingly, participants got a higher score in the time-expanded multi-talker speech than in the standard-speed single-talker speech in the speech comprehension and, in the sentence recognition task. However, this positive effect could not be attributed merely to the time expansion, as we could not repeat the same positive effect in a time-expanded single-talker speech. Discussion: The results obtained in this study suggest a facilitating effect of the presence of the background speech in a simultaneously speaking bilingual robot provided that both languages are presented in a time-expanded manner. The implications of such a simultaneously speaking robot are discussed.

Association of achieved blood pressure after treatment for primary aldosteronism with long-term kidney function.

Little is known regarding the association of blood pressure (BP) after treatment for primary aldosteronism (PA) (i.e., adrenalectomy and mineralocorticoid receptor antagonists) with long-term renal outcomes, and whether the association is independent of BP before treatment. Using a dataset from a nationwide registry of PA in Japan, we assessed whether achieved BP levels 6 months after treatment for PA are associated with annual changes in estimated glomerular filtration rate (eGFR), rapid eGFR decline, and incident chronic kidney disease (CKD) during the 5-year follow-up period. The cohort included 1266 PA patients. In multivariable linear regression including systolic BP (SBP) levels before treatment for PA, estimates (95% confidence interval [CI]) for annual changes in eGFR after month 6 associated with one-standard deviation (1-SD) higher SBP at month 6 were -0.08 (-0.15, -0.02) mL/min/1.73 m2/year. After multivariable adjustment, the estimate (95% CI) for annual changes in eGFR after month 6 was -0.12 (-0.21, -0.02) for SBP ≥ 130 mmHg vs. SBP < 130 mmHg at month 6. Among 537 participants without CKD at baseline, a 1-SD higher SBP was associated with a higher risk for incident CKD events (hazard ratio [95% CI]: 1.40 [1.00, 1.94]). Higher SBP after treatment for PA was associated with a higher risk for kidney dysfunction over time, independently of BP levels before treatment. Achieving SBP lower than 130 mmHg after treatment for PA may be linked to better kidney outcomes.

Brief Report: The Effectiveness of Hugging a Huggable Device Before Having a Conversation with an Unfamiliar Person for Autism Spectrum Disorders.

Sensory overresponsivity (SOR) emerges before anxiety and positively predicts subsequent increasing levels of anxiety in individuals with autism spectrum disorder (ASD). Sensory seeking behavior occurs as compensation for SOR, and individuals may seek sensory input in one sensory domain to compensate for SOR. Tactile seeking behavior is sufficient to decrease social anxiety in communicating with unfamiliar people. We assessed the effectiveness of hugging a huggable device before a conversation for reducing the psychological stress associated with speaking to an unfamiliar person or robot. Our analysis showed a significant effect, with Hugvie contributing to decreased stress for both interlocutors. Thus, this study demonstrated the efficacy of hugging it before conversation, which emphasizes the importance of tactile seeking for individuals with ASD.