Quantitative Evaluation of Facial Expression in a Patient With Minimally Conscious State After Severe Traumatic Brain Injury.

OBJECTIVE: To investigate whether automatic facial expression analysis can quantify differences in the intensity of facial responses depending on the affective stimuli in a patient with minimally conscious state (MCS). METHODS: We filmed the facial responses of a patient with MCS during the delivery of three 1-minute auditory stimuli: audio clips of comedy movies, a nurse hilariously talking, and recitation of a novel (comedy, nurse, and recitation conditions, respectively). These measures were repeated at least 13 times for each condition on different days for approximately 10 months. The intensity of being "happy" was estimated from the smiling face using a software called FaceReader. The intensity among 5 conditions including those at 2 resting conditions (pre- and poststimuli) was compared using the Kruskal-Wallis test and the Dunn-Bonferroni test for multiple comparisons. RESULTS: Significantly higher values were found in the intensity of being "happy" in the comedy and nurse conditions versus other conditions, with no significant differences between the recitation and pre- or poststimulus conditions. These findings indicate that the automated facial expression analysis can quantify differences in context-dependent facial responses in the patient recruited in this study. CONCLUSIONS: This case study demonstrates the feasibility of using automated facial expression analysis to quantitatively evaluate the differences in facial expressions and their corresponding emotions in a single patient with MCS.

Functional inter-cortical connectivity among motor-related cortices during motor imagery: A magnetoencephalographic study.

Neural connectivity was measured during motor imagery (MI) and motor execution (ME) using magnetoencephalography in nine healthy subjects, MI, and at rest. Lower coherence values during ME and MI between sensorimotor areas than at rest, and lower values during MI between the left supplementary motor area and inferior frontal gyrus than ME suggested the sensorimotor network of MI functioned with similar connectivity to ME and that the inhibitory activity functioned continuously during MI, respectively.

The other person's smiling amount affects one's smiling response during face-to-face conversations.

Introduction: Smiling during conversation occurs interactively between people and is known to build good interpersonal relationships. However, whether and how much the amount that an individual smiles is influenced by the other person's smile has remained unclear. This study aimed to quantify the amount of two individuals' smiles during conversations and investigate the dependency of one's smile amount (i.e., intensity and frequency) on that of the other. Method: Forty participants (20 females) engaged in three-minute face-to-face conversations as speakers with a listener (male or female), under three conditions, where the amount of smiling response by listeners was controlled as "less," "moderate," and "greater." The amount of the smiles was quantified based on their facial movements through automated facial expression analysis. Results: The results showed that the amount of smiling by the speaker changed significantly depending on the listener's smile amount; when the listeners smiled to a greater extent, the speakers tended to smile more, especially when they were of the same gender (i.e., male-male and female-female pairs). Further analysis revealed that the smiling intensities of the two individuals changed in a temporally synchronized manner. Discussion: These results provide quantitative evidence for the dependence of one's smile on the other's smile, and the differential effect between gender pairs.

Positive Emotional Responses to Socially Assistive Robots in People With Dementia: Pilot Study.

Background: Interventions and care that can evoke positive emotions and reduce apathy or agitation are important for people with dementia. In recent years, socially assistive robots used for better dementia care have been found to be feasible. However, the immediate responses of people with dementia when they are given multiple sensory modalities from socially assistive robots have not yet been sufficiently elucidated. Objective: This study aimed to quantitatively examine the immediate emotional responses of people with dementia to stimuli presented by socially assistive robots using facial expression analysis in order to determine whether they elicited positive emotions. Methods: This pilot study adopted a single-arm interventional design. Socially assistive robots were presented to nursing home residents in a three-step procedure: (1) the robot was placed in front of participants (visual stimulus), (2) the robot was manipulated to produce sound (visual and auditory stimuli), and (3) participants held the robot in their hands (visual, auditory, and tactile stimuli). Expression intensity values for "happy," "sad," "angry," "surprised," "scared," and "disgusted" were calculated continuously using facial expression analysis with FaceReader. Additionally, self-reported feelings were assessed using a 5-point Likert scale. In addition to the comparison between the subjective and objective emotional assessments, expression intensity values were compared across the aforementioned 3 stimuli patterns within each session. Finally, the expression intensity value for "happy" was compared between the different types of robots. Results: A total of 29 participants (mean age 88.7, SD 6.2 years; n=27 female; Japanese version of Mini-Mental State Examination mean score 18.2, SD 5.1) were recruited. The expression intensity value for "happy" was the largest in both the subjective and objective assessments and increased significantly when all sensory modalities (visual, auditory, and tactile) were presented (median expression intensity 0.21, IQR 0.09-0.35) compared to the other 2 patterns (visual alone: median expression intensity 0.10, IQR 0.03-0.22; P<.001; visual and auditory: median expression intensity 0.10, IQR 0.04-0.23; P<.001). The comparison of different types of robots revealed a significant increase when all stimuli were presented by doll-type and animal-type robots, but not humanoid-type robots. Conclusions: By quantifying the emotional responses of people with dementia, this study highlighted that socially assistive robots may be more effective in eliciting positive emotions when multiple sensory stimuli, including tactile stimuli, are involved. More studies, including randomized controlled trials, are required to further explore the effectiveness of using socially assistive robots in dementia care.