Effective Emoticon Suggestion Technique Based on Active Emotional Input Using Facial Expressions and Heart Rate Signals.

The evolution of mobile communication technology has brought about significant changes in the way people communicate. However, the lack of nonverbal cues in computer-mediated communication can make the accurate interpretation of emotions difficult. This study proposes a novel approach for using emotions as active input in mobile systems. This approach combines psychological and neuroscientific principles to accurately and comprehensively assess an individual's emotions for use as input in mobile systems. The proposed technique combines facial and heart rate information to recognize users' five prime emotions, which can be implemented on mobile devices using a front camera and a heart rate sensor. A user evaluation was conducted to verify the efficacy and feasibility of the proposed technique, and the results showed that users could express emotions faster and more accurately, with average recognition accuracies of 90% and 82% for induced and intended emotional expression, respectively. The proposed technique has the potential to enhance the user experience and provide more personalized and dynamic interaction with mobile systems.

Emojis vs. facial expressions: An electrical neuroimaging study on perceptual recognition.

The aim of this study was to investigate the neural underpinnings and the time course of emoji recognition through the recording of event-related potentials in 51 participants engaged in a categorization task involving an emotional word paradigm. Forty-eight happy, sad, surprised, disgusted, fearful, angry emojis, and as many facial expressions, were used as stimuli. Behavioral data showed that emojis were recognized faster and more accurately (92.7%) than facial expressions displaying the same emotions (87.35%). Participants were better at recognizing happy, disgusted, and sad emojis, and happy and angry faces. Fear was difficult to recognize in both faces and emojis. The N400 response was larger to incongruently primed emojis and faces, while the opposite was observed for the P300 component. However, both N400 and P300 were considerably later in response to faces than emojis. The emoji-related N170 component (150-190 ms) discriminated stimulus affective content, similar to face-related N170, but its neural generators did not include the face fusiform area but the occipital face area (OFA) for processing face details, and object-related areas. Both faces and emojis activated the limbic system and the orbitofrontal cortex supporting anthropomorphization. The schematic nature of emojis might determine an easier classification of their emotional content.

Emojis Are Comprehended Better than Facial Expressions, by Male Participants.

Emojis are colorful ideograms resembling stylized faces commonly used for expressing emotions in instant messaging, on social network sites, and in email communication. Notwithstanding their increasing and pervasive use in electronic communication, they are not much investigated in terms of their psychological properties and communicative efficacy. Here, we presented 112 different human facial expressions and emojis (expressing neutrality, happiness, surprise, sadness, anger, fear, and disgust) to a group of 96 female and male university students engaged in the recognition of their emotional meaning. Analyses of variance showed that male participants were significantly better than female participants at recognizing emojis (especially negative ones) while the latter were better than male participants at recognizing human facial expressions. Quite interestingly, male participants were better at recognizing emojis than human facial expressions per se. These findings are in line with more recent evidence suggesting that male individuals may be more competent and inclined to use emojis to express their emotions in messaging (especially sarcasm, teasing, and love) than previously thought. Finally, the data indicate that emojis are less ambiguous than facial expressions (except for neutral and surprise emotions), possibly because of the limited number of fine-grained details and the lack of morphological features conveying facial identity.

Sentiment Classification Method Based on Blending of Emoticons and Short Texts.

With the development of Internet technology, short texts have gradually become the main medium for people to obtain information and communicate. Short text reduces the threshold of information production and reading by virtue of its short length, which is in line with the trend of fragmented reading in the context of the current fast-paced life. In addition, short texts contain emojis to make the communication immersive. However, short-text content means it contains relatively little information, which is not conducive to the analysis of sentiment characteristics. Therefore, this paper proposes a sentiment classification method based on the blending of emoticons and short-text content. Emoticons and short-text content are transformed into vectors, and the corresponding word vector and emoticon vector are connected into a sentencing matrix in turn. The sentence matrix is input into a convolution neural network classification model for classification. The results indicate that, compared with existing methods, the proposed method improves the accuracy of analysis.

Emoji as Affective Symbols: Affective Judgments of Emoji, Emoticons, and Human Faces Varying in Emotional Content.

An important function of emoji as communicative symbols is to convey emotional content from sender to receiver in computer-mediated communication, e. g., WhatsApp. However, compared with real faces, pictures or words, many emoji are ambiguous because they do not symbolize a discrete emotion or feeling state. Thus, their meaning relies on the context of the message in which they are embedded. Previous studies investigated affective judgments of pictures, faces, and words suggesting that these stimuli show a typical distribution along the big two emotion dimensions of valence and arousal. Also, emoji and emoticons have been investigated recently for their affective significance. The present study extends previous research by investigating affective ratings of emoji, emoticons and human faces and by direct comparison between them. In total, 60 stimuli have been rated by 83 participants (eight males, age: 18-49 years), using the non-verbal Self-Assessment Manikin Scales for valence and arousal. The emotionality of the stimuli was measured on a 9-point Likert scale. The results show significant main effects of the factors "stimulus category" and "discrete emotion" including emotionality, valence and arousal. Also, the interaction between these two main factors was significant. Emoji elicited highest arousal, whereas stimuli related to happiness were rated highest in valence across stimulus categories. Angry emoji were rated highest in emotionality. Also, the discrete emotion was best recognized in emoji, followed by human face stimuli and lastly emoticons.

Emojis in public health and how they might be used for hand hygiene and infection prevention and control.

Emojis are frequently used picture characters known as possible surrogates for non-verbal aspects of behavior. Considering the ability of emojis to enhance and facilitate communication, there has been a growing interest in studying their effects in scientific and health-related topics over the past few years. Infection prevention and control (IPC) is a field of medicine that is directly associated with specific behaviors. These include hand hygiene, which is the cornerstone of the prevention of healthcare-associated infections, and essential in stemming the spread of antimicrobial resistance. This paper aims to provide an overview of how emojis have been used in the medical and public health literature and proposes their possible use in IPC and hand hygiene to put forth a vision for the future research.

Influence of Color on Emotion Recognition Is Not Bidirectional: An Investigation of the Association Between Color and Emotion Using a Stroop-Like Task.

The association between color and emotion has been shown, with red facilitating recognition of anger and green facilitating recognition of happiness. However, it has been unclear if emotional stimulus conversely facilitates and/or inhibits recognition of such colors. This study used a Stroop-like task, which required participants to ignore facial expressions and recognize color, in order to investigate the influence of emotion on recognition of color. In addition, this study investigated the association between color and emotion recognition from emoticons, as it was recently suggested that the process of emotion recognition from emoticons was different from that of actual faces. Results revealed that for facial expressions and emoticons, color influenced emotion recognition, in line with previous studies. Conversely, facial expression did not influence recognition of color. The results suggest that in emotion recognition people consider surrounding contextual information and integrate it automatically; however, in color recognition, they do not.

Emotion sharing in remote patient monitoring of patients with chronic kidney disease.

OBJECTIVE: To investigate the relationship between emotion sharing and technically troubled dialysis (TTD) in a remote patient monitoring (RPM) setting. MATERIALS AND METHODS: A custom software system was developed for home hemodialysis patients to use in an RPM setting, with focus on emoticon sharing and sentiment analysis of patients' text data. We analyzed the outcome of emoticon and sentiment against TTD. Logistic regression was used to assess the relationship between patients' emotions (emoticon and sentiment) and TTD. RESULTS: Usage data were collected from January 1, 2015 to June 1, 2018 from 156 patients that actively used the app system, with a total of 31 159 dialysis sessions recorded. Overall, 122 patients (78%) made use of the emoticon feature while 146 patients (94%) wrote at least 1 or more session notes for sentiment analysis. In total, 4087 (13%) sessions were classified as TTD. In the multivariate model, when compared to sessions with self-reported very happy emoticons, those with sad emoticons showed significantly higher associations to TTD (aOR 4.97; 95% CI 4.13-5.99; P = < .001). Similarly, negative sentiments also revealed significant associations to TTD (aOR 1.56; 95% CI 1.22-2; P = .003) when compared to positive sentiments. DISCUSSION: The distribution of emoticons varied greatly when compared to sentiment analysis outcomes due to the differences in the design features. The emoticon feature was generally easier to understand and quicker to input while the sentiment analysis required patients to manually input their personal thoughts. CONCLUSION: Patients on home hemodialysis actively expressed their emotions during RPM. Negative emotions were found to have significant associations with TTD. The use of emoticons and sentimental analysis may be used as a predictive indicator for prolonged TTD.

Mood Detection in Ambiguous Messages: The Interaction Between Text and Emoticons.

Face-to-face communication has several sources of contextual information that enables language comprehension. This information is used, for instance, to perceive mood of interlocutors, clarifying ambiguous messages. However, these contextual cues are absent in text-based communication. Emoticons have been proposed as cues used to stress the emotional intentions on this channel of communication. Most studies have suggested that their role is to contribute to a more accurate perception of emotions. Nevertheless, it is not clear if their influence on disambiguation is independent of their emotional valence and its interaction with text message valence. In the present study, we designed an emotional congruence paradigm, where participants read a set of messages composed by a positive or negative emotional situation sentence followed by a positive or negative emoticon. Participants were instructed to indicate if the sender was in a good or bad mood. With the aim of analyzing the disambiguation process and observing if the role of the emoticons in disambiguation is different according their valence, we measure the rate of responses of perceived mood and the reaction times (RTs) for each condition. Our results showed that the perceived mood in ambiguous messages tends to be more negative regardless of emotion valence. Nonetheless, we observed that this tendency was not the same for positive and negative emoticons. Specifically, negative mood perception was higher for incongruent positive emoticons. On the other hand, RTs for positive emoticons were faster than for the negative ones. Responses for incongruent messages were slower than for the congruent ones. However, the incongruent condition showed different RTs depending on the emoticons' valence. In the incongruent condition, responses for negative emoticons was the slowest. Results are discussed taking into account previous observations about the potential role of emoticons in mood perception and cognitive processing. We concluded that the role of emoticons in disambiguation and mood perception is due to the interaction of emoticon valence with the entire message.

Seven- to 11-Year-Olds' Developing Ability to Recognize Natural Facial Expressions of Basic Emotions.

Being able to recognize facial expressions of basic emotions is of great importance to social development. However, we still know surprisingly little about children's developing ability to interpret emotions that are expressed dynamically, naturally, and subtly, despite real-life expressions having such appearance in the vast majority of cases. The current research employs a new technique of capturing dynamic, subtly expressed natural emotional displays (happy, sad, angry, shocked, and disgusted). Children aged 7, 9, and 11 years (and adults) were systematically able to discriminate each emotional display from alternatives in a five-way choice. Children were most accurate in identifying the expression of happiness and were also relatively accurate in identifying the expression of sadness; they were far less accurate than adults in identifying shocked and disgusted. Children who performed well academically also tended to be the most accurate in recognizing expressions, and this relationship maintained independently of chronological age. Generally, the findings testify to a well-developed ability to recognize very subtle naturally occurring expressions of emotions.

Simple lines and shapes are associated with, and communicate, distinct emotions.

We investigated whether lines and shapes that present face-like features would be associated with emotions. In Experiment 1, participants associated concave, convex, or straight lines with the words happy or sad. Participants found it easiest to associate the concave line with happy and the convex line with sad. In Experiment 2, participants rated (valence, pleasantness, liking, and tension) and categorised (valence and emotion words) two convex and concave lines that were paired with six distinct pairs of eyes. The presence of eyes affected participants' valence ratings and response latencies; more congruent eye-mouth matches produced more consistent ratings and faster reaction times. In Experiment 3, we examined whether dots that resembled eyes would be associated with emotional words. Participants found it easier to match certain sets of dots with specific emotions. These results suggest that facial gestures that are associated with specific emotions can be captured using relatively simple shapes and lines.

Neural correlates of text-based emoticons: a preliminary fMRI study.

INTRODUCTION: Like nonverbal cues in oral interactions, text-based emoticons, which are textual portrayals of a writer's facial expressions, are commonly used in electronic device-mediated communication. Little is known, however, about how text-based emoticons are processed in the human brain. With this study, we investigated whether the text-based emoticons are processed as face expressions using fMRI. METHODS: During fMRI scan, subjects were asked to respond by pressing a button, indicating whether text-based emoticons represented positive or negative emotions. Voxel-wise analyses were performed to compare the responses and contrasted with emotional versus scrambled emoticons and among emoticons with different emotions. To explore processing strategies for text-based emoticons, brain activity in the bilateral occipital and fusiform face areas were compared. RESULTS: In the voxel-wise analysis, both emotional and scrambled emoticons were processed mainly in the bilateral fusiform gyri, inferior division of lateral occipital cortex, inferior frontal gyri, dorsolateral prefrontal cortex (DLPFC), dorsal anterior cingulate cortex (dACC), and parietal cortex. In a percent signal change analysis, the right occipital and fusiform face areas showed significantly higher activation than left ones. In comparisons among emoticons, sad one showed significant BOLD signal decrease in the dACC, the left AIC, the bilateral thalamus, and the precuneus as compared with other conditions. CONCLUSION: The results of this study imply that people recognize text-based emoticons as pictures representing face expressions. Even though text-based emoticons contain emotional meaning, they are not associated with the amygdala while previous studies using emotional stimuli documented amygdala activation.

Face and Emoticon: Behavioral Difference and Gender Effect / 대한정신약물학회지

OBJECTIVE: Emoticon as well as face is a tool for the communication of emotion. However, little is known about behavioral response to emoticon, unlike face. To explore the characteristics of behavioral response of emoticon and face, we measured both response time and accuracy in healthy young subjects. METHODS: The 29 subjects were asked to respond to emoticons or faces which contained one among happy, sad, angry/fearful or neutral emotion. Using univariate analysis of variance, behavioral responses were analyzed for three main effects of stimulation (face, emoticon), emotion (happy, sad, angry/fearful, neutral), gender (male, female) and also their interactions. RESULTS: The response to face was faster and more accurate than that to emoticon. Female's response to face, not to emoticon, was faster than male. A common finding of face and emoticon stimuli was that their responses were slower and less accurate in angry/fearful condition than in the rest emotional ones. There was not any interaction among three factors. The different finding was that neutral condition was less informative in only emoticon, not in face, condition. CONCLUSION: Our results demonstrated emoticon is different from face in terms of the characteristics in the transfer ability of emotion and in gender effect.