C3I-SynFace: A synthetic head pose and facial depth dataset using seed virtual human models.

This article presents C3I-SynFace: a large-scale synthetic human face dataset with corresponding ground truth annotations of head pose and face depth generated using the iClone 7 Character Creator "Realistic Human 100" toolkit with variations in ethnicity, gender, race, age, and clothing. The data is generated from 15 female and 15 male synthetic 3D human models extracted from iClone software in FBX format. Five facial expressions - neutral, angry, sad, happy, and scared are added to the face models to add further variations. With the help of these models, an open-source data generation pipeline in Python is proposed to import these models into the 3D computer graphics tool Blender and render the facial images along with the ground truth annotations of head pose and face depth in raw format. The datasets contain more than 100k ground truth samples with their annotations. With the help of virtual human models, the proposed framework can generate extensive synthetic facial datasets (e.g., head pose or face depths datasets) with a high degree of control over facial and environmental variations such as pose, illumination, and background. Such large datasets can be used for the improved and targeted training of deep neural networks.

3D face-model reconstruction from a single image: A feature aggregation approach using hierarchical transformer with weak supervision.

Convolutional Neural Networks (CNN) have gained popularity as the de-facto model for any computer vision task. However, CNN have drawbacks, i.e. they fail to extract long-range perceptions in images. Due to their ability to capture long-range dependencies, transformer networks are adopted in computer vision applications, where they show state-of-the-art (SOTA) results in popular tasks like image classification, instance segmentation, and object detection. Although they gained ample attention, transformers have not been applied to 3D face reconstruction tasks. In this work, we propose a novel hierarchical transformer model, added to a feature pyramid aggregation structure, to extract the 3D face parameters from a single 2D image. More specifically, we use pre-trained Swin Transformer backbone networks in a hierarchical manner and add the feature fusion module to aggregate the features in multiple stages. We use a semi-supervised training approach and train our model in a supervised way with the 3DMM parameters from a publicly available dataset and unsupervised training with a differential renderer on other parameters like facial keypoints and facial features. We also train our network on a hybrid unsupervised loss and compare the results with other SOTA approaches. When evaluated across two public datasets on face reconstruction and dense 3D face alignment tasks, our method can achieve comparable results to the current SOTA performance and in some instances do better than the SOTA methods. A detailed subjective evaluation also shows that our method performs better than the previous works in realism and occlusion resistance.

Perceiving emotion in crowds: the role of dynamic body postures on the perception of emotion in crowded scenes.

Although the perception of emotion in individuals is an important social skill, very little is known about how emotion is determined from a crowd of individuals. We investigated the perception of emotion in scenes of crowds populated by dynamic characters each expressing an emotion. Facial expressions were masked in these characters and emotion was conveyed using body motion and posture only. We systematically varied the proportion of characters in each scene depicting one of two emotions and participants were required to categorise the overall emotion of the crowd. In Experiment 1, we found that the perception of emotions in a crowd is efficient even with relatively brief exposures of the crowd stimuli. Furthermore, the emotion of a crowd was generally determined by the relative proportions of characters conveying it, although we also found that some emotions dominated perception. In Experiment 2, we found that an increase in crowd size was not associated with a relative decrease in the efficiency with which the emotion was categorised. Our findings suggest that body motion is an important social cue in perceiving the emotion of crowds and have implications for our understanding of how we perceive social information from groups.

The Effect of Realistic Appearance of Virtual Characters in Immersive Environments - Does the Character's Personality Play a Role?

Virtual characters that appear almost photo-realistic have been shown to induce negative responses from viewers in traditional media, such as film and video games. This effect, described as the uncanny valley, is the reason why realism is often avoided when the aim is to create an appealing virtual character. In Virtual Reality, there have been few attempts to investigate this phenomenon and the implications of rendering virtual characters with high levels of realism on user enjoyment. In this paper, we conducted a large-scale experiment on over one thousand members of the public in order to gather information on how virtual characters are perceived in interactive virtual reality games. We were particularly interested in whether different render styles (realistic, cartoon, etc.) would directly influence appeal, or if a character's personality was the most important indicator of appeal. We used a number of perceptual metrics such as subjective ratings, proximity, and attribution bias in order to test our hypothesis. Our main result shows that affinity towards virtual characters is a complex interaction between the character's appearance and personality, and that realism is in fact a positive choice for virtual characters in virtual reality.

Data-Driven Approach to Synthesizing Facial Animation Using Motion Capture.

Producing cartoon animations is a laborious task, and there is a distinct lack of automatic tools to help animators, particularly with creating facial animation. The proposed method uses real-time video-based motion tracking to generate facial motion as input and then matches it to existing hand-created animation curves. The synthesized animations can then be refined and polished by an animator, saving considerable time in overall production.