Attempt to extract features and classify subjective poor physical conditions in facial images using deep metric learning.

With the spread of COVID-19, the need for remote detection of physical conditions is increasing, for example, there are several situations wherein the body temperature has to be measured remotely to detect febrile individuals. Aiming to remotely detect physical conditions, the study attempted to investigate anomaly detection based on facial color and skin temperature, which are indicators related to hemodynamics. Triplet loss was used to extract features related to subjective health feelings from facial images to evaluate whether there is a relationship between subjective health feelings and facial images. A classification of subjective health feelings related to poor physical conditions based on these features was also attempted. To obtain the data, an experiment was conducted for approximately 1 year to measure facial visual and thermal images, and subjective feelings related to physical conditions. Anomaly levels were defined based on subjective health feelings. Anomaly detection models were constructed by classifying anomaly and normal data based on subjective health feelings. Facial visible and thermal images were applied to the trained model to quantitatively evaluate the accuracy of the classification of anomaly conditions related to subjective health. At higher levels of anomaly, a combination of facial visible and thermal images resulted in the classification of subjective health feelings with moderate accuracy. Further, the results suggest that the eyes and sides of the nose may indicate subjective health feelings.

An attempt to construct the individual model of daily facial skin temperature using variational autoencoder.

Facial skin temperature (FST) has also gained prominence as an indicator for detecting anomalies such as fever due to the COVID-19. When FST is used for engineering applications, it is enough to be able to recognize normal. We are also focusing on research to detect some anomaly in FST. In a previous study, it was confirmed that abnormal and normal conditions could be separated based on FST by using a variational autoencoder (VAE), a deep generative model. However, the simulations so far have been a far cry from reality. In this study, normal FST with a diurnal variation component was defined as a normal state, and a model of normal FST in daily life was individually reconstructed using VAE. Using the constructed model, the anomaly detection performance was evaluated by applying the Hotelling theory. As a result, the area under the curve (AUC) value in ROC analysis was confirmed to be 0.89 to 1.00 in two subjects.

Face Alignment in Thermal Infrared Images Using Cascaded Shape Regression.

The evaluation of physiological and psychological states using thermal infrared images is based on the skin temperature of specific regions of interest, such as the nose, mouth, and cheeks. To extract the skin temperature of the region of interest, face alignment in thermal infrared images is necessary. To date, the Active Appearance Model (AAM) has been used for face alignment in thermal infrared images. However, computation using this method is costly, and it has a low real-time performance. Conversely, face alignment of visible images using Cascaded Shape Regression (CSR) has been reported to have high real-time performance. However, no studies have been reported on face alignment in thermal infrared images using CSR. Therefore, the objective of this study was to verify the speed and robustness of face alignment in thermal infrared images using CSR. The results suggest that face alignment using CSR is more robust and computationally faster than AAM.