Attempt to extract features and classify subjective poor physical conditions in facial images using deep metric learning.

With the spread of COVID-19, the need for remote detection of physical conditions is increasing, for example, there are several situations wherein the body temperature has to be measured remotely to detect febrile individuals. Aiming to remotely detect physical conditions, the study attempted to investigate anomaly detection based on facial color and skin temperature, which are indicators related to hemodynamics. Triplet loss was used to extract features related to subjective health feelings from facial images to evaluate whether there is a relationship between subjective health feelings and facial images. A classification of subjective health feelings related to poor physical conditions based on these features was also attempted. To obtain the data, an experiment was conducted for approximately 1 year to measure facial visual and thermal images, and subjective feelings related to physical conditions. Anomaly levels were defined based on subjective health feelings. Anomaly detection models were constructed by classifying anomaly and normal data based on subjective health feelings. Facial visible and thermal images were applied to the trained model to quantitatively evaluate the accuracy of the classification of anomaly conditions related to subjective health. At higher levels of anomaly, a combination of facial visible and thermal images resulted in the classification of subjective health feelings with moderate accuracy. Further, the results suggest that the eyes and sides of the nose may indicate subjective health feelings.

Laparoscopic surgical skill evaluation with motion capture and eyeglass gaze cameras: A pilot study.

INTRODUCTION: An eyeglass gaze camera and a skeletal coordinate camera without sensors attached to the operator's body were used to monitor gaze and movement during a simulated surgical procedure. These new devices have the potential to change skill assessment for laparoscopic surgery. The suitability of these devices for skill assessment was investigated. MATERIAL AND METHODS: Six medical students, six intermediate surgeons, and four experts performed suturing tasks in a dry box. The tip positions of the instruments were identified from video recordings. Performance was evaluated based on instrument movement, gaze, and skeletal coordination. RESULTS: Task performance time and skeletal coordinates were not significantly different among skill levels. The total movement distance of the right instrument was significantly different depending on the skill level. The SD of the gaze coordinates was significantly different depending on skill level and was less for experts. The expert's gaze stayed in a small area with little blurring. CONCLUSIONS: The SD of gaze point coordinates correlates with laparoscopic surgical skill level. These devices may facilitate objective intraoperative skill evaluation in future studies.

An attempt to construct the individual model of daily facial skin temperature using variational autoencoder.

Facial skin temperature (FST) has also gained prominence as an indicator for detecting anomalies such as fever due to the COVID-19. When FST is used for engineering applications, it is enough to be able to recognize normal. We are also focusing on research to detect some anomaly in FST. In a previous study, it was confirmed that abnormal and normal conditions could be separated based on FST by using a variational autoencoder (VAE), a deep generative model. However, the simulations so far have been a far cry from reality. In this study, normal FST with a diurnal variation component was defined as a normal state, and a model of normal FST in daily life was individually reconstructed using VAE. Using the constructed model, the anomaly detection performance was evaluated by applying the Hotelling theory. As a result, the area under the curve (AUC) value in ROC analysis was confirmed to be 0.89 to 1.00 in two subjects.

Years of experience is more effective in defining experts in the gaze analysis of laparoscopic suturing task than task duration.

In this study, the relationship between gaze patterns, task duration, and years of experience, which are commonly used to define and evaluate expert surgeons in laparoscopic surgery, was identified. Ten surgeons with 1-28 years of experience and six inexperienced students were included. Subjects used forceps to repeat the task of suturing a suture pad. Each subject wore an eye-marking recorder, and gaze points were recorded and analyzed. No significant relationship between task duration and gaze pattern was observed. However, there was a significant relationship between a surgeon's years of experience and the percentage of time spent gazing at the forceps. Subjects with more years of experience operated without looking at the forceps and fixed their gaze on the operational target. Therefore, when analyzing laparoscopic gazing patterns, it may be more appropriate to define an "expert" based on the years of experience rather than task duration.

Face Alignment in Thermal Infrared Images Using Cascaded Shape Regression.

The evaluation of physiological and psychological states using thermal infrared images is based on the skin temperature of specific regions of interest, such as the nose, mouth, and cheeks. To extract the skin temperature of the region of interest, face alignment in thermal infrared images is necessary. To date, the Active Appearance Model (AAM) has been used for face alignment in thermal infrared images. However, computation using this method is costly, and it has a low real-time performance. Conversely, face alignment of visible images using Cascaded Shape Regression (CSR) has been reported to have high real-time performance. However, no studies have been reported on face alignment in thermal infrared images using CSR. Therefore, the objective of this study was to verify the speed and robustness of face alignment in thermal infrared images using CSR. The results suggest that face alignment using CSR is more robust and computationally faster than AAM.