CalibrationPhys: Self-Supervised Video-Based Heart and Respiratory Rate Measurements by Calibrating Between Multiple Cameras.

Video-based heart and respiratory rate measurements using facial videos are more useful and user-friendly than traditional contact-based sensors. However, most of the current deep learning approaches require ground-truth pulse and respiratory waves for model training, which are expensive to collect. In this paper, we propose CalibrationPhys, a self-supervised video-based heart and respiratory rate measurement method that calibrates between multiple cameras. CalibrationPhys trains deep learning models without supervised labels by using facial videos captured simultaneously by multiple cameras. Contrastive learning is performed so that the pulse and respiratory waves predicted from the synchronized videos using multiple cameras are positive and those from different videos are negative. CalibrationPhys also improves the robustness of the models by means of a data augmentation technique and successfully leverages a pre-trained model for a particular camera. Experimental results utilizing two datasets demonstrate that CalibrationPhys outperforms state-of-the-art heart and respiratory rate measurement methods. Since we optimize camera-specific models using only videos from multiple cameras, our approach makes it easy to use arbitrary cameras for heart and respiratory rate measurements.

Evaluation of Correlation between Psychomotor Vigilance Task Scores and Drowsiness Estimation Levels Obtained from Facial Videos.

This paper reports the results of an experiment to evaluate the relationship between results obtained with a drowsiness estimation system we have developed using facial videos and those obtained with the Psychomotor Vigilance Task (PVT), which is a standard index of sleepiness used in sleep medicine. The correlation between PVT scores and the output of the drowsiness estimation system, which outputs drowsiness levels from assigned facial expressions, was calculated using data from 30 subjects. The Spearman's correlation coefficients between the drowsiness estimation results and the PVT mean response time, the slowest 10% response time, and the number of lapses were 0.36 (p <0.001), 0.43 (p <0.001), and 0.40 (p <0.001), respectively. Since this experiment showed a correlation between the drowsiness estimation results and those with PVT, it would seem possible to make specific interventions based on drowsiness estimation results learned from ground-truth drowsiness levels. Such estimation results could help prevent accidents resulting from drowsiness or insufficient vigilance while driving or working.

Forecasting stress, mood, and health from daytime physiology in office workers and students.

We examine the problem of forecasting tomorrow morning's three self-reported levels (on scales from 0 to 100) of stressed-calm, sad-happy, and sick-healthy based on physiological data (skin conductance, skin temperature, and acceleration) from a sensor worn on the wrist from 10am-5pm today. We train automated forecasting regression algorithms using Random Forests and compare their performance over two sets of data: "workers" consisting of 490 days of weekday data from 39 employees at a high-tech company in Japan and "students" consisting of 3,841 days of weekday data from 201 New England USA college students. Mean absolute errors on held-out test data achieved 10.8, 13.5, and 14.4 for the estimated levels of mood, stress, and health respectively of office workers, and 17.8, 20.3, and 20.4 for the mood, stress, and health respectively of students. Overall the two groups reported comparable stress and mood scores, while employees reported slightly poorer health, and engaged in significantly lower levels of physical activity as measured by accelerometers. We further examine differences in population features and how systems trained on each population performed when tested on the other.

Daytime Data and LSTM can Forecast Tomorrow's Stress, Health, and Happiness.

Accurately forecasting well-being may enable people to make desirable behavioral changes that could improve their future well-being. In this paper, we evaluate how well an automated model can forecast the next-day's well-being (specifically focusing on stress, health, and happiness) from static models (support vector machine and logistic regression) and time-series models (long short-term memory neural network models (LSTM)) using the previous seven days of physiological, mobile phone, and behavioral survey data. We especially examine how using only a portion of the day's data (e.g. just night-time, or just daytime) influences the forecasting accuracy. The results show that accuracy is improved, across every condition tested, by using an LSTM instead of using static models. We find that daytime-only physiology data from wearable sensors, using an LSTM, can provide an accurate forecast of tomorrow's well-being using students' daily life data (stress: 80.4%, health: 86.0%, and happiness: 79.1%), achieving the same accuracy as using data collected from around the clock. These findings are valuable steps toward developing a practical and convenient well-being forecasting system.

Head-motion Robust Video-based Heart Rate Estimation Using Facial Feature Point Fluctuations.

Interest in measuring heart rates (HRs) without physical contact has increased in the area of stress checking and health care. In this paper, we propose head-motion robust video-based heart rate estimation using facial feature point fluctuations. The proposed method adaptively estimates and removes such rigid-noise components as noise stemming from horizontal head motion and extracts relatively small heart signals. Rigid-noise components can be accurately estimated and removed by using changes in facial feature points which are not dominant over heart signals and are more dominant over noise signals than are such luminance signals as RGB. In evaluation experiments on a benchmark dataset, our method achieved the highest accuracy among state-of-the-art methods.