Ultra-short-term stress measurement using RGB camera-based remote photoplethysmography with reduced effects of Individual differences in heart rate.

Stress is linked to health problems, increasing the need for immediate monitoring. Traditional methods like electrocardiograms or contact photoplethysmography require device attachment, causing discomfort, and ultra-short-term stress measurement research remains inadequate. This paper proposes a method for ultra-short-term stress monitoring using remote photoplethysmography (rPPG). Previous predictions of ultra-short-term stress have typically used pulse rate variability (PRV) features derived from time-segmented heart rate data. However, PRV varies at the same stress levels depending on heart rates, necessitating a new method to account for these differences. This study addressed this by segmenting rPPG data based on normal-to-normal intervals (NNIs), converted from peak-to-peak intervals, to predict ultra-short-term stress indices. We used NNI counts corresponding to average durations of 10, 20, and 30 s (13, 26, and 39 NNIs) to extract PRV features, predicting the Baevsky stress index through regressors. The Extra Trees Regressor achieved R2 scores of 0.6699 for 13 NNIs, 0.8751 for 26 NNIs, and 0.9358 for 39 NNIs, surpassing the time-segmented approach, which yielded 0.4162, 0.6528, and 0.7943 for 10, 20, and 30-s intervals, respectively. These findings demonstrate that using NNI counts for ultra-short-term stress prediction improves accuracy by accounting for individual bio-signal variations.

Machine learning-based classification of Parkinson's disease using acoustic features: Insights from multilingual speech tasks.

This study advances the automation of Parkinson's disease (PD) diagnosis by analyzing speech characteristics, leveraging a comprehensive approach that integrates a voting-based machine learning model. Given the growing prevalence of PD, especially among the elderly population, continuous and efficient diagnosis is of paramount importance. Conventional monitoring methods suffer from limitations related to time, cost, and accessibility, underscoring the need for the development of automated diagnostic tools. In this paper, we present a robust model for classifying speech patterns in Korean PD patients, addressing a significant research gap. Our model employs straightforward preprocessing techniques and a voting-based machine learning approach, demonstrating superior performance, particularly when training data is limited. Furthermore, we emphasize the effectiveness of the eGeMAPSv2 feature set in PD analysis and introduce new features that substantially enhance classification accuracy. The proposed model, achieving an accuracy of 84.73 % and an area under the ROC (AUC) score of 92.18 % on a dataset comprising 100 Korean PD patients and 100 healthy controls, offers a practical solution for automated diagnosis applications, such as smartphone apps. Future research endeavors will concentrate on enhancing the model's performance and delving deeper into the relationship between high-importance features and PD.

Exploring Spectrogram-Based Audio Classification for Parkinson's Disease: A Study on Speech Classification and Qualitative Reliability Verification.

Patients suffering from Parkinson's disease suffer from voice impairment. In this study, we introduce models to classify normal and Parkinson's patients using their speech. We used an AST (audio spectrogram transformer), a transformer-based speech classification model that has recently outperformed CNN-based models in many fields, and a CNN-based PSLA (pretraining, sampling, labeling, and aggregation), a high-performance model in the existing speech classification field, for the study. This study compares and analyzes the models from both quantitative and qualitative perspectives. First, qualitatively, PSLA outperformed AST by more than 4% in accuracy, and the AUC was also higher, with 94.16% for AST and 97.43% for PSLA. Furthermore, we qualitatively evaluated the ability of the models to capture the acoustic features of Parkinson's through various CAM (class activation map)-based XAI (eXplainable AI) models such as GradCAM and EigenCAM. Based on PSLA, we found that the model focuses well on the muffled frequency band of Parkinson's speech, and the heatmap analysis of false positives and false negatives shows that the speech features are also visually represented when the model actually makes incorrect predictions. The contribution of this paper is that we not only found a suitable model for diagnosing Parkinson's through speech using two different types of models but also validated the predictions of the model in practice.

Peptide-Drug Conjugate with Statistically Designed Transcellular Peptide for Psoriasis-Like Inflammation.

Peptide-drug conjugates (PDCs) are a promising class of drug delivery systems that utilize covalently conjugated carrier peptides with therapeutic agents. PDCs offer several advantages over traditional drug delivery systems including enhanced target engagement, improved bioavailability, and increased cell permeability. However, the development of efficient transcellular peptides capable of effectively transporting drugs across biological barriers remains an unmet need. In this study, physicochemical criteria based on cell-penetrating peptides are employed to design transcellular peptides derived from an antimicrobial peptides library. Among the statistically designed transcellular peptides (SDTs), SDT7 exhibits higher skin permeability, faster kinetics, and improved cell permeability in human keratinocyte cells compared to the control peptide. Subsequently, it is found that 6-Paradol (PAR) exhibits inhibitory activity against phosphodiesterase 4, which can be utilized for an anti-inflammatory PDC. The transcellular PDC (SDT7-conjugated with PAR, named TM5) is evaluated in mouse models of psoriasis, exhibiting superior therapeutic efficacy compared to PAR alone. These findings highlight the potential of transcellular PDCs (TDCs) as a promising approach for the treatment of inflammatory skin disorders.

Analysis of Nursing Students' Nonverbal Communication Patterns during Simulation Practice: A Pilot Study.

Therapeutic communication, of which nonverbal communication is a vital component, is an essential skill for professional nurses. The aim of this study is to assess the possibility of incorporating computer analysis programs into nursing education programs to improve the nonverbal communication skills of those preparing to become professional nurses. In this pilot observational study, the research team developed a computer program for nonverbal communication analysis including facial expressions and poses. The video clip data captured during nursing simulation practice by 10 3rd- and 4th-grade nursing students at a university in South Korea involved two scenarios of communication with a child's mother regarding the child's pre- and post-catheterization care. The dominant facial expressions varied, with sadness (30.73%), surprise (30.14%), and fear (24.11%) being the most prevalent, while happiness (7.96%) and disgust (6.79%) were less common. The participants generally made eye contact with the mother, but there were no instances of light touch by hand and the physical distance for nonverbal communication situations was outside the typical range. These results confirm the potential use of facial expression and pose analysis programs for communication education in nursing practice.

Facial Expressions Track Depressive Symptoms in Old Age.

Facial expressions play a crucial role in the diagnosis of mental illnesses characterized by mood changes. The Facial Action Coding System (FACS) is a comprehensive framework that systematically categorizes and captures even subtle changes in facial appearance, enabling the examination of emotional expressions. In this study, we investigated the association between facial expressions and depressive symptoms in a sample of 59 older adults without cognitive impairment. Utilizing the FACS and the Korean version of the Beck Depression Inventory-II, we analyzed both "posed" and "spontaneous" facial expressions across six basic emotions: happiness, sadness, fear, anger, surprise, and disgust. Through principal component analysis, we summarized 17 action units across these emotion conditions. Subsequently, multiple regression analyses were performed to identify specific facial expression features that explain depressive symptoms. Our findings revealed several distinct features of posed and spontaneous facial expressions. Specifically, among older adults with higher depressive symptoms, a posed face exhibited a downward and inward pull at the corner of the mouth, indicative of sadness. In contrast, a spontaneous face displayed raised and narrowed inner brows, which was associated with more severe depressive symptoms in older adults. These findings suggest that facial expressions can provide valuable insights into assessing depressive symptoms in older adults.

Experimental Verification of the Possibility of Reducing Photoplethysmography Measurement Time for Stress Index Calculation.

Stress is a direct or indirect cause of reduced work efficiency in daily life. It can damage physical and mental health, leading to cardiovascular disease and depression. With increased interest and awareness of the risks of stress in modern society, there is a growing demand for quick assessment and monitoring of stress levels. Traditional ultra-short-term stress measurement classifies stress situations using heart rate variability (HRV) or pulse rate variability (PRV) information extracted from electrocardiogram (ECG) or photoplethysmography (PPG) signals. However, it requires more than one minute, making it difficult to monitor stress status in real-time and accurately predict stress levels. In this paper, stress indices were predicted using PRV indices acquired at different lengths of time (60 s, 50 s, 40 s, 30 s, 20 s, 10 s, and 5 s) for the purpose of real-time stress monitoring. Stress was predicted with Extra Tree Regressor, Random Forest Regressor, and Gradient Boost Regressor models using a valid PRV index for each data acquisition time. The predicted stress index was evaluated using an R2 score between the predicted stress index and the actual stress index calculated from one minute of the PPG signal. The average R2 score of the three models by the data acquisition time was 0.2194 at 5 s, 0.7600 at 10 s, 0.8846 at 20 s, 0.9263 at 30 s, 0.9501 at 40 s, 0.9733 at 50 s, and 0.9909 at 60 s. Thus, when stress was predicted using PPG data acquired for 10 s or more, the R2 score was confirmed to be over 0.7.

Photoplethysmogram Biometric Authentication Using a 1D Siamese Network.

In the head-mounted display environment for experiencing metaverse or virtual reality, conventional input devices cannot be used, so a new type of nonintrusive and continuous biometric authentication technology is required. Since the wrist wearable device is equipped with a photoplethysmogram sensor, it is very suitable for use for nonintrusive and continuous biometric authentication purposes. In this study, we propose a one-dimensional Siamese network biometric identification model using a photoplethysmogram. To maintain the unique characteristics of each person and reduce noise in preprocessing, we adopted a multicycle averaging method without using a bandpass or low-pass filter. In addition, to verify the effectiveness of the multicycle averaging method, the number of cycles was changed and the results were compared. Genuine and impostor data were used to verify the biometric identification. We used the one-dimensional Siamese network to verify the similarity between the classes and found that the method with five overlapping cycles was the most effective. Tests were conducted on the overlapping data of five single-cycle signals and excellent identification results were observed, with an AUC score of 0.988 and an accuracy of 0.9723. Thus, the proposed biometric identification model is time-efficient and shows excellent security performance, even in devices with limited computational capabilities, such as wearable devices. Consequently, our proposed method has the following advantages compared with previous works. First, the effect of noise reduction and information preservation through multicycle averaging was experimentally verified by varying the number of photoplethysmogram cycles. Second, by analyzing authentication performance through genuine and impostor matching analysis based on a one-dimensional Siamese network, the accuracy that is not affected by the number of enrolled subjects was derived.

Multi-Currency Integrated Serial Number Recognition Model of Images Acquired by Banknote Counters.

The objective of this study was to establish an automated system for the recognition of banknote serial numbers by developing a deep learning (DL)-based optical character recognition framework. An integrated serial number recognition model for the banknotes of four countries (South Korea (KRW), the United States (USD), India (INR), and Japan (JPY)) was developed. One-channel image data obtained from banknote counters were used in this study. The dataset used for the multi-currency integrated serial number recognition contains about 150,000 images. The class imbalance problem and model accuracy were improved through data augmentation based on geometric transforms that consider the range of errors that occur when a bill is inserted into the counter. In addition, by fine-tuning the recognition network, it was confirmed that the performance was improved when the serial numbers of the banknotes of four countries were recognized instead of the serial number of a banknote from each country from a single-currency dataset, and the generalization performance was improved by training the model to recognize the diverse serial numbers of multiple currencies. Therefore, the proposed method shows that real-time processing of less than 30 ms per image and character recognition with 99.99% accuracy are possible, even though there is a tradeoff between inference speed and serial number recognition accuracy when data augmentation based on the characteristics of banknote counters and a 1-stage object detector for banknote serial number recognition is used.

Face Biometric Spoof Detection Method Using a Remote Photoplethysmography Signal.

Spoofing attacks in face recognition systems are easy because faces are always exposed. Various remote photoplethysmography-based methods to detect face spoofing have been developed. However, they are vulnerable to replay attacks. In this study, we propose a remote photoplethysmography-based face recognition spoofing detection method that minimizes the susceptibility to certain database dependencies and high-quality replay attacks without additional devices. The proposed method has the following advantages. First, because only an RGB camera is used to detect spoofing attacks, the proposed method is highly usable in various mobile environments. Second, solutions are incorporated in the method to obviate new attack scenarios that have not been previously dealt with. In this study, we propose a remote photoplethysmography-based face recognition spoofing detection method that improves susceptibility to certain database dependencies and high-quality replay attack, which are the limitations of previous methods without additional devices. In the experiment, we also verified the cut-off attack scenario in the jaw and cheek area where the proposed method can be counter-attacked. By using the time series feature and the frequency feature of the remote photoplethysmography signal, it was confirmed that the accuracy of spoof detection was 99.7424%.

Preeminently Robust Neural PPG Denoiser.

Photoplethysmography (PPG) is a simple and cost-efficient technique that effectively measures cardiovascular response by detecting blood volume changes in a noninvasive manner. A practical challenge in the use of PPGs in real-world applications is noise reduction. PPG signals are likely to be compromised by various types of noise, such as scattering or motion artifacts, and removing such compounding noises using a monotonous method is not easy. To this end, this paper proposes a neural PPG denoiser that can robustly remove multiple types of noise from a PPG signal. By casting the noise reduction problem into a signal restoration approach, we aim to achieve a solid performance in the reduction of different noise types using a single neural denoiser built upon transformer-based deep generative models. Using this proposed method, we conducted the experiments on the noise reduction of a PPG signal synthetically contaminated with five types of noise. Following this, we performed a comparative study using six different noise reduction algorithms, each of which is known to be the best model for each noise. Evaluation results of the peak signal-to-noise ratio (PSNR) show that the neural PPG denoiser is superior in three out of five noise types to the performance of conventional noise reduction algorithms. The salt-and-pepper noise type showed the best performance, with the PSNR of the neural PPG denoiser being 36.6080, and the PSNRs of the other methods were 19.8160 and 32.8234. The Poisson noise type performed the worst, showing a PSNR of 33.0090; the PSNRs of other methods were 35.1822 and 33.4795, respectively. Thereafter, an experiment to recover a signal synthesized with two or more of the five noise types was conducted. When the number of mixed noises was two, three, four, and five, the PSNRs were 29.2759, 27.8759, 26.5608, and 25.9402, respectively. Finally, an experiment to recover motion artifacts was also conducted. The synthesized motion artifact signal was created by synthesizing only a certain ratio of the total signal length. As a result of the motion artifact signal restoration, the PSNRs were 25.2872, 22.8240, 21.2901, and 19.9577 at 30%, 50%, 70%, and 90% motion artifact ratios, respectively. In the three experiments conducted, the neural PPG denoiser showed that various types of noise were effectively removed. This proposal contributes to the universal denoising of continuous PPG signals and can be further expanded to denoise continuous signals in the general domain.

Multi-Class Parrot Image Classification Including Subspecies with Similar Appearance.

Owing to climate change and human indiscriminate development, the population of endangered species has been decreasing. To protect endangered species, many countries worldwide have adopted the CITES treaty to prevent the extinction of endangered plants and animals. Moreover, research has been conducted using diverse approaches, particularly deep learning-based animal and plant image recognition methods. In this paper, we propose an automated image classification method for 11 endangered parrot species included in CITES. The 11 species include subspecies that are very similar in appearance. Data images were collected from the Internet and built in cooperation with Seoul Grand Park Zoo to build an indigenous database. The dataset for deep learning training consisted of 70% training set, 15% validation set, and 15% test set. In addition, a data augmentation technique was applied to reduce the data collection limit and prevent overfitting. The performance of various backbone CNN architectures (i.e., VGGNet, ResNet, and DenseNet) were compared using the SSD model. The experiment derived the test set image performance for the training model, and the results show that the DenseNet18 had the best performance with an mAP of approximately 96.6% and an inference time of 0.38 s.

Non-Contact Oxygen Saturation Measurement Using YCgCr Color Space with an RGB Camera.

Oxygen saturation (SPO2) is an important indicator of health, and is usually measured by placing a pulse oximeter in contact with a finger or earlobe. However, this method has a problem in that the skin and the sensor must be in contact, and an additional light source is required. To solve these problems, we propose a non-contact oxygen saturation measurement technique that uses a single RGB camera in an ambient light environment. Utilizing the fact that oxygenated and deoxygenated hemoglobin have opposite absorption coefficients at green and red wavelengths, the color space of photoplethysmographic (PPG) signals recorded from the faces of study participants were converted to the YCgCr color space. Substituting the peaks and valleys extracted from the converted Cg and Cr PPG signals into the Beer-Lambert law yields the SPO2 via a linear equation. When the non-contact SPO2 measurement value was evaluated based on the reference SPO2 measured with a pulse oximeter, the mean absolute error was 0.537, the root mean square error was 0.692, the Pearson correlation coefficient was 0.86, the cosine similarity was 0.99, and the intraclass correlation coefficient was 0.922. These results confirm the feasibility of non-contact SPO2 measurements.

Pulse Rate Variability Analysis Using Remote Photoplethysmography Signals.

Pulse rate variability (PRV) refers to the change in the interval between pulses in the blood volume pulse (BVP) signal acquired using photoplethysmography (PPG). PRV is an indicator of the health status of an individual's autonomic nervous system. A representative method for measuring BVP is contact PPG (CPPG). CPPG may cause discomfort to a user, because the sensor is attached to the finger for measurements. In contrast, noncontact remote PPG (RPPG) extracts BVP signals from face data using a camera without the need for a sensor. However, because the existing RPPG is a technology that extracts a single pulse rate rather than a continuous BVP signal, it is difficult to extract additional health status indicators. Therefore, in this study, PRV analysis is performed using lab-based RPPG technology that can yield continuous BVP signals. In addition, we intended to confirm that the analysis of PRV via RPPG can be performed with the same quality as analysis via CPPG. The experimental results confirmed that the temporal and frequency parameters of PRV extracted from RPPG and CPPG were similar. In terms of correlation, the PRVs of RPPG and CPPG yielded correlation coefficients between 0.98 and 1.0.

Restoration of Remote PPG Signal through Correspondence with Contact Sensor Signal.

Photoplethysmography (PPG) is an optical measurement technique that detects changes in blood volume in the microvascular layer caused by the pressure generated by the heartbeat. To solve the inconvenience of contact PPG measurement, a remote PPG technology that can measure PPG in a non-contact way using a camera was developed. However, the remote PPG signal has a smaller pulsation component than the contact PPG signal, and its shape is blurred, so only heart rate information can be obtained. In this study, we intend to restore the remote PPG to the level of the contact PPG, to not only measure heart rate, but to also obtain morphological information. Three models were used for training: support vector regression (SVR), a simple three-layer deep learning model, and SVR + deep learning model. Cosine similarity and Pearson correlation coefficients were used to evaluate the similarity of signals before and after restoration. The cosine similarity before restoration was 0.921, and after restoration, the SVR, deep learning model, and SVR + deep learning model were 0.975, 0.975, and 0.977, respectively. The Pearson correlation coefficient was 0.778 before restoration and 0.936, 0.933, and 0.939, respectively, after restoration.

Non-Contact Respiration Measurement Method Based on RGB Camera Using 1D Convolutional Neural Networks.

Conventional respiration measurement requires a separate device and/or can cause discomfort, so it is difficult to perform routinely, even for patients with respiratory diseases. The development of contactless respiration measurement technology would reduce discomfort and help detect and prevent fatal diseases. Therefore, we propose a respiration measurement method using a learning-based region-of-interest detector and a clustering-based respiration pixel estimation technique. The proposed method consists of a model for classifying whether a pixel conveys respiration information based on its variance and a method for classifying pixels with clear breathing components using the symmetry of the respiration signals. The proposed method was evaluated with the data of 14 men and women acquired in an actual environment, and it was confirmed that the average error was within approximately 0.1 bpm. In addition, a Bland-Altman analysis confirmed that the measurement result had no error bias, and regression analysis confirmed that the correlation of the results with the reference is high. The proposed method, designed to be inexpensive, fast, and robust to noise, is potentially suitable for practical use in clinical scenarios.

Siamese Architecture-Based 3D DenseNet with Person-Specific Normalization Using Neutral Expression for Spontaneous and Posed Smile Classification.

Clinical studies have demonstrated that spontaneous and posed smiles have spatiotemporal differences in facial muscle movements, such as laterally asymmetric movements, which use different facial muscles. In this study, a model was developed in which video classification of the two types of smile was performed using a 3D convolutional neural network (CNN) applying a Siamese network, and using a neutral expression as reference input. The proposed model makes the following contributions. First, the developed model solves the problem caused by the differences in appearance between individuals, because it learns the spatiotemporal differences between the neutral expression of an individual and spontaneous and posed smiles. Second, using a neutral expression as an anchor improves the model accuracy, when compared to that of the conventional method using genuine and imposter pairs. Third, by using a neutral expression as an anchor image, it is possible to develop a fully automated classification system for spontaneous and posed smiles. In addition, visualizations were designed for the Siamese architecture-based 3D CNN to analyze the accuracy improvement, and to compare the proposed and conventional methods through feature analysis, using principal component analysis (PCA).

Analyzing Facial and Eye Movements to Screen for Alzheimer's Disease.

Brain disease can be screened using eye movements. Degenerative brain disorders change eye movement because they affect not only memory and cognition but also the cranial nervous system involved in eye movement. We compared the facial and eye movement patterns of patients with mild Alzheimer's disease and cognitively normal people to analyze the neurological signs of dementia. After detecting the facial landmarks, the coordinate values for the movements were extracted. We used Spearman's correlation coefficient to examine associations between horizontal and vertical facial and eye movements. We analyzed the correlation between facial and eye movements without using special eye-tracking equipment or complex conditions in order to measure the behavioral aspect of the natural human gaze. As a result, we found differences between patients with Alzheimer's disease and cognitively normal people. Patients suffering from Alzheimer's disease tended to move their face and eyes simultaneously in the vertical direction, whereas the cognitively normal people did not, as confirmed by a Mann-Whitney-Wilcoxon test. Our findings suggest that objective and accurate measurement of facial and eye movements can be used to screen such patients quickly. The use of camera-based testing for the early detection of patients showing signs of neurodegeneration can have a significant impact on the public care of dementia.

Experimental Verification of Objective Visual Fatigue Measurement Based on Accurate Pupil Detection of Infrared Eye Image and Multi-Feature Analysis.

As the use of electronic displays increases rapidly, visual fatigue problems are also increasing. The subjective evaluation methods used for visual fatigue measurement have individual difference problems, while objective methods based on bio-signal measurement have problems regarding motion artifacts. Conventional eye image analysis-based visual fatigue measurement methods do not accurately characterize the complex changes in the appearance of the eye. To solve this problem, in this paper, an objective visual fatigue measurement method based on infrared eye image analysis is proposed. For accurate pupil detection, a convolutional neural network-based semantic segmentation method was used. Three features are calculated based on the pupil detection results: (1) pupil accommodation speed, (2) blink frequency, and (3) eye-closed duration. In order to verify the calculated features, differences in fatigue caused by changes in content color components such as gamma, color temperature, and brightness were compared with a reference video. The pupil detection accuracy was confirmed to be 96.63% based on the mean intersection over union. In addition, it was confirmed that all three features showed significant differences from the reference group; thus, it was verified that the proposed analysis method can be used for the objective measurement of visual fatigue.

Differences in Facial Expressions between Spontaneous and Posed Smiles: Automated Method by Action Units and Three-Dimensional Facial Landmarks.

Research on emotion recognition from facial expressions has found evidence of different muscle movements between genuine and posed smiles. To further confirm discrete movement intensities of each facial segment, we explored differences in facial expressions between spontaneous and posed smiles with three-dimensional facial landmarks. Advanced machine analysis was adopted to measure changes in the dynamics of 68 segmented facial regions. A total of 57 normal adults (19 men, 38 women) who displayed adequate posed and spontaneous facial expressions for happiness were included in the analyses. The results indicate that spontaneous smiles have higher intensities for upper face than lower face. On the other hand, posed smiles showed higher intensities in the lower part of the face. Furthermore, the 3D facial landmark technique revealed that the left eyebrow displayed stronger intensity during spontaneous smiles than the right eyebrow. These findings suggest a potential application of landmark based emotion recognition that spontaneous smiles can be distinguished from posed smiles via measuring relative intensities between the upper and lower face with a focus on left-sided asymmetry in the upper region.