Facial Video-Based Non-Contact Stress Recognition Utilizing Multi-Task Learning With Peak Attention.

Negative emotional states, such as anxiety and depression, pose significant challenges in contemporary society, often stemming from the stress encountered in daily activities. Stress (state or level) recognition is a crucial prerequisite for effective stress management and intervention. Presently, wearable devices have been employed to capture physiological signals and analyze stress states. However, their constant skin contact can lead to discomfort and disturbance during prolonged monitoring. In this paper, a peak attention-based multitasking framework is presented for non-contact stress recognition. The framework extracts rPPG signals from RGB facial videos, utilizing them as inputs for a novel multi-task attentional convolutional neural network for stress recognition (MTASR). It incorporates peak detection and HR estimation as auxiliary tasks to facilitate stress recognition. By leveraging multi-task learning, MTASR can utilize information related to stress physiological responses, thereby enhancing feature extraction efficiency. For stress recognition, two binary classification tasks are applied: stress state recognition and stress level recognition. The model is validated on the UBFC-Phys public dataset and demonstrates an accuracy of 94.33% for stress state recognition and 83.83% for stress level recognition. The proposed method outperforms the dataset's baseline methods and other competing approaches.

Facial Video-Based Remote Physiological Measurement via Self-Supervised Learning.

Facial video-based remote physiological measurement aims to estimate remote photoplethysmography (rPPG) signals from human facial videos and then measure multiple vital signs (e.g., heart rate, respiration frequency) from rPPG signals. Recent approaches achieve it by training deep neural networks, which normally require abundant facial videos and synchronously recorded photoplethysmography (PPG) signals for supervision. However, the collection of these annotated corpora is not easy in practice. In this paper, we introduce a novel frequency-inspired self-supervised framework that learns to estimate rPPG signals from facial videos without the need of ground truth PPG signals. Given a video sample, we first augment it into multiple positive/negative samples which contain similar/dissimilar signal frequencies to the original one. Specifically, positive samples are generated using spatial augmentation; negative samples are generated via a learnable frequency augmentation module, which performs non-linear signal frequency transformation on the input without excessively changing its visual appearance. Next, we introduce a local rPPG expert aggregation module to estimate rPPG signals from augmented samples. It encodes complementary pulsation information from different face regions and aggregates them into one rPPG prediction. Finally, we propose a series of frequency-inspired losses, i.e., frequency contrastive loss, frequency ratio consistency loss, and cross-video frequency agreement loss, for the optimization of estimated rPPG signals from multiple augmented video samples. We conduct rPPG-based heart rate, heart rate variability, and respiration frequency estimation on five standard benchmarks. The experimental results demonstrate that our method improves the state of the art by a large margin.

Non-Contact Negative Mood State Detection Using Reliability-Focused Multi-Modal Fusion Model.

Negative mood states include tension, depression, anger, fatigue, and confusion, which represent the weak internal emotions of a human. Negative mood states exert adverse impact on individuals' ability to make rational decisions, which entails the practicable method of negative mood state detection. The most commonly used negative mood state detection methods are based on the psychological scale, which requires additional work and brings inconvenience to the subject in the application scenarios. To overcome this challenge, this paper proposes a novel non-contact negative mood state detection method according to the knowledge of affective computing. The POMS-net model is used to extract temporal-spatial features from visible and infrared thermal videos, and the negative mood state detection is realized using data reliability-focused multi-modal fusion. The proposed method is verified using the HDT-BR dataset collected in the aerospace medicine experiment "Earth-Star II" and the VIRI public dataset. The experimental results on the datasets verify that our method outperforms the comparison methods.

Automatic Acetowhite Lesion Segmentation via Specular Reflection Removal and Deep Attention Network.

Automatic acetowhite lesion segmentation in colposcopy images (cervigrams) is essential in assisting gynecologists for the diagnosis of cervical intraepithelial neoplasia grades and cervical cancer. It can also help gynecologists determine the correct lesion areas for further pathological examination. Existing computer-aided diagnosis algorithms show poor segmentation performance because of specular reflections, insufficient training data and the inability to focus on semantically meaningful lesion parts. In this paper, a novel computer-aided diagnosis algorithm is proposed to segment acetowhite lesions in cervigrams automatically. To reduce the interference of specularities on segmentation performance, a specular reflection removal mechanism is presented to detect and inpaint these areas with precision. Moreover, we design a cervigram image classification network to classify pathology results and generate lesion attention maps, which are subsequently leveraged to guide a more accurate lesion segmentation task by the proposed lesion-aware convolutional neural network. We conducted comprehensive experiments to evaluate the proposed approaches on 3045 clinical cervigrams. Our results show that our method outperforms state-of-the-art approaches and achieves better Dice similarity coefficient and Hausdorff Distance values in acetowhite legion segmentation.

Automatic CIN Grades Prediction of Sequential Cervigram Image Using LSTM With Multistate CNN Features.

Cervical cancer ranks as the second most common cancer in women worldwide. In clinical practice, colposcopy is an indispensable part of screening for cervical intraepithelial neoplasia (CIN) grades and cervical cancer but exhibits high misdiagnosis rate. Existing computer-assisted algorithms for analyzing cervigram images have neglected that colposcopy is a sequential and multistate process, which is unsuitable for clinical applications. In this work, we construct a cervigram-based recurrent convolutional neural network (C-RCNN) to classify different CIN grades and cervical cancer. Convolutional neural networks are leveraged to extract spatial features. We develop a sequence-encoding module to encode discriminative temporal features and a multistate-aware convolutional layer to integrate features from different states of cervigram images. To train and evaluate the performance of C-RCNN, we leveraged a dataset of 4,753 real cervigrams and obtained 96.13% test accuracy with a specificity and sensitivity of 98.22% and 95.09%, respectively. Areas under each receiver operating characteristic curves are above 0.94, proving that visual representations and sequential dynamics can be jointly and effectively optimized in the training phase. Comparative analysis demonstrated the effectiveness of the proposed C-RCNN against competing methods, showing significant improvement over only focusing on a single frame. This architecture can be extended to other applications in medical image analysis.

[Effect of tooth extraction and antibiotics on carotid artery wall and serum interleukin 6 in chronic periodontitis rats with or without atherosclerosis].

OBJECTIVE: To investigate the effect of tooth extraction with antibiotics on atherosclerosis, and to examine the expression of serum interleukin 6(IL-6) and the pathological changes of the carotid artery in chronic periodontitis(CP) rats with or without atherosclerosis(As). METHODS: A total of 44 SD rats were randomly divided into four groups, group A(normal control), group B(As), group C(CP), group D(CP+As). After model establishment, group C and group D were randomly divided into group C1/D1 (tooth extraction) and group C2/D2(tooth extraction with antibiotics) according to random number table and received the corresponding oral intervention treatment respectively. Serum IL-6 levels were determined by enzyme linked immunosorbent assay(ELISA) respectively one week before the intervention, one week after the first intervention, one, three, five weeks after the second intervention. The pathological changes of the carotid artery were accessed under light microscope. RESULTS: At all sampling time points, the levels of serum IL-6 in group B, C, D were higher than that of group A, with group D1 being increased most obviously, significantly higher than that of group A(P< 0.001). One week after the second intervention, the content of IL-6 in group C and group D peaked[C1(127.0 ± 29.9) ng/L, C2: (120.6 ± 23.1) ng/L, D1: (175.1 ± 50.8) ng/L, D2: (160.5 ± 37.7) ng/L], and was significantly higher than that of group B[B: (43.4 ± 7.5) ng/L,P<0.001]. Then they all had varying degrees of decline, 5 weeks after the second intervention, group C1 and D1 were still higher than that of group B, but group C2 and D2 were lower than that of group B. At all sampling time points, the levels of serum IL- 6 in group C2/D2 were lower than those in group C1/D1, 5 weeks after the second intervention the difference was most obvious and statistically significant(P<0.001). Pathology showed that the carotid artery wall in group A was normal. The carotid artery wall was thickened in group B, inflammatory cells and foam cells could be seen, and elastic fibers disordered. The carotid artery wall in group C1 was uneven, foam cells and a small amount of inflammatory cells were visible, and elastic fiber disordered. Obvious thickening was not seen in the carotid artery wall of group C2, a small amount of foam cells and inflammatory cells were found, and elastic fiber mildly disordered. The carotid artery wall in group D1 was obviously uneven, calcium salt deposits were visible in the artery wall, a large amount of inflammatory cells and foam cells could be found, and elastic fiber disordered. Obvious thickening was not seen in the carotid artery wall of group D2, a small amount of inflammatory cells and a large amount of foam cells could be seen, and elastic fiber disordered. CONCLUSIONS: Periodontitis and hyperlipidemia could increase the level of serum IL- 6 and the risk of the As. In chronic periodontitis rats with or without atherosclerosis, when periodontal inflammation was not controlled, tooth extraction may increase the risk of the As. At the time of tooth extraction, giving the anti-inflammatory treatment can reduce the risk to a certain extent.

[Effect of oral intervention on matrix metalloproteinase-2, 9 expression in carotid arteries and serum interleukin-6 in rats with chronic periodontitis].

OBJECTIVE: To establish chronic periodontitis model in SD rats, and to investigate the effect of oral intervention on atherosclerosis. METHODS: Fifty male SD rats were randomly divided into three groups, group A (normal control), group B (atherosclerosis,As) and group C (chronic periodontitis, CP). Group C was further divided into group C1 (natural process), group C2 (simple mechanical treatment), group C3 (systemic antibiotics), group C4-1 (teeth extraction) and group C4-2 (teeth extraction+systemic antibiotics), each group consisted of 7 rats. Every group received oral intervention. Serum interleukin (IL)- 6 levels were detected in five different time points (1, 3, 5, 7, 9 weeks after a successful modeling) by enzyme linked immunosorbent assay. All animals were killed after 24 weeks. Matrix metalloproteinase (MMP)- 2, 9 in the proximal aorta was detected by immuno histochemistry. RESULTS: The levels of serum IL-6 in groups B and C1 increased gradually with time and became significantly higher than that in group A (P < 0.01). Levels of serum IL-6 were increased gradually in each intervention group (C2, C3, C4-1, C4-2) and reached its peak at 5 weeks after modeling [C2:(62.3 ± 14.3) ng/L, C3:(58.2 ± 8.7) ng/L, C4-1:(127.0 ± 29.9) ng/L, C4-2:(120.6 ± 23.1) ng/L]. Compared with group B, group C4- 1 and C4- 2 increased most significantly (P < 0.01). Levels of serum IL- 6 decreased gradually. Eventually, group C2 [(28.6 ± 8.1) ng/L], C3 [(40.8 ± 15.1) ng/L] and C4-2 [(32.7 ± 11.1) ng/L] were significantly lower than group B (P < 0.05), and in group C2 IL- 6 was the lowest. Although levels of serum of IL-6 significantly decreased in group C4-1 [(72.8 ± 16.4) ng/L], but remained the highest. Immunohistochemistry showed that MMP-2, 9 were expressed in group B, C1 and C4-1, and significantly higher than in group A (183.0 ± 2.0, 181.3 ± 2.0), the gray value differences were statistically significant (P < 0.01). Group C4-1 (123.1 ± 2.9, 121.0 ± 3.2) was the strongest, group B (126.4 ± 2.0, 124.8 ± 2.8) and C1 (140.0 ± 2.2, 139.7 ± 3.2) were decreased (P < 0.01). While group C2(169.3 ± 3.4, 169.7 ± 2.3), C3 (149.0 ± 1.7, 145.1 ± 2.5) and C4-2 (157.7 ± 1.2, 155.8 ± 2.7) were significantly lower than group C1 (P < 0.01), and group C2 was close to normal. CONCLUSIONS: Periodontitis could increase the risk of atherosclerosis in rats with chronic periodontitis. Periodontal mechanical treatment and teeth extraction may increase the risk of As in the short time. However, the risk would gradually reduce in a long time.

[Influence of periodontal therapy on serum interleukin-6 and carotid metalloproteinases in animals with chronic periodontitis associated with atherosclerosis].

OBJECTIVE: To investigate the effect of periodontal mechanical treatment on serum interleukin-6 (IL-6) and carotid artery matrix metalloproteinase (MMP)-2 and MMP-9 expression in chronic periodontitis (CP) SD rats with atherosclerosis (As). METHODS: Forty-four six-week-old male SD rats were randomly divided into three groups: control group (group A), As group (group B), As+CP group(group C). According to different periodontal interventions, group C was randomly subdivided into four groups: natural process group (C1), the periodontal mechanical treatment group (C2), the periodontal mechanical treatment+ local drugs group (C3), and the periodontal mechanical treatment+local and system drugs group (C4). Each group received the appropriate treatment and periodontal interventions. Serum IL-6 levels were determined by enzyme linked immunosorrbent assay (ELISA). MMP-2 and MMP-9 levels in the proximal aorta were examined by immunohistochemistry. RESULTS: The gray value of MMP-2 and MMP-9 was basically the same in all groups. Compared with group A, the gray value of MMP-2 and MMP-9 of group B and C were decreased. C1 group showed the formation of atherosclerotic plaque and fibrous cap. Compared with group B (126.4 ± 2.0, 124.8 ± 2.8) , the gray value of group C1 (101.3 ± 2.4, 101.2 ± 4.1) was significantly weaker (P < 0.05). The staining depth of MMP-2 and MMP-9 of groups C1, C2, C3 and C4 were sequentially decreased, and the differences of gray value were statistically significant(P < 0.05). The levels of serum IL-6 in groups B and C1 increased gradually with time and became significantly higher than that of group A (P < 0.01). The levels of serum IL-6 in groups C2, C3, and C4 increased gradually and reached the peak 5 weeks after the establishment of model (P < 0.001). After that, the levels of serum IL-6 decreased gradually and was lower than baseline. The levels of serum IL-6 in groups C3 and C4 were significantly lower than that in group C2 7 weeks after the establishment of model(P < 0.01). CONCLUSIONS: In rats with periodontitis and cardiovascular diseases, chronic periodontal inflammation may significantly increase the severity of As and promote the formation of atherosclerotic plaque. Mechanical periodontal therapy may cause short-term systemic inflammation and then reduce vascular inflammation in long term.With supplement use of local and systemic antibiotics, the mechanical periodontal therapy may get the vascular disease and systemic inflammation improved.

[Effects of periodontal mechanical therapy with local and systemic drugs on carotid artery and serum high-sensitivity C-reactive protein in rats with chronic periodontitis associated with atherosclerosis].

OBJECTIVE: The aim of this study is to investigate the effects of serum high-sensitivity C-reactive protein (hsCRP) and the pathological changes in the carotid artery after periodontal mechanical therapy with local and systemic drugs in SD rats with chronic periodontitis (CP) associated with atherosclerosis (As). METHODS: Thirty-five SD rats were randomly divided into two groups: control group (group A) and CP+As group (group B). Group B was further divided into the natural process group (B1), the periodontal mechanical treatment group (B2), the periodontal mechanical treatment plus local drugs group (B3), and the periodontal mechanical treatment plus local and systemic drugs group (B4). Each group comprised seven rats. Serum hsCRP levels were evaluated at baseline 1 week after the first periodontal therapy and 1, 3, and 5 weeks after the second periodontal therapy by enzyme linked immunosorbent assay (ELISA). The pathological lesion in the carotid artery plaque was stained with hematine and eosin. RESULTS: The levels of serum hsCRP in group B1 increased gradually as time passed and became significantly higher than that of the other groups five weeks after periodontal therapy (P < 0.001). The levels of serum hsCRP in groups B2, B3, and B4 increased gradually and reached the peak 1 week after the second periodontal therapy. After that, the levels of serum hsCRP decreased gradually but were still higher than that of group A (P < 0.05). The levels of serum hsCRP in groups B3 and B4 were significantly lower than that in group B2 3 and 5 weeks after the second periodontal therapy (P < 0.001). Histologic sections revealed increased foam cell infiltration and disordered and destructed elastic fibers in groups B1 and B2. The thickness of the blood vessels in groups B3 and B4 was more uniform than that in groups B1 and B2. The elastic fibers in groups B3 and B4 were lined up in order. CONCLUSION: Direct periodontal mechanical treatment results in acute, short-term, systemic inflammation and might increase the risk of atherosclerosis in SD rats. However, the levels of serum hsCRP decreased gradually 3 to 5 weeks after therapy. With periodontal mechanical treatment, the benefits of local and systemic drugs are associated with improvement in atherosclerotic lesion progression.