Pulse Rate Guided Oxygen Saturation Monitoring Using a Wearable Armband Sensor.

Continuous clinical grade measurement of SpO2 in out-of-hospital settings remains a challenge despite the widespread use of photoplethysmography (PPG) based wearable devices for health and wellness applications. This article presents two SpO2 algorithms: PRR (pulse rate derived ratio-of-ratios) and GPDR (green-assisted peak detection ratio-of-ratios), that utilize unique pulse rate frequency estimations to isolate the pulsatile (AC) component of red and infrared PPG signals and derive SpO2 measurements. The performance of the proposed SpO2 algorithms are evaluated using an upper-arm wearable device derived green, red, and infrared PPG signals, recorded in both controlled laboratory settings involving healthy subjects (n=36) and an uncontrolled clinic application involving COVID-19 patients (n=52). GPDR exhibits the lowest root mean square error (RMSE) of 1.6±0.6% for a respiratory exercise test, 3.6 ±1.0% for a standard hypoxia test, and 2.2±1.3% for an uncontrolled clinic use-case. In contrast, PRR provides relatively higher error but with greater coverage overall. Mean error across all combined datasets were 0.2±2.8% and 0.3±2.4% for PRR and GPDR respectively. Both SpO2 algorithms achieve great performance of low error with high coverage on both uncontrolled clinic and controlled laboratory conditions.

Novel Continuous Respiratory Rate Monitoring Using an Armband Wearable Sensor.

Photoplethysmography (PPG) and accelerometer (ACC) are commonly integrated into wearable devices for continuous unobtrusive pulse rate and activity monitoring of individuals during daily life. However, obtaining continuous and clinically accurate respiratory rate measurements using such wearable sensors remains a challenge. This article presents a novel algorithm for estimation of respiration rate (RR) using an upper-arm worn wearable device by deriving multiple respiratory surrogate signals from PPG and ACC sensing. This RR algorithm is retrospectively evaluated on a controlled respiratory clinical testing dataset from 38 subjects with simultaneously recorded wearable sensor data and a standard capnography monitor as an RR reference. The proposed RR method shows great performance and robustness in determining RR measurements over a wide range of 4-59 brpm with an overall bias of -1.3 brpm, mean absolute error (MAE) of 2.7±1.6 brpm, and a meager outage of 0.3±1.2%, while a standard PPG Smart Fusion method produces a bias of -3.6 brpm, an MAE of 5.5±3.1 brpm, and an outage of 0.7±2.5% for direct comparison. In addition, the proposed algorithm showed no significant differences (p=0.63) in accurately determining RR values in subjects with darker skin tones, while the RR performance of the PPG Smart Fusion method is significantly (P<0.001) affected by the darker skin pigmentation. This study demonstrates a highly accurate RR algorithm for unobtrusive continuous RR monitoring using an armband wearable device.

Respiratory Rate Estimation using PPG: A Deep Learning Approach.

Respiratory rate (RR) is an important vital sign marker of health, and it is often neglected due to a lack of unobtrusive sensors for objective and convenient measurement. The respiratory modulations present in simple photoplethysmogram (PPG) have been useful to derive RR using signal processing, waveform fiducial markers, and hand-crafted rules. An end- to-end deep learning approach based on residual network (ResNet) architecture is proposed to estimate RR using PPG. This approach takes time-series PPG data as input, learns the rules through the training process that involved an additional synthetic PPG dataset generated to overcome the insufficient data problem of deep learning, and provides RR estimation as outputs. The inclusion of a synthetic dataset for training improved the performance of the deep learning model by 34%. The final mean absolute error performance of the deep learning approach for RR estimation was 2.5±0.6 brpm using 5-fold cross-validation in two widely used public PPG datasets (n=95) with reliable RR references. The deep learning model achieved comparable performance to that of a classical method, which was also implemented for comparison. With large real-world data and reference ground truth, deep learning can be valuable for RR or other vital sign monitoring using PPG and other physiological signals.

Decoupling of spontaneous facial mimicry from emotion recognition in schizophrenia.

Past research indicates that spontaneous mimicry facilitates the decoding of others' emotions, leading to enhanced social perception and interpersonal rapport. Individuals with schizophrenia (SZ) show consistent deficits in emotion recognition and expression associated with poor social functioning. Given the prominence of blunted affect in schizophrenia, it is possible that spontaneous facial mimicry may also be impaired. However, studies assessing automatic facial mimicry in schizophrenia have yielded mixed results. It is therefore unknown whether emotion recognition deficits and impaired automatic facial mimicry are related in schizophrenia. SZ and demographically matched controls (CO) participated in a dynamic emotion recognition task. Electromyographic activity in muscles responsible for producing facial expressions was recorded during the task to assess spontaneous facial mimicry. SZ showed deficits in emotion identification compared to CO, but there was no group difference in the predictive power of spontaneous facial mimicry for avatar's expressed emotion. In CO, facial mimicry supported accurate emotion recognition, but it was decoupled in SZ. The finding of intact facial mimicry in SZ bears important clinical implications. For instance, clinicians might be able to improve the social functioning of patients by teaching them to pair specific patterns of facial muscle activation with distinct emotion words.

The acceptability and feasibility of a novel virtual reality based social skills training game for schizophrenia: Preliminary findings.

Social impairment is a core feature of schizophrenia that presents a major barrier toward recovery. Some of the psychotic symptoms are partly ameliorated by medication but the route to recovery is hampered by social impairments. Since existing social skills interventions tend to suffer from lack of availability, high-burden and low adherence, there is a dire need for an effective, alternative strategy. The present study examined the feasibility and acceptability of Multimodal Adaptive Social Intervention in Virtual Reality (MASI-VR) for improving social functioning and clinical outcomes in schizophrenia. Out of eighteen patients with schizophrenia who enrolled, seventeen participants completed the pre-treatment assessment and 10 sessions of MASI-VR, but one patient did not complete the post-treatment assessments. Therefore, the complete training plus pre- and post-treatment assessment data are available from sixteen participants. Clinical ratings of symptom severity were obtained at pre- and post-training. Retention rates were very high and training was rated as extremely satisfactory for the majority of participants. Participants exhibited a significant reduction in overall clinical symptoms, especially negative symptoms following 10 sessions of MASI-VR. These preliminary results support the feasibility and acceptability of a novel virtual reality social skills training program for individuals with schizophrenia.

A Novel Multisensory Stimulation and Data Capture System (MADCAP) for Investigating Sensory Trajectories in Infancy.

Sensory processing differences, including responses to auditory, visual, and tactile stimuli, are ideal targets for early detection of neurodevelopmental risks, such as autism spectrum disorder. However, most existing studies focus on the audiovisual paradigm and ignore the sense of touch. In this paper, we present a multisensory delivery system that can deliver audio, visual, and tactile stimuli in a controlled manner and capture peripheral physiological, eye gaze, and electroencephalographic response data. The novelty of the system is the ability to provide affective touch. In particular, we have developed a tactile stimulation device that delivers tactile stimuli to infants with precisely controlled brush stroking speed and force on the skin. A usability study of 10 3-20 month-old infants was conducted to investigate the tolerability and feasibility of the system. Results have shown that the system is well tolerated by infants and all the data were collected robustly. This paper paves the way for future studies charting the sensory response trajectories in infancy.

Extraction of Emotional Information via Visual Scanning Patterns: A Feasibility Study of Participants with Schizophrenia and Neurotypical Individuals.

Emotion recognition impairment is a core feature of schizophrenia (SZ), present throughout all stages of this condition, and leads to poor social outcome. However, the underlying mechanisms that give rise to such deficits have not been elucidated and hence, it has been difficult to develop precisely targeted interventions. Evidence supports the use of methods designed to modify patterns of visual attention in individuals with SZ in order to effect meaningful improvements in social cognition. To date, however, attention-shaping systems have not fully utilized available technology (e.g., eye tracking) to achieve this goal. The current work consisted of the design and feasibility testing of a novel gaze-sensitive social skills intervention system called MASI-VR. Adults from an outpatient clinic with confirmed SZ diagnosis (n=10) and a comparison sample of neurotypical participants (n=10) were evaluated on measures of emotion recognition and visual attention at baseline assessment, and a pilot test of the intervention system was evaluated on the SZ sample following five training sessions over three weeks. Consistent with the literature, participants in the SZ group demonstrated lower recognition of faces showing medium intensity fear, spent more time deliberating about presented emotions, and had fewer fixations in comparison to neurotypical peers. Furthermore, participants in the SZ group showed significant improvement in the recognition of fearful faces post-training. Preliminary evidence supports the feasibility of a gaze-sensitive paradigm for use in assessment and training of emotion recognition and social attention in individuals with SZ, thus warranting further evaluation of the novel intervention.

Cognitive Load Measurement in a Virtual Reality-based Driving System for Autism Intervention.

Autism Spectrum Disorder (ASD) is a highly prevalent neurodevelopmental disorder with enormous individual and social cost. In this paper, a novel virtual reality (VR)-based driving system was introduced to teach driving skills to adolescents with ASD. This driving system is capable of gathering eye gaze, electroencephalography, and peripheral physiology data in addition to driving performance data. The objective of this paper is to fuse multimodal information to measure cognitive load during driving such that driving tasks can be individualized for optimal skill learning. Individualization of ASD intervention is an important criterion due to the spectrum nature of the disorder. Twenty adolescents with ASD participated in our study and the data collected were used for systematic feature extraction and classification of cognitive loads based on five well-known machine learning methods. Subsequently, three information fusion schemes-feature level fusion, decision level fusion and hybrid level fusion-were explored. Results indicate that multimodal information fusion can be used to measure cognitive load with high accuracy. Such a mechanism is essential since it will allow individualization of driving skill training based on cognitive load, which will facilitate acceptance of this driving system for clinical use and eventual commercialization.

A Pilot Study Assessing Performance and Visual Attention of Teenagers with ASD in a Novel Adaptive Driving Simulator.

Individuals with Autism Spectrum Disorder (ASD), compared to typically-developed peers, may demonstrate behaviors that are counter to safe driving. The current work examines the use of a novel simulator in two separate studies. Study 1 demonstrates statistically significant performance differences between individuals with (N = 7) and without ASD (N = 7) with regards to the number of turning-related driving errors (p < 0.01). Study 2 shows that both the performance-based feedback group (N = 9) and combined performance- and gaze-sensitive feedback group (N = 8) achieved statistically significant reductions in driving errors following training (p < 0.05). These studies are the first to present results of fine-grained measures of visual attention of drivers and an adaptive driving intervention for individuals with ASD.

A Robotic Coach Architecture for Elder Care (ROCARE) Based on Multi-User Engagement Models.

The aging population with its concomitant medical conditions, physical and cognitive impairments, at a time of strained resources, establishes the urgent need to explore advanced technologies that may enhance function and quality of life. Recently, robotic technology, especially socially assistive robotics has been investigated to address the physical, cognitive, and social needs of older adults. Most system to date have predominantly focused on one-on-one human robot interaction (HRI). In this paper, we present a multi-user engagement-based robotic coach system architecture (ROCARE). ROCARE is capable of administering both one-on-one and multi-user HRI, providing implicit and explicit channels of communication, and individualized activity management for long-term engagement. Two preliminary feasibility studies, a one-on-one interaction and a triadic interaction with two humans and a robot, were conducted and the results indicated potential usefulness and acceptance by older adults, with and without cognitive impairment.

Complexities of emotional responses to social and non-social affective stimuli in schizophrenia.

BACKGROUND: Adaptive emotional responses are important in interpersonal relationships. We investigated self-reported emotional experience, physiological reactivity, and micro-facial expressivity in relation to the social nature of stimuli in individuals with schizophrenia (SZ). METHOD: Galvanic skin response (GSR) and facial electromyography (fEMG) were recorded in medicated outpatients with SZ and demographically matched healthy controls (CO) while they viewed social and non-social images from the International Affective Pictures System. Participants rated the valence and arousal, and selected a label for experienced emotions. Symptom severity in the SZ and psychometric schizotypy in CO were assessed. RESULTS: The two groups did not differ in their labeling of the emotions evoked by the stimuli, but individuals with SZ were more positive in their valence ratings. Although self-reported arousal was similar in both groups, mean GSR was greater in SZ, suggesting differential awareness, or calibration of internal states. Both groups reported social images to be more arousing than non-social images but their physiological responses to non-social vs. social images were different. Self-reported arousal to neutral social images was correlated with positive symptoms in SZ. Negative symptoms in SZ and disorganized schizotypy in CO were associated with reduced mean fEMG. Greater corrugator mean fEMG activity for positive images in SZ indicates valence-incongruent facial expressions. CONCLUSION: The patterns of emotional responses differed between the two groups. While both groups were in broad agreement in self-reported arousal and emotion labels, their mean GSR, and fEMG correlates of emotion diverged in relation to the social nature of the stimuli and clinical measures. Importantly, these results suggest disrupted self awareness of internal states in SZ and underscore the complexities of emotion processing in health and disease.

A Step towards EEG-based brain computer interface for autism intervention.

Autism Spectrum Disorder (ASD) is a prevalent and costly neurodevelopmental disorder. Individuals with ASD often have deficits in social communication skills as well as adaptive behavior skills related to daily activities. We have recently designed a novel virtual reality (VR) based driving simulator for driving skill training for individuals with ASD. In this paper, we explored the feasibility of detecting engagement level, emotional states, and mental workload during VR-based driving using EEG as a first step towards a potential EEG-based Brain Computer Interface (BCI) for assisting autism intervention. We used spectral features of EEG signals from a 14-channel EEG neuroheadset, together with therapist ratings of behavioral engagement, enjoyment, frustration, boredom, and difficulty to train a group of classification models. Seven classification methods were applied and compared including Bayes network, naïve Bayes, Support Vector Machine (SVM), multilayer perceptron, K-nearest neighbors (KNN), random forest, and J48. The classification results were promising, with over 80% accuracy in classifying engagement and mental workload, and over 75% accuracy in classifying emotional states. Such results may lead to an adaptive closed-loop VR-based skill training system for use in autism intervention.