Contactless Vital Signs Acquisition Using Video Photoplethysmography, Motion Analysis and Passive Infrared Thermography Devices During Emergency Department Walk-In Triage in Pandemic Conditions.

BACKGROUND: Contactless vital signs (VS) measurement with video photoplethysmography (vPPG), motion analysis (MA), and passive infrared thermometry (pIR) has shown promise. OBJECTIVES: To compare conventional (contact-based) and experimental contactless VS measurement approaches for emergency department (ED) walk-in triage in pandemic conditions. METHODS: Patients' heart rates (HR), respiratory rates (RR), and temperatures were measured with cardiorespiratory monitor and vPPG, manual count and MA, and contact thermometers and pIR, respectively. RESULTS: There were 475 walk-in ED patients studied (95% of eligible). Subjects were 35.2 ± 20.8 years old (range 4 days‒95 years); 52% female, 0.2% transgender; had Fitzpatrick skin type of 2.3 ± 1.4 (range 1‒6), Emergency Severity Index of 3.0 ± 0.6 (range 2‒5), and contact temperature of 36.83°C (range 35.89-39.4°C) (98.3°F [96.6‒103°F]). Pediatric HR and RR data were excluded from analysis due to research challenges associated with pandemic workflow. For a 30-s, unprimed "Triage" window in 377 adult patients, vPPG-MA acquired 377 (100%) HR measurements featuring a mean difference with cardiorespiratory monitor HR of 5.9 ± 12.8 beats/min (R = 0.6833) and 252 (66.8%) RR measurements featuring a mean difference with manual RR of -0.4 ± 2.6 beats/min (R = 0.8128). Subjects' Emergency Severity Index components based on conventional VS and contactless VS matched for 83.8% (HR) and 89.3% (RR). Filtering out vPPG-MA measurements with low algorithmic confidence reduced VS acquired while improving correlation with conventional measurements. The mean difference between contact and pIR temperatures was 0.83 ± 0.67°C (range -1.16-3.5°C) (1.5 ± 1.2°F [range -2.1-6.3°F]); pIR fever detection improved with post hoc adjustment for mean bias. CONCLUSION: Contactless VS acquisition demonstrated good agreement with contact methods during adult walk-in ED patient triage in pandemic conditions; clinical applications will need further study.

Domiciliary Cough Monitoring for the Prediction of COPD Exacerbations.

INTRODUCTION: Acute exacerbations of COPD (AE-COPD) are a leading cause of health service utilisation and are associated with morbidity and mortality. Identifying the prodrome of AE-COPD by monitoring symptoms and physiological parameters (telemonitoring) has proven disappointing and false alerts limit clinical utility. We report objective monitoring of cough counts around AE-COPD and the performance of a novel alert system identifying meaningful change in cough frequency. METHODS: This prospective longitudinal study of cough monitoring included chronic obstructive pulmonary disease (COPD) patients experienced in telemonitoring that had two or more AE-COPD in the past year. Participants underwent cough monitoring and completed a daily questionnaire for 90 days. The automated system identified deteriorating trends in cough and this was compared with alerts generated by an established telemonitoring questionnaire. RESULTS: 28 patients [median age 66 (range 46-86), mean FEV-1% predicted 36% (SD 18%)] completed the study and had a total of 58 exacerbations (43 moderate and 15 severe). Alerts based on cough monitoring were generated mean 3.4 days before 45% of AE-COPD with one false alert every 100 days. In contrast, questionnaire-based alerts occurred in the prodrome of 88% of AE-COPD with one false alert every 10 days. CONCLUSION: An alert system based on cough frequency alone predicted 45% AE-COPD; the low false alert rate with cough monitoring suggests it is a practical and clinically relevant tool. In contrast, the utility of questionnaire-based symptom monitoring is limited by frequent false alerts.

Continuous-Spectrum Infrared Illuminator for Camera-PPG in Darkness.

Many camera-based remote photoplethysmography (PPG) applications require sensing in near infrared (NIR). The performance of PPG systems benefits from multi-wavelength processing. The illumination source in such system is explored in this paper. We demonstrate that multiple narrow-band LEDs have inferior color homogeneity compared to broadband light sources. Therefore, we consider the broadband option based on phosphor material excited by LEDs. A first prototype was realized and its details are discussed. It was tested within a remote-PPG monitoring scenario in darkness and the full system demonstrates robust pulse-rate measurement. Given its accuracy in pulse rate extraction, the proposed illumination principle is considered a valuable asset for large-scale NIR-PPG applications as it enables multi-wavelength processing, lightweight set-ups with relatively low-power infrared light sources.

Analysis of nocturnal actigraphic sleep measures in patients with COPD and their association with daytime physical activity.

BACKGROUND: Sleep disturbances are common in patients with chronic obstructive pulmonary disease (COPD) with a considerable negative impact on their quality of life. However, factors associated with measures of sleep in daily life have not been investigated before nor has the association between sleep and the ability to engage in physical activity on a day-to-day basis been studied. AIMS: To provide insight into the relationship between actigraphic sleep measures and disease severity, exertional dyspnoea, gender and parts of the week; and to investigate the association between sleep measures and next day physical activity. METHODS: Data were analysed from 932 patients with COPD (66% male, 66.4±8.3 years, FEV1% predicted=50.8±20.5). Participants had sleep and physical activity continuously monitored using a multisensor activity monitor for a median of 6 days. Linear mixed effects models were applied to investigate the factors associated with sleep impairment and the association between nocturnal sleep and patients' subsequent daytime physical activity. RESULTS: Actigraphic estimates of sleep impairment were greater in patients with worse airflow limitation and worse exertional dyspnoea. Patients with better sleep measures (ie, non-fragmented sleep, sleeping bouts ≥225 min, sleep efficiency ≥91% and time spent awake after sleep onset <57 min) spent significantly more time in light (p<0.01) and moderate-to-vigorous physical activity (p<0.01). CONCLUSIONS: There is a relationship between measures of sleep in patients with COPD and the amount of activity they undertake during the waking day. Identifying groups with specific sleep characteristics may be useful information when designing physical activity-enhancing interventions.

Continuous Cough Monitoring Using Ambient Sound Recording During Convalescence from a COPD Exacerbation.

PURPOSE: Cough is common in chronic obstructive pulmonary disease (COPD) and is associated with frequent exacerbations and increased mortality. Cough increases during acute exacerbations (AE-COPD), representing a possible metric of clinical deterioration. Conventional cough monitors accurately report cough counts over short time periods. We describe a novel monitoring system which we used to record cough continuously for up to 45 days during AE-COPD convalescence. METHODS: This is a longitudinal, observational study of cough monitoring in AE-COPD patients discharged from a single teaching hospital. Ambient sound was recorded from two sites in the domestic environment and analysed using novel cough classifier software. For comparison, the validated hybrid HACC/LCM cough monitoring system was used on days 1, 5, 20 and 45. Patients were asked to record symptoms daily using diaries. RESULTS: Cough monitoring data were available for 16 subjects with a total of 568 monitored days. Daily cough count fell significantly from mean ± SEM 272.7 ± 54.5 on day 1 to 110.9 ± 26.3 on day 9 (p < 0.01) before plateauing. The absolute cough count detected by the continuous monitoring system was significantly lower than detected by the hybrid HACC/LCM system but normalised counts strongly correlated (r = 0.88, p < 0.01) demonstrating an ability to detect trends. Objective cough count and subjective cough scores modestly correlated (r = 0.46). CONCLUSIONS: Cough frequency declines significantly following AE-COPD and the reducing trend can be detected using continuous ambient sound recording and novel cough classifier software. Objective measurement of cough frequency has the potential to enhance our ability to monitor the clinical state in patients with COPD.

Algorithmic insights of camera-based respiratory motion extraction.

Objective. Measuring the respiratory signal from a video based on body motion has been proposed and recently matured in products for contactless health monitoring. The core algorithm for this application is the measurement of tiny chest/abdominal motions induced by respiration (i.e. capturing sub-pixel displacement caused by subtle motion between subsequent video frames), and the fundamental challenge is motion sensitivity. Though prior art reported on the validation with real human subjects, there is no thorough or rigorous benchmark to quantify the sensitivities and boundary conditions of motion-based core respiratory algorithms.Approach. A set-up was designed with a fully-controllable physical phantom to investigate the essence of core algorithms, together with a mathematical model incorporating two motion estimation strategies and three spatial representations, leading to six algorithmic combinations for respiratory signal extraction. Their promises and limitations are discussed and clarified through the phantom benchmark.Main results. With the variation of phantom motion intensity between 0.5 mm and 8 mm, the recommended approach obtains an average precision, recall, coverage and MAE of 88.1%, 91.8%, 95.5% and 2.1 bpm in the day-light condition, and 81.7%, 90.0%, 93.9% and 4.4 bpm in the night condition.Significance. The insights gained in this paper are intended to improve the understanding and applications of camera-based respiration measurement in health monitoring. The limitations of this study stem from the used physical phantom that does not consider human factors like body shape, sleeping posture, respiratory diseases, etc., and the investigated scenario is focused on sleep monitoring, not including scenarios with a sitting or standing patient like in clinical ward and triage.

Fundamentals of Camera-PPG Based Magnetic Resonance Imaging.

In Magnetic Resonance Imaging (MRI), cardiac triggering that synchronizes data acquisition with cardiac contractions is an essential technique for acquiring high-quality images. Triggering is typically based on the Electrocardiogram (ECG) signal (e.g. R-peak). Since ECG acquisition involves extra workflow steps like electrode placement and ECG signals are usually disturbed by magnetic fields in high Magnetic Resonance (MR) systems, we explored camera-based photoplethysmography (PPG) as an alternative. We used the in-bore camera of a clinical MR system to investigate the feasibility and challenges of camera-based cardiac triggering. Data from ECG, finger oximeter and camera were synchronously collected. Compared to finger-PPG, camera-based PPG provides a higher availability of the signal and the PPG marker delay relative to the ECG R-peak is considerably less with a camera monitoring the forehead. The insights obtained in this study provide a basis for an envisioned system-design phase.

Modified Camera Setups for Day-and-Night Pulse-rate Monitoring.

Camera systems have been studied as a means for ubiquitous remote photoplethysmography. It was first considered for daytime applications using ambient light. However, main applications for continuous monitoring are in dark or low-light conditions (e.g. sleep monitoring) and, more recently, suitable light sources and simple camera adaptations have been considered for infrared-based solutions. This paper explores suitable camera configurations for pulse-rate monitoring during both day and night (24/7). Various configurations differing in the recorded spectral range are defined, i.e. straight-forward adaptations of a standard RGB camera by choosing proper optical filters. These systems have been studied in a benchmark involving day and night monitoring with various degrees of motion disturbances. The results indicate that, for the 24/7 monitoring, it is best to deploy the full spectral band of an RGB camera, and this can be done without compromising the monitoring performance at night.

Nighttime features derived from topic models for classification of patients with COPD.

Nighttime symptoms are important indicators of impairment for many diseases and particularly for respiratory diseases such as chronic obstructive pulmonary disease (COPD). The use of wearable sensors to assess sleep in COPD has mainly been limited to the monitoring of limb motions or the duration and continuity of sleep. In this paper we present an approach to concisely describe sleep patterns in subjects with and without COPD. The methodology converts multimodal sleep data into a text representation and uses topic modeling to identify patterns across the dataset composed of more than 6000 assessed nights. This approach enables the discovery of higher level features resembling unique sleep characteristics that are then used to discriminate between healthy subjects and those with COPD and to evaluate patients' disease severity and dyspnea level. Compared to standard features, the discovered latent structures in nighttime data seem to capture important aspects of subjects sleeping behavior related to the effects of COPD and dyspnea.

Modified RGB Cameras for Infrared Remote-PPG.

Multi-wavelength cameras play an essential role in remote photoplethysmography (PPG). Whereas these are readily available for visible light, this is not the case for near infrared (NIR). We propose to modify existing RGB cameras to make them suited for NIR-PPG. In particular, we exploit the spectral leakage of the RGB channels in infrared in combination with a narrow dual-band optical filter. Such camera modification is simple, cost-effective, easy to implement, and it is shown to attain a pulse-rate extraction performance comparable to that of multiple narrow-band NIR cameras.

Discriminative Signatures for Remote-PPG.

Near-infrared (NIR) remote photoplethysmography (PPG) promises attractive applications in darkness, as it involves unobtrusive, invisible light. However, since the PPG strength (AC/DC) is much lower in the NIR spectrum than in the RGB spectrum, robust vital signs monitoring is more challenging. In this paper, we propose a new PPG-extraction method, DIScriminative signature based extraction (DIS), to significantly improve the pulse-rate measurement in NIR. Our core idea is to use both the color signals containing blood absorption variations and additional disturbance signals as input for PPG extraction. By defining a discriminative signature, we use one-step least-squares regression (joint optimization) to retrieve the pulsatile component from color signals and suppress disturbance signals simultaneously. A large-scale lab experiment, recorded in NIR with heavy body motions, shows the significant improvement of DIS over the state-of-the-art method, whereas its principle is simple and generally applicable.

Full video pulse extraction.

This paper introduces a new method to automate heart-rate detection using remote photoplethysmography (rPPG). The method replaces the commonly used region of interest (RoI) detection and tracking, and does not require initialization. Instead, it combines a number of candidate pulse-signals computed in the parallel, each biased towards differently colored objects in the scene. The method is based on the observation that the temporally averaged colors of video objects (skin and background) are usually quite stable over time in typical application-driven scenarios, such as the monitoring of a subject sleeping in bed, or an infant in an incubator. The resulting system, called full video pulse extraction (FVP), allows the direct use of raw video streams for pulse extraction. Our benchmark set of diverse videos shows that FVP enables long-term sleep monitoring in visible light and in infrared, and works for adults and neonates. Although we only demonstrate the concept for heart-rate monitoring, we foresee the adaptation to a range of vital signs, thus benefiting the larger video health monitoring field.

Single Element Remote-PPG.

Camera-based remote photoplethysmography technology (remote-PPG) has shown great potential for contactless pulse-rate monitoring. However, remote-PPG systems typically analyze face images, which may restrict applications in view of privacy-preserving regulations such as the recently announced General Data Protection Regulation in the European Union. In this paper, we investigate the case of using single-element sensing as an input for remote-PPG extraction, which prohibits facial analysis and thus evades privacy issues. It also improves the efficiency of data storage and transmission. In contrast to known remote-PPG solutions using skin-selection techniques, the input signals in a single-element setup will contain a non-negligible degree of signal components associated with non-skin areas. Current remote-PPG extraction methods based on physiological and optical properties of skin reflections are therefore no longer valid. A new remote-PPG method, named Soft Signature based extraction (SoftSig), is proposed to deal with this situation by softening the dependence of pulse extraction on prior knowledge. A large scale experiment validates the concept of single-element remote-PPG monitoring and shows the improvement of SoftSig over general purpose solutions.

Robust heart rate from fitness videos.

Remote photoplethysmography (rPPG) enables contactless heart-rate monitoring using a regular video camera. OBJECTIVE: This paper aims to improve the rPPG technology targeting continuous heart-rate measurement during fitness exercises. The fundamental limitation of the existing (multi-wavelength) rPPG methods is that they can suppress at most n - 1 independent distortions by linearly combining n wavelength color channels. Their performance are highly restricted when more than n - 1 independent distortions appear in a measurement, as typically occurs in fitness applications with vigorous body motions. APPROACH: To mitigate this limitation, we propose an effective yet very simple method that algorithmically extends the number of possibly suppressed distortions without using more wavelengths. Our core idea is to increase the degrees-of-freedom of noise reduction by decomposing the n wavelength camera-signals into multiple orthogonal frequency bands and extracting the pulse-signal per band-basis. This processing, namely Sub-band rPPG (SB), can suppress different distortion-frequencies using independent combinations of color channels. MAIN RESULTS: A challenging fitness benchmark dataset is created, including 25 videos recorded from 7 healthy adult subjects (ages from 25 to 40 yrs; six male and one female) running on a treadmill in an indoor environment. Various practical challenges are simulated in the recordings, such as different skin-tones, light sources, illumination intensities, and exercising modes. The basic form of SB is benchmarked against a state-of-the-art method (POS) on the fitness dataset. Using non-biased parameter settings, the average signal-to-noise-ratio (SNR) for POS varies in [-4.18, -2.07] dB, for SB varies in [-1.08, 4.77] dB. The ANOVA test shows that the improvement of SB over POS is statistically significant for almost all settings (p-value <0.05). SIGNIFICANCE: The results suggest that the proposed SB method considerably increases the robustness of heart-rate measurement in challenging fitness applications, and outperforms the state-of-the-art method.

Amplitude-selective filtering for remote-PPG.

Biometric signatures of remote photoplethysmography (rPPG), including the pulse-induced characteristic color absorptions and pulse frequency range, have been used to design robust algorithms for extracting the pulse-signal from a video. In this paper, we look into a new biometric signature, i.e., the relative pulsatile amplitude, and use it to design a very effective yet computationally low-cost filtering method for rPPG, namely "amplitude-selective filtering" (ASF). Based on the observation that the human relative pulsatile amplitude varies in a specific lower range as a function of RGB channels, our basic idea is using the spectral amplitude of, e.g., the R-channel, to select the RGB frequency components inside the assumed pulsatile amplitude-range for pulse extraction. Similar to band-pass filtering (BPF), the proposed ASF can be applied to a broad range of rPPG algorithms to pre-process the RGB-signals before extracting the pulse. The benchmark in challenging fitness use-cases shows that applying ASF (ASF+BPF) as a pre-processing step brings significant and consistent improvements to all multi-channel pulse extraction methods. It improves different (multi-wavelength) rPPG algorithms to the extent where quality differences between the individual approaches almost disappear. The novelty of the proposed method is its simplicity and effectiveness in providing a solution for the extremely challenging application of rPPG to a fitness setting. The proposed method is easy to understand, simple to implement, and low-cost in running. It is the first time that the physiological property of pulsatile amplitude is used as a biometric signature for generic signal filtering.

Algorithmic Principles of Remote PPG.

This paper introduces a mathematical model that incorporates the pertinent optical and physiological properties of skin reflections with the objective to increase our understanding of the algorithmic principles behind remote photoplethysmography (rPPG). The model is used to explain the different choices that were made in existing rPPG methods for pulse extraction. The understanding that comes from the model can be used to design robust or application-specific rPPG solutions. We illustrate this by designing an alternative rPPG method, where a projection plane orthogonal to the skin tone is used for pulse extraction. A large benchmark on the various discussed rPPG methods shows that their relative merits can indeed be understood from the proposed model.

Identifying Physical Activity Profiles in COPD Patients Using Topic Models.

With the growing amount of physical activity (PA) measures, the need for methods and algorithms that automatically analyze and interpret unannotated data increases. In this paper, PA is seen as a combination of multimodal constructs that can cooccur in different ways and proportions during the day. The design of a methodology able to integrate and analyze them is discussed, and its operation is illustrated by applying it to a dataset comprising data from COPD patients and healthy subjects acquired in daily life. The method encompasses different stages. The first stage is a completely automated method of labeling low-level multimodal PA measures. The information contained in the PA labels are further structured using topic modeling techniques, a machine learning method from the text processing community. The topic modeling discovers the main themes that pervade a large set of data. In our case, topic models discover PA routines that are active in the assessed days of the subjects under study. Applying the designed algorithm to our data provides new learnings and insights. As expected, the algorithm discovers that PA routines for COPD patients and healthy subjects are substantially different regarding their composition and moments in time in which transitions occur. Furthermore, it shows consistent trends relating to disease severity as measured by standard clinical practice.