Respiratory Rate Estimation from Thermal Video Data Using Spatio-Temporal Deep Learning.

Thermal videos provide a privacy-preserving yet information-rich data source for remote health monitoring, especially for respiration rate (RR) estimation. This paper introduces an end-to-end deep learning approach to RR measurement using thermal video data. A detection transformer (DeTr) first finds the subject's facial region of interest in each thermal frame. A respiratory signal is estimated from a dynamically cropped thermal video using 3D convolutional neural networks and bi-directional long short-term memory stages. To account for the expected phase shift between the respiration measured using a respiratory effort belt vs. a facial video, a novel loss function based on negative maximum cross-correlation and absolute frequency peak difference was introduced. Thermal recordings from 22 subjects, with simultaneous gold standard respiratory effort measurements, were studied while sitting or standing, both with and without a face mask. The RR estimation results showed that our proposed method outperformed existing models, achieving an error of only 1.6 breaths per minute across the four conditions. The proposed method sets a new State-of-the-Art for RR estimation accuracy, while still permitting real-time RR estimation.

The use of a thermal camera and Eulerian enhancement in the examination of pedal pulse and microvascular health.

Early detection of impaired blood flow and microvascular functioning is important to prevent ulceration in diabetic patients. This paper aims to first determine if thermal video in conjunction with Eulerian Video Magnification (EVM) can be used to find the pedal pulse rate, and reveal patterns indicative of the foot's microvascular health. Thermal video was captured of a healthy adult's foot while a Doppler ultrasound captured pedal pulse. Another thermal video was captured of a patient's heels. These videos were subjected to EVM, areas of interest were defined and the mean intensity signal was calculated temporally, within each defined area. The healthy adult signals were compared to Doppler data to determine the signal best representative of pedal pulse. The patient signals were examined for patterns. The mean intensity signals best representing pedal pulse in the healthy adult resulted from areas containing an artery close to the skin. The most significant pattern in the patient data was a large difference in signal amplitude from areas containing the left posterior tibial artery and the right; the left, colder heel had a weaker signal amplitude. These results suggest that thermal video subjected to EVM can reveal the pedal pulse rate by extracting intensity signals from areas in which arteries are close to the skin, and may reveal differences in the microvascular health of the left versus right foot. The ability to detect pedal pulse and differences in microvascular health using an inexpensive and non-intrusive thermal camera would of great value to a podiatric clinic.

Using integrated bio-physiotherapy informatics in home health-care settings: A qualitative analysis of a point-of-care decision support system.

The growing need to gain efficiencies within a home care setting has prompted home care practitioners to focus on health informatics to address the needs of an aging clientele. The remote and heterogeneous nature of the home care environment necessitates the use of non-intrusive client monitoring and a portable, point-of-care graphical user interface. Using a grounded theory approach, this article examines the simulated use of a graphical user interface by practitioners in a home care setting to explore the salient features of monitoring the activity of home care clients. The results demonstrate the need for simple, interactive displays that can provide large amounts of geographical and temporal data relating to patient activity. Additional emerging themes from interviews indicate that home care professionals would use a graphical user interface of this type for patient education and goal setting as well as to assist in the decision-making process of home care practitioners.

Usage monitoring of electrical devices in a smart home.

Profiling the usage of electrical devices within a smart home can be used as a method for determining an occupant's activities of daily living. A nonintrusive load monitoring system monitors the electrical consumption at a single electrical source (e.g., main electric utility service entry) and the operating schedules of individual devices are determined by disaggregating the composite electrical consumption waveforms. An electrical device's load signature plays a key role in nonintrusive load monitoring systems. A load signature is the unique electrical behaviour of an individual device when it is in operation. This paper proposes a feature-based model, using the real power and reactive power as features for describing the load signatures of individual devices. Experimental results for single device recognition for 7 devices show that the proposed approach can achieve 100% classification accuracy with discriminant analysis using Mahalanobis distances.

Identification of food spoilage in the smart home based on neural and fuzzy processing of odour sensor responses.

Adopting the use of real-time odour monitoring in the smart home has the potential to alert the occupant of unsafe or unsanitary conditions. In this paper, we measured (with a commercial metal-oxide sensor-based electronic nose) the odours of five household foods that had been left out at room temperature for a week to spoil. A multilayer perceptron (MLP) neural network was trained to recognize the age of the samples (a quantity related to the degree of spoilage). For four of these foods, median correlation coefficients (between target values and MLP outputs) of R > 0.97 were observed. Fuzzy C-means clustering (FCM) was applied to the evolving odour patterns of spoiling milk, which had been sampled more frequently (4h intervals for 7 days). The FCM results showed that both the freshest and oldest milk samples had a high degree of membership in "fresh" and "spoiled" clusters, respectively. In the future, as advancements in electronic nose development remove the present barriers to acceptance, signal processing methods like those explored in this paper can be incorporated into odour monitoring systems used in the smart home.

Determination of sit-to-stand transfer duration using bed and floor pressure sequences.

The duration of a sit-to-stand (SiSt) transfer is a representative measure of a person's status of physical mobility. This paper measured the duration unobtrusively and automatically using a pressure sensor array under a bed mattress and a floor plate beside the bed. Pressure sequences were extracted from frames of sensor data measuring bed and floor pressure over time. The start time was determined by an algorithm based on the motion of the center of pressure (COP) on the mattress toward the front edge of the bed. The end time was determined by modeling the foot pressure exerted on the floor in the wavelet domain as the step response of a third-order transfer function. As expected, young and old healthy adults generated shorter SiSt durations of around 2.31 and 2.88 s, respectively, whereas post-hip fracture and post-stroke adults produced longer SiSt durations of around 3.32 and 5.00 s. The unobtrusive nature of pressure sensing techniques used in this paper provides valuable information that can be used for the ongoing monitoring of patients within extended-care facilities or within the smart home environment.

An investigation into the suitability of using three electronic nose instruments for the detection and discrimination of bacteria types.

The use of electronic nose (e-nose) technology for detection of food-borne bacteria has several practical advantages over current laboratory procedures, such as lower cost and reduced testing time. In this work, we are interested in using electronic nose systems to detect E. coli and Listeria in a nutrient broth, and discriminate between these bacteria types at various concentrations. To do this, we use instruments based on three different technologies - fingerprint mass spectrometry, metal oxide sensors, and conductive polymer sensors. Our results indicate that separation between groups can be achieved. We describe the relative merits and drawbacks of each technology and discuss how this rich multimodal dataset can be used to build a classification system.