Evaluation of Respiratory Sounds Using Image-Based Approaches for Health Measurement Applications.

Goal: The evaluation of respiratory events using audio sensing in an at-home setting can be indicative of worsening health conditions. This paper investigates the use of image-based transfer learning applied to five audio visualizations to evaluate three classification tasks (C1: wet vs. dry vs. whooping cough vs. restricted breathing; C2: wet vs. dry cough; C3: cough vs. restricted breathing). Methods: The five visualizations (linear spectrogram, logarithmic spectrogram, Mel-spectrogram, wavelet scalograms, and aggregate images) are applied to a pre-trained AlexNet image classifier for all tasks. Results: The aggregate image-based classifier achieved the highest overall performance across all tasks with C1, C2, and C3 having testing accuracies of 0.88, 0.88, and 0.91 respectively. However, the Mel-spectrogram method had the highest testing accuracy (0.94) for C2. Conclusions: The classification of respiratory events using aggregate image inputs to transfer learning approaches may help healthcare professionals by providing information that would otherwise be unavailable to them.

Implementation of smart supportive dementia technology in a hospital transitional care setting using human-centred design.

Supportive smart home technology, for older adults living with dementia and their informal care partners, has shown some benefits in private homes. In this study, a supportive smart home system is being implemented in a hospital alternative level of care setting. This case report describes how a team of researchers and healthcare managers are navigating the complexities of a hospital setting, using human-centred design and implementation strategies, to facilitate the implementation and adoption of the technology.

Automated vs. manual pain coding and heart rate estimations based on videos of older adults with and without dementia.

INTRODUCTION: Technological advances have allowed for the estimation of physiological indicators from video data. FaceReader™ is an automated facial analysis software that has been used widely in studies of facial expressions of emotion and was recently updated to allow for the estimation of heart rate (HR) using remote photoplethysmography (rPPG). We investigated FaceReader™-based heart rate and pain expression estimations in older adults in relation to manual coding by experts. METHODS: Using a video dataset of older adult patients with and without dementia, we assessed the relationship between FaceReader's™ HR estimations against a well-established Video Magnification (VM) algorithm during baseline and pain conditions. Furthermore, we examined the correspondence between the Facial Action Coding System (FACS)-based pain scores obtained through FaceReader™ and manual coding. RESULTS: FaceReader's™ HR estimations were correlated with VM algorithm in baseline and pain conditions. Non-verbal FaceReader™ pain scores and manual coding were also highly correlated despite discrepancies between the FaceReader™ and manual coding in the absolute value of scores based on pain-related facial action coding of the events preceding and following the pain response. CONCLUSIONS: Compared to expert manual FACS coding and optimized VM algorithm, FaceReader™ showed good results in estimating HR values and non-verbal pain scores.

Smart home technology solution for night-time wandering in persons with dementia.

INTRODUCTION: More than half of persons with dementia will experience night-time wandering, increasing their risk of falls and unattended home exits. This is a major predictor of caregiver burnout and one of the major causes of early institutionalization. METHODS: Using smart home technologies such as sensors, smart bulbs, pressure mats and speakers, the Night-time Wandering Detection and Diversion system is designed to assist caregivers and persons with dementia that are at risk of wandering at night. Being placed in homes around Ottawa for a 12-week trial, the system allows caregivers to rest peacefully in the night, as it detects when the person with dementia gets out of bed and automatically provides cue lighting to guide them safely to the washroom. The system also uses prerecorded audio prompts, if they venture from the bedroom, only waking the caregiver when the person with dementia opens an exit door. RESULTS: Thus far, the average depression and anxiety in caregivers have been improved after the 12 weeks, and most have said that they sleep more peacefully. CONCLUSION: The system has proven successful in supporting the safety of persons with dementia as well as their caregivers.

Novel Coronavirus Cough Database: NoCoCoDa.

The current pandemic associated with the novel coronavirus (COVID-19) presents a new area of research with its own set of challenges. Creating unobtrusive remote monitoring tools for medical professionals that may aid in diagnosis, monitoring and contact tracing could lead to more efficient and accurate treatments, especially in this time of physical distancing. Audio based sensing methods can address this by measuring the frequency, severity and characteristics of the COVID-19 cough. However, the feasibility of accumulating coughs directly from patients is low in the short term. This article introduces a novel database (NoCoCoDa), which contains COVID-19 cough events obtained through public media interviews with COVID-19 patients, as an interim solution. After manual segmentation of the interviews, a total of 73 individual cough events were extracted and cough phase annotation was performed. Furthermore, the COVID-19 cough is typically dry but can present as a more productive cough in severe cases. Therefore, an investigation of cough sub-type (productive vs. dry) of the NoCoCoDa was performed using methods previously published by our research group. Most of the NoCoCoDa cough events were recorded either during or after a severe period of the disease, which is supported by the fact that 77% of the COVID-19 coughs were classified as productive based on our previous work. The NoCoCoDa is designed to be used for rapid exploration and algorithm development, which can then be applied to more extensive datasets and potentially real time applications. The NoCoCoDa is available for free to the research community upon request.

Semi-Autonomous Vehicles as a Cognitive Assistive Device for Older Adults.

Losing the capacity to drive due to age-related cognitive decline can have a detrimental impact on the daily life functioning of older adults living alone and in remote areas. Semi-autonomous vehicles (SAVs) could have the potential to preserve driving independence of this population with high health needs. This paper explores if SAVs could be used as a cognitive assistive device for older aging drivers with cognitive challenges. We illustrate the impact of age-related changes of cognitive functions on driving capacity. Furthermore, following an overview on the current state of SAVs, we propose a model for connecting cognitive health needs of older drivers to SAVs. The model demonstrates the connections between cognitive changes experienced by aging drivers, their impact on actual driving, car sensors' features, and vehicle automation. Finally, we present challenges that should be considered when using the constantly changing smart vehicle technology, adapting it to aging drivers and vice versa. This paper sheds light on age-related cognitive characteristics that should be considered when developing future SAVs manufacturing policies which may potentially help decrease the impact of cognitive change on older adult drivers.

Comparison of Silence Removal Methods for the Identification of Audio Cough Events.

Sensing technologies are embedded in our everyday lives. Smart homes typically use an Audio Virtual Assistant (AVA) (e.g. Alexa, Siri, and Google Home) interface that collects sensor information, which can provide security, assist in everyday activities and monitor health related information. One such measure is cough, changes of which can be a marker of worsening conditions for many respiratory diseases. Creating a reliable monitoring system utilizing technology that may already be present in the home (i.e. AVA) may provide an opportunity for early intervention and reductions in the number of long-term hospitalizations. This paper focuses on the optimization of the silence removal and segmentation step in an at home setting with low to moderate background noise to identify cough events. Three commonly used methods (Standard deviation (SD), Short-term Energy (SE), Zero-crossing rate (ZCR)) were compared to manual segmentations. Each method was applied to 209 audio files that were manually verified to contain at least one cough event and the average segmentation accuracy, over segmentation and under segmentation results were compared. The ZCR method had the highest accuracy (89%); however, it completely failed under moderate noise conditions. The SD method had the best combination of accuracy (86%), ability to perform under noisy conditions and low prevalence of over and under segmentation (22% and 15% respectively). Therefore, we recommend using an adaptive approach to silence removal among cough events based on the level of background noise (i.e use the ZCR method when the background noise is low and the SD method when it is higher) prior to implementation of a cough classification system.

Examining the Effect of Noise on Biosignal Estimates Extracted through Spatio-Temporal Video Processing.

Spatio-temporal video processing has been used to extract subject vital signals from optical video and, more recently, thermal video. Thermal video, in conjunction with spatio-temporal video processing can extract biosignals optical video cannot; namely temperature data, but also biosignals in poor light conditions. Video processing involves many system parameters that can result in false biosignal reporting. This paper aimed to robustly test spatio-temporal processing algorithms and determine patterns with respect to increasing noise levels. Over 500 simulated thermal videos were generated at 29 different signal frequencies representing heart rates. These videos were contaminated with 18 different levels of Gaussian noise and were used as inputs to the algorithmic system. The algorithmic system processed each video at 6 different filter widths. The results were examined individually and as a collective. Individual results were as expected; the processing resulted in an accurate heart rate estimate if the original signal was inside the filter passband. If the signal was outside of the filter passband, the processing simply amplified noise. These same patterns were observed in the cumulative results, in addition to overarching patterns with respect to noise. Two main patterns were observed; a failure threshold was determined and quantified and a pattern of error behavior beyond this threshold was quantified. The failure threshold occurred at a noise variance of approximately 500, and around this parameter value, all detected signal frequencies were approaching 1.5 Hz (90bpm). This study was able to characterize patterns of failure, which helps to prevent future false reporting.

Implementation of a Brain Training Pilot Study For People With Mild Cognitive Impairment.

OBJECTIVE: A pilot study to determine the feasibility of recruiting patients with MCI to test for cognitive interventions. METHOD: Thirty patients with amnestic MCI were to be divided into two intervention arms and one control group. Participants went to local sites and completed brain training for one hour three times per week for nine weeks. Outcome measures were: recruitment, computer abilities, compliance, task performance, neuropsychological tests, and electroencephalography. RESULTS: After six months, only 20 participants had been recruited. Seventeen were allocated to one of the two intervention groups. Compliance was good and computer skills were not an obstacle. Participants improved their abilities in the modules, but there were no statistically significant changes on neuropsychological tests or EEG. CONCLUSIONS: Recruitment of MCI participants for extensive cognitive intervention is challenging, but achievable. This pilot study was not powered to detect clinical changes. Future trials should consider recruitment criteria, intervention duration, scheduling, and study location.

Naturalistic Driving: A Framework and Advances in Using Big Data.

Driving is an activity that facilitates physical, cognitive, and social stimulation in older adults, ultimately leading to better physical and cognitive health. However, aging is associated with declines in vision, physical health, and cognitive health, all of which can affect driving ability. One way of assessing driving ability is with the use of sensors in the older adult's own vehicle. This paper provides a framework for driving assessment and addresses how naturalistic driving studies can assist in such assessments. The framework includes driving characteristics (how much driving, speed, position, type of road), actions and reactions (lane changes, intersections, passing, merging, traffic lights, pedestrians, other vehicles), destinations (variety and distance, sequencing and route planning), and driving conditions (time of day and season). Data from a subset of Ottawa drivers from the Candrive study is used to illustrate the use of naturalistic driving data. Challenges in using naturalistic driving big data and the changing technology in vehicles are discussed.

Long term monitoring of a pressure ulcer risk patient using thermal images.

Patients in an immobile state are susceptible to pressure ulcers, which are localized injuries to the skin and/or underlying tissues due to prolonged pressure. This paper builds upon a body of work examining in-hospital older adult patients at-risk of developing pedal pressure ulcers by examining thermal images of one patient, who was reporting pain in her right foot, over 112 days of a hospital stay. Thermal images of the patient's left and right heels and malleoli were subjected to image processing to remove noise and enhance contrast, region selection and feature extraction to observe changes in temperature over time. Mean intensity within each ROI was extracted, and the difference in temperature between the left and right heels was calculated over time. The resulting temperature pattern was consistent with the physical phenomenon related to ulcer development, intervention and recovery; the right heel was similar in temperature when starting the study and at the end of the study, but was drastically warmer when experiencing erythema and drastically colder when experiencing ischaemia. These results suggest that consistent thermal imaging, in conjunction with image processing may be able to detect the formation of pressure ulcers faster than can be visually observed. Early detection of pressure ulcers is critical in the prevention of pressure ulcers, and is of great importance to any hospital or nursing home.

Comparison of motion-based analysis to thermal-based analysis of thermal video in the extraction of respiration patterns.

Non-contact methods of extracting vital signals has become a popular area of research. This is likely due to the world's aging population and the increased need for long term and remote monitoring. This paper examines and compares the potential for one modality to capture a vital sign, specifically respiration, in the presence of signal abnormalities. This paper compares temperature based-methods to motion-based methods of extracting respiration rate from thermal video of a subject performing computationally difficult respiration tests. The thermal video was subjected to segmentation-based image processing and region tracking to encompass temperature changes over time. All methods were successful in identifying regular breathing and the absence of breathing, but differed in performance identifying hyperventilation and obstructive sleep apnea simulated breathing. The temperature-based method better depicted airflow volume, while the motion-based method better depicted absence of breath and chest movement; neither signal on its own was able to accurately depict OSA breathing. These results suggest that the fusion of information from different physical phenomenon (i.e. motion and temperature) is important here in detecting abnormal breathing patterns, but also in the detection of all vital signals, adding algorithmic robustness in the presence of signal abnormalities.

Event-related potentials elicited during working memory are altered in mild cognitive impairment.

Persons with Mild Cognitive Impairment (MCI) can experience deficits in working memory. In the present study, we investigated working memory in persons with MCI and cognitively healthy older adults using event-related potentials (ERPs). Participants performed an n-back working memory task with baseline (0-back), low load (1-back), and high load (2-back) working memory conditions. MCI participants' performance was less accurate than that of healthy older adults in both the 1-back and 2-back conditions, and reaction times were longer in MCI than control participants in the 0-back, 1-back and 2-back conditions. ERP analyses revealed delayed P200 and N200 latencies and smaller P300 amplitudes in MCI relative to control participants in the 0-back, 1-back and 2-back conditions. Deterioration in working memory performance concomitant with marked electrophysiological alterations suggests that persons with MCI exhibit deficits in several cognitive processes that include early attention, stimulus discrimination and classification, and updating and manipulation of information held in working memory.

P300 amplitude alterations during inhibitory control in persons with Mild Cognitive Impairment.

Deficits in executive function are highly noticeable in Alzheimer's disease, and recent behavioral studies have shown that such deficits - particularly during inhibitory control - can also be found in persons with Mild Cognitive Impairment (MCI). Thus, the objective of this study was to investigate behavioral and electrophysiological correlates of inhibitory control in persons with MCI. A group of persons with MCI and a group healthy older adults performed a Go/NoGo task while electroencephalogram was recorded. Our results revealed that persons with MCI performed less accurately than healthy controls during the Go and NoGo conditions. In addition, we found reduced P300 amplitudes during Go and NoGo conditions relative to healthy older adults. Our results suggest that neurocognitive mechanisms associated with target detection and evaluation (Go P300) and response inhibition (NoGo P300) are compromised in persons with MCI.

The use of a thermal camera and Eulerian enhancement in the examination of pedal pulse and microvascular health.

Early detection of impaired blood flow and microvascular functioning is important to prevent ulceration in diabetic patients. This paper aims to first determine if thermal video in conjunction with Eulerian Video Magnification (EVM) can be used to find the pedal pulse rate, and reveal patterns indicative of the foot's microvascular health. Thermal video was captured of a healthy adult's foot while a Doppler ultrasound captured pedal pulse. Another thermal video was captured of a patient's heels. These videos were subjected to EVM, areas of interest were defined and the mean intensity signal was calculated temporally, within each defined area. The healthy adult signals were compared to Doppler data to determine the signal best representative of pedal pulse. The patient signals were examined for patterns. The mean intensity signals best representing pedal pulse in the healthy adult resulted from areas containing an artery close to the skin. The most significant pattern in the patient data was a large difference in signal amplitude from areas containing the left posterior tibial artery and the right; the left, colder heel had a weaker signal amplitude. These results suggest that thermal video subjected to EVM can reveal the pedal pulse rate by extracting intensity signals from areas in which arteries are close to the skin, and may reveal differences in the microvascular health of the left versus right foot. The ability to detect pedal pulse and differences in microvascular health using an inexpensive and non-intrusive thermal camera would of great value to a podiatric clinic.

Smart monitoring of fluid intake and bladder voiding using pressure sensitive mats.

Pressure sensitive mats have been used in noninvasive smart monitoring for a variety of problems including breathing rate monitoring, sleep monitoring, mobility, and weight. This paper describes a proof of concept application of pressure mats to monitor fluid intake/output (fluid cycle) events during the night. The ability to more accurately track such events has potential implications for monitoring those individuals who have nocturia, a condition where a person wakes at night to urinate. Data were collected from a healthy young female subject instructed to drink as much water as was comfortable (700mL) and lie in a supine position on a mattress located directly on three pressure mats. This was compared to an initial data set collected immediately after voiding but before drinking, 30 minutes after drinking, 60 minutes after drinking and a final data set after again voiding the bladder. The additional pressure from the 700mL of water was detectible and tracked over the course of the hour-long testing session under idealized conditions. This provides a proof-of-concept that nocturnal fluid intake and bladder voiding events can be tracked using non-invasive pressure-sensitive mats, however additional testing and development is required to achieve a deployable monitoring system.

The detection of breathing behavior using Eulerian-enhanced thermal video.

The current gold standard for detecting and distinguishing between types of sleep apnea is expensive and invasive. This paper aims to examine the potential of inexpensive and unobtrusive thermal cameras in the identification and distinction between types of sleep apnea. A thermal camera was used to gather video of a subject performing regular nasal breathing, nasal hyperventilation and an additional trial simulating one type of sleep apnea. Simultaneously, a respiratory inductance plethysmography (RIP) band gathered respiratory data. Thermal video of all three trials were subjected to Eulerian Video Magnification; a procedure developed at MIT for enhancing subtle color variations in video data. Post magnification, nasal regions of interest were defined and mean region intensities were found for each frame of each trial. These signals were compared to determine the best performing region and compared to RIP data to validate breathing behavior. While some regions performed better, all region intensity signals depicted correct breathing behavior. The mean intensity signals for normal breathing and hyperventilation were correct and correlated well with RIP data. Furthermore, the RIP data resulting from the sleep apnea simulation clearly depicted chest movement while the corresponding mean intensity signal depicted lack of cyclical air flow. These results indicate that a subject's breathing behavior can be captured using thermal video and suggest that, with further development and additional equipment, thermal video can be used to detect and distinguish between types of sleep apnea.

Using integrated bio-physiotherapy informatics in home health-care settings: A qualitative analysis of a point-of-care decision support system.

The growing need to gain efficiencies within a home care setting has prompted home care practitioners to focus on health informatics to address the needs of an aging clientele. The remote and heterogeneous nature of the home care environment necessitates the use of non-intrusive client monitoring and a portable, point-of-care graphical user interface. Using a grounded theory approach, this article examines the simulated use of a graphical user interface by practitioners in a home care setting to explore the salient features of monitoring the activity of home care clients. The results demonstrate the need for simple, interactive displays that can provide large amounts of geographical and temporal data relating to patient activity. Additional emerging themes from interviews indicate that home care professionals would use a graphical user interface of this type for patient education and goal setting as well as to assist in the decision-making process of home care practitioners.

Analyzing center of pressure progression during bed exits.

This paper presents a new approach for analyzing center of pressure (COP) progression using pressure data collected from a pressure-sensitive array placed under the bed mattress. Pressure data were collected from a young female participant who was healthy and an older 78 year old female participant who had a history of falls. Information relevant to movement direction, time, path trajectory, magnitude and frequency was presented in three dimensional plots and color differentiated displays. When tested on data collected from an older participant who experienced a fall, this method of analyzing COP was able to illustrate distinct differences in bed exit patterns used pre and post fall episode. This analysis approach shows the potential to detect changes in bed exit patterns indicative of a critical health event. Future applications include home monitoring to assist with early intervention in the event of bed mobility decline.

Measurement of signal use and vehicle turns as indication of driver cognition.

This paper uses data analytics to provide a method for the measurement of a key driving task, turn signal usage as a measure of an automatic over-learned cognitive function drivers. The paper augments previously reported more complex executive function cognition measures by proposing an algorithm that analyzes dashboard video to detect turn indicator use with 100% accuracy without any false positives. The paper proposes two algorithms that determine the actual turns made on a trip. The first through analysis of GPS location traces for the vehicle, locating 73% of the turns made with a very low false positive rate of 3%. A second algorithm uses GIS tools to retroactively create turn by turn directions. Fusion of GIS and GPS information raises performance to 77%. The paper presents the algorithm required to measure signal use for actual turns by realigning the 0.2Hz GPS data, 30fps video and GIS turn events. The result is a measure that can be tracked over time and changes in the driver's performance can result in alerts to the driver, caregivers or clinicians as indication of cognitive change. A lack of decline can also be shared as reassurance.