Occupancy Estimation from Blurred Video: A Multifaceted Approach with Privacy Consideration.

Building occupancy information is significant for a variety of reasons, from allocation of resources in smart buildings to responding during emergency situations. As most people spend more than 90% of their time indoors, a comfortable indoor environment is crucial. To ensure comfort, traditional HVAC systems condition rooms assuming maximum occupancy, accounting for more than 50% of buildings' energy budgets in the US. Occupancy level is a key factor in ensuring energy efficiency, as occupancy-controlled HVAC systems can reduce energy waste by conditioning rooms based on actual usage. Numerous studies have focused on developing occupancy estimation models leveraging existing sensors, with camera-based methods gaining popularity due to their high precision and widespread availability. However, the main concern with using cameras for occupancy estimation is the potential violation of occupants' privacy. Unlike previous video-/image-based occupancy estimation methods, we addressed the issue of occupants' privacy in this work by proposing and investigating both motion-based and motion-independent occupancy counting methods on intentionally blurred video frames. Our proposed approach included the development of a motion-based technique that inherently preserves privacy, as well as motion-independent techniques such as detection-based and density-estimation-based methods. To improve the accuracy of the motion-independent approaches, we utilized deblurring methods: an iterative statistical technique and a deep-learning-based method. Furthermore, we conducted an analysis of the privacy implications of our motion-independent occupancy counting system by comparing the original, blurred, and deblurred frames using different image quality assessment metrics. This analysis provided insights into the trade-off between occupancy estimation accuracy and the preservation of occupants' visual privacy. The combination of iterative statistical deblurring and density estimation achieved a 16.29% counting error, outperforming our other proposed approaches while preserving occupants' visual privacy to a certain extent. Our multifaceted approach aims to contribute to the field of occupancy estimation by proposing a solution that seeks to balance the trade-off between accuracy and privacy. While further research is needed to fully address this complex issue, our work provides insights and a step towards a more privacy-aware occupancy estimation system.

Dynamic Arctangent Center-Tracking Method for Respiratory Displacement Monitoring of Subjects in Arbitrary Positions.

Accurate continuous measurement of respiratory displacement using continuous wave Doppler radar requires rigorous management of dc offset which changes when a subject changes distance from the radar measurement system. Effective measurement, therefore, requires robust dynamic calibration which can recognize and compensate for changes in the nominal position of a subject. In this paper, a respiratory displacement measurement algorithm is proposed which can differentiate between sedentary and non-sedentary conditions and continuously adapt to provide long-term monitoring of a subject's sedentary respiration. Arctangent demodulation is an effective means of quantifying continuous displacement using a quadrature Doppler radar, yet it depends on accurate identification of dc offset and dc information contributions in the radar I-Q arc with the subject in a particular position. The dynamic calibration method proposed here is demonstrated to differentiate between sedentary and non-sedentary conditions for six subjects to produce accurate sedentary respiration measurements even when the subject arbitrarily changes position, once the appropriate thresholds are established for the measurement environment.

Doppler radar remote sensing of respiratory function.

Doppler radar remote sensing of torso kinematics can provide an indirect measure of cardiopulmonary function. Motion at the human body surface due to heart and lung activity has been successfully used to characterize such measures as respiratory rate and depth, obstructive sleep apnea, and even the identity of an individual subject. For a sedentary subject, Doppler radar can track the periodic motion of the portion of the body moving as a result of the respiratory cycle as distinct from other extraneous motions that may occur, to provide a spatial temporal displacement pattern that can be combined with a mathematical model to indirectly assess quantities such as tidal volume, and paradoxical breathing. Furthermore, it has been demonstrated that even healthy respiratory function results in distinct motion patterns between individuals that vary as a function of relative time and depth measures over the body surface during the inhalation/exhalation cycle. Potentially, the biomechanics that results in different measurements between individuals can be further exploited to recognize pathology related to lung ventilation heterogeneity and other respiratory diagnostics.

Unobtrusive occupancy and vital signs sensing for human building interactive systems.

Cognitive buildings use data on how occupants respond to the built environment to proactively make occupant-centric adjustments to lighting, temperature, ventilation, and other environmental parameters. However, sensors that unobtrusively and ubiquitously measure occupant responses are lacking. Here we show that Doppler-radar based sensors, which can sense small physiological motions, provide accurate occupancy detection and estimation of vital signs in challenging, realistic circumstances. Occupancy was differentiated from an empty room over 93% of the time in a 3.4 m × 8.5 m conference room with a single sensor in both wall and ceiling-mounted configurations. Occupancy was successfully detected while an occupant was under the table, visibly blocked from the sensor, a scenario where infrared, ultrasound, and video-based occupancy sensors would fail. Heart and respiratory rates were detected in all seats in the conference room with a single ceiling-mounted sensor. The occupancy sensor can be used to control HVAC and lighting with a short, 1-2 min delay and to provide information for space utilization optimization. Heart and respiratory rate sensing could provide additional feedback to future human-building interactive systems that use vital signs to determine how occupant comfort and wellness is changing with time.

The field of human building interaction for convergent research and innovation for intelligent built environments.

Human-Building Interaction (HBI) is a convergent field that represents the growing complexities of the dynamic interplay between human experience and intelligence within built environments. This paper provides core definitions, research dimensions, and an overall vision for the future of HBI as developed through consensus among 25 interdisciplinary experts in a series of facilitated workshops. Three primary areas contribute to and require attention in HBI research: humans (human experiences, performance, and well-being), buildings (building design and operations), and technologies (sensing, inference, and awareness). Three critical interdisciplinary research domains intersect these areas: control systems and decision making, trust and collaboration, and modeling and simulation. Finally, at the core, it is vital for HBI research to center on and support equity, privacy, and sustainability. Compelling research questions are posed for each primary area, research domain, and core principle. State-of-the-art methods used in HBI studies are discussed, and examples of original research are offered to illustrate opportunities for the advancement of HBI research.

Heart rate detection using single-channel Doppler radar system.

A number of algorithms have been developed to extract heart rate from physiological motion data with the Doppler radar system. Yet, it is very challenging to elimi-nate the noise associated with surroundings, especially with a single-channel Doppler radar system. However, single-channel Doppler radars provide the advantage of operating at lower power. Additionally, the heart rate extraction using single-channel Doppler radar has remained somewhat unexplored. This has motivated us to develop effective signal processing algorithms for signals received from single-channel Doppler radars. In this paper, we have proposed and studied three algorithms for estimating heart rate. The first algorithm is based on applying FFT on an FIR filtered signal. In the second algorithm, autocorrelation was performed on the filtered data. Thirdly, we used a peak finding algorithm in conjunction with a moving average preceded by a clipper to determine the heart rate. The results obtained were compared with the heart rate readings from a pulse oximeter. With a mean difference of 2.6 bpm, the heart rate from Doppler radar matched that from the pulse oximeter most frequently when the peak finding algorithm was used. The results obtained using autocorrelation and peak finding algorithm (with standard deviations of 2.6 bpm and 4.0 bpm) suggest that a single channel Doppler radar system can be a viable alternative to contact heart rate monitors in patients for whom contact measurements are not feasible.

Phase Correlation Single Channel Continuous Wave Doppler Radar Recognition of Multiple Sources.

Continuous-wave Doppler radar (CWDR) can be used to remotely detect physiological parameters, such as respiration and heart signals. However, detecting and separating multiple targets remains a challenging task for CWDR. While complex transceiver architectures and advanced signal processing algorithms have been demonstrated as effective for multiple target separations in some scenarios, the separation of equidistant sources within a single antenna beam remains a challenge. This paper presents an alternative phase tuning approach that exploits the diversity among target distances and physiological parameters for multi-target detection. The design utilizes a voltage-controlled analog phase shifter to manipulate the phase correlation of the CWDR and thus create different signal mixtures from the multiple targets, then separates them in the frequency domain by suppressing individual signals sequentially. We implemented the phase correlation system based on a 2.4 GHz single-channel CWDR and evaluated it against multiple mechanical and human targets. The experimental results demonstrated successful separation of nearly equidistant targets within an antenna beam, equivalent to separating physiological signals of two people seated shoulder to shoulder.

Identification of COVID-19 Type Respiratory Disorders Using Channel State Analysis of Wireless Communications Links.

One deadly aspect of COVID-19 is that those infected can often be contagious before exhibiting overt symptoms. While methods such as temperature checks and sinus swabs have aided with early detection, the former does not always provide a reliable indicator of COVID-19, and the latter is invasive and requires significant human and material resources to administer. This paper presents a non-invasive COVID-19 early screening system implementable with commercial off-the-shelf wireless communications devices. The system leverages the Doppler radar principle to monitor respiratory-related chest motion and identifies breathing rates that indicate COVID-19 infection. A prototype was developed from software-defined radios (SDRs) designed for 5G NR wireless communications and system performance was evaluated using a robotic mover simulating human breathing, and using actual breathing, resulting in a consistent respiratory rate accuracy better than one breath per minute, exceeding that used in common medical practice.Clinical Relevance-This establishes the potential efficacy of wireless communications based radar for recognizing respiratory disorders such as COVID-19.

Non-Invasive, Continuous, Pulse Pressure Monitoring Method.

Many individuals suffer from ailments such hypertension that require frequent health monitoring. Unfortunately, current technology does not possess the ability for unobtrusive, continuous monitoring. This paper proposes estimation of pulse pressure based on pulse transient time determined from one non-contact, and one contact sensor: Doppler radar for non-contact detection of heart beat, and piezoelectric finger pulse sensor. The time delay between heart beat and finger pulse was determined using MATLAB software, and pulse wave velocity (PWV) was calculated. Finally, subjects' pulse pressure estimated using PWV was found to be in good agreement with pulse pressure measured using an off the shelf sphygmomanometer by reading and taking difference of systolic and diastolic blood pressure.

Subject Count Estimation by Using Doppler Radar Occupancy Sensor.

Occupancy information and occupant counts can save significant amount of energy for occupant related building automation systems. Doppler radar occupancy detection sensor not only can detect the human presence but also has the potential to count the number of occupants. Continuous wave Doppler radar monitoring system is employed for occupant count estimation. The received signal strength (RSS) which is directly related to radar cross section is used as a measure for occupant counts. The preliminary results show RSS is a promising tool for occupant number estimation.

Separation of Respiratory Signatures for Multiple Subjects Using Independent Component Analysis with the JADE Algorithm.

Respiration monitoring using microwave Doppler radar has attracted significant interest over the last four decades due to its non-invasive and non-contact form of measurement. However, this technology is still not at the level of practical implementations in healthcare due to motion artifacts and interference from multiple subjects within the range of the Doppler radar sensor. Most reported results in literature focus only on single subject measurements because when multiple subjects are present there are interfering respiration signals which are difficult to separate as individual respiration signals. This paper investigates the feasibility of separating respiratory signatures from the multiple subjects. We employed a new approach using Independent Component Analysis (ICA) with the Joint Approximate Diagonalization of Eignematrices (JADE) algorithm to achieve this for closely spaced subjects, and the system is also capable of estimating Direction of Arrival (DOA) for well-spaced subjects. Experimental results demonstrated that the ICA-JADE method can separate respiratory signatures from two subjects one meter apart from each other at a distance from the radar of 2.89 meters. The separated respiratory pattern closely correlates with reference chest belt respiration patterns, and the mean square error is approximately 11.58%. Concisely, this paper clearly demonstrates that by integrating ICA with the JADE algorithm in a Doppler radar physiological monitoring system, multiple subjects can be monitored simultaneously.

Effects of respiration depth on human body radar cross section Using 2.4GHz continuous wave radar.

In this study, it was tested whether deep and shallow breathing has an effect on the cardiopulmonary radar cross-section (RCS). Continuous wave radar with quadrature architecture at 2.4GHz was used to test 2 human subjects breathing deep and shallow for 30 seconds each while seated 2 meters away from the radar. A retro-reflective marker was placed on the sternum of each subject and measured by infrared motion capture cameras to accurately track displacement of the chest. The quadrature radar outputs were processed to find the radius of the arc on the IQ plot using a circle-fitting algorithm. Results showed that the effective RCS ratio of deep to shallow breathing for subjects 1 and 2 was 6.99 and 2.24 respectively.

Synchrosqueezing an effective method for analyzing Doppler radar physiological signals.

Doppler radar can monitor vital sign wirelessly. Respiratory and heart rate have time-varying behavior. Capturing the rate variability provides crucial physiological information. However, the common time-frequency methods fail to detect key information. We investigate Synchrosqueezing method to extract oscillatory components of the signal with time varying spectrum. Simulation and experimental result shows the potential of the proposed method for analyzing signals with complex time-frequency behavior like physiological signals. Respiration and heart signals and their components are extracted with higher resolution and without any pre-filtering and signal conditioning.

Wrist-worn heartbeat monitoring system based on bio-impedance analysis.

In this paper, a bio-impedance analysis (BIA) based wrist-worn heartbeat monitoring system is proposed. The system is able to estimate heart rate from a subject's wrist with only four electrodes. The design is achieved with a standard BIA device and off-the-shelf components for signal conditioning. The measured heartbeat-related impedance signal is compared with a reference heart rate signal obtained from piezoelectric finger pulse transducer. The BIA results agree with the reference, which validates the feasibility of the proposed system. To the best of our knowledge, this is the first reported BIA heartbeat monitoring system in the wristband configuration.

Estimation of physiological sub-millimeter displacement with CW Doppler radar.

Doppler radar physiological sensing has been studied for non-contact detection of vital signs including respiratory and heartbeat rates. This paper presents the first micrometer resolution Wi-Fi band Doppler radar for sub-millimeter physiological displacement measurement. A continuous-wave Doppler radar working at 2.4GHz is used for the measurement. It is intended for estimating small displacements on the body surface resulting from physiological activity. A mechanical mover was used as target, and programmed to conduct sinusoidal motions to simulate pulse motions. Measured displacements were compared with a reference system, which indicates a superior performance in accuracy for having absolute errors less than 10µm, and relative errors below 4%. It indicates the feasibility of highly accurate non-contact monitoring of physiological movements using Doppler radar.

Packet radar spectrum recovery for physiological signals.

Packet Doppler radar is investigated for extracting physiological signals. System on Chip is employed as a signal source in packet mode, and it transmits signals intermittently at 2.405 GHz to save power. Reflected signals are demodulated directly by spectral analysis of received pulses in the baseband. Spectral subtraction, using data from an empty room, is applied to extract the periodic movement. It was experimentally demonstrated that frequency of the periodic motion can be accurately extracted using this technique. Proposed approach reduces the computation complexity of the signal processing part effectively.

Measuring chest circumference change during respiration with an electromagnetic biosensor.

In this paper, an off-the-shelf DC motor is modified into a chest belt and used to successfully measure circumference change on a mechanical chest model, while simultaneously harvesting significant power. Chest circumference change can provide information on tidal volume, which is vital in assessing lung function. The chest circumference change is calculated from the motor's voltage output. Calculated values are within 0.95mm of measured circumference changes, with a standard deviation of 0.37mm. The wearable motor can also harvest at least 29.4µW during normal breathing.

Piezoelectric and electromagnetic respiratory effort energy harvesters.

The movements of the torso due to normal breathing could be harvested as an alternative, and renewable power source for an ultra-low power electronic device. The same output signal could also be recorded as a physiological signal containing information about breathing, thus enabling self-powered wearable biosensors/harvesters. In this paper, the selection criteria for such a biosensor, optimization procedure, trade-offs, and challenges as a sensor and harvester are presented. The empirical data obtained from testing different modules on a mechanical torso and a human subject demonstrated that an electromagnetic generator could be used as an unobtrusive self-powered medical sensor by harvesting more power, offering reasonable amount of output voltage for rectification purposes, and detecting respiratory effort.

Considerations for integration of a physiological radar monitoring system with gold standard clinical sleep monitoring systems.

A design for a physiological radar monitoring system (PRMS) that can be integrated with clinical sleep monitoring systems is presented. The PRMS uses two radar systems at 2.45 GHz and 24 GHz to achieve both high sensitivity and high resolution. The system can acquire data, perform digital processing and output appropriate conventional analog outputs with a latency of 130 ms, which can be recorded and displayed by a gold standard sleep monitoring system, along with other standard sensor measurements.

Respiratory effort energy estimation using Doppler radar.

Human respiratory effort can be harvested to power wearable biosensors and mobile electronic devices. The very first step toward designing a harvester is to estimate available energy and power. This paper describes an estimation of the available power and energy due to the movements of the torso during breathing, using Doppler radar by detecting breathing rate, torso displacement, torso movement velocity and acceleration along the sagittal movement of the torso. The accuracy of the detected variables is verified by two reference methods. The experimental result obtained from a healthy female human subject shows that the available power from circumferential movement can be higher than the power from the sagittal movement.