Smart Multimodal In-Bed Pose Estimation Framework Incorporating Generative Adversarial Neural Network.

Monitoring in-bed pose estimation based on the Internet of Medical Things (IoMT) and ambient technology has a significant impact on many applications such as sleep-related disorders including obstructive sleep apnea syndrome, assessment of sleep quality, and health risk of pressure ulcers. In this research, a new multimodal in-bed pose estimation has been proposed using a deep learning framework. The Simultaneously-collected multimodal Lying Pose (SLP) dataset has been used for performance evaluation of the proposed framework where two modalities including long wave infrared (LWIR) and depth images are used to train the proposed model. The main contribution of this research is the feature fusion network and the use of a generative model to generate RGB images having similar poses to other modalities (LWIR/depth). The inclusion of a generative model helps to improve the overall accuracy of the pose estimation algorithm. Moreover, the method can be generalized for situations to recover human pose both in home and hospital settings under various cover thickness levels. The proposed model is compared with other fusion-based models and shows an improved performance of 97.8% at PCKh @0.5. In addition, performance has been evaluated for different cover conditions, and under home and hospital environments which present improvements using our proposed model.

A Novel Model to Generate Heterogeneous and Realistic Time-Series Data for Post-Stroke Rehabilitation Assessment.

The application of machine learning-based tele-rehabilitation faces the challenge of limited availability of data. To overcome this challenge, data augmentation techniques are commonly employed to generate synthetic data that reflect the configurations of real data. One such promising data augmentation technique is the Generative Adversarial Network (GAN). However, GANs have been found to suffer from mode collapse, a common issue where the generated data fails to capture all the relevant information from the original dataset. In this paper, we aim to address the problem of mode collapse in GAN-based data augmentation techniques for post-stroke assessment. We applied the GAN to generate synthetic data for two post-stroke rehabilitation datasets and observed that the original GAN suffered from mode collapse, as expected. To address this issue, we propose a Time Series Siamese GAN (TS-SGAN) that incorporates a Siamese network and an additional discriminator. Our analysis, using the longest common sub-sequence (LCSS), demonstrates that TS-SGAN generates data uniformly for all elements of two testing datasets, in contrast to the original GAN. To further evaluate the effectiveness of TS-SGAN, we encode the generated dataset into images using Gramian Angular Field and classify them using ResNet-18. Our results show that TS-SGAN achieves a significant accuracy increase of classification accuracy (35.2%-42.07%) for both selected datasets. This represents a substantial improvement over the original GAN.

A Generic Deep Learning Based Cough Analysis System From Clinically Validated Samples for Point-of-Need Covid-19 Test and Severity Levels.

In an attempt to reduce the infection rate of the COrona VIrus Disease-19 (Covid-19) countries around the world have echoed the exigency for an economical, accessible, point-of-need diagnostic test to identify Covid-19 carriers so that they (individuals who test positive) can be advised to self isolate rather than the entire community. Availability of a quick turn-around time diagnostic test would essentially mean that life, in general, can return to normality-at-large. In this regards, studies concurrent in time with ours have investigated different respiratory sounds, including cough, to recognise potential Covid-19 carriers. However, these studies lack clinical control and rely on Internet users confirming their test results in a web questionnaire (crowdsourcing) thus rendering their analysis inadequate. We seek to evaluate the detection performance of a primary screening tool of Covid-19 solely based on the cough sound from 8,380 clinically validated samples with laboratory molecular-test (2,339 Covid-19 positive and 6,041 Covid-19 negative) under quantitative RT-PCR (qRT-PCR) from certified laboratories. All collected samples were clinically labelled, i.e., Covid-19 positive or negative, according to the results in addition to the disease severity based on the qRT-PCR threshold cycle (Ct) and lymphocytes count from the patients. Our proposed generic method is an algorithm based on Empirical Mode Decomposition (EMD) for cough sound detection with subsequent classification based on a tensor of audio sonographs and deep artificial neural network classifier with convolutional layers called 'DeepCough'. Two different versions of DeepCough based on the number of tensor dimensions, i.e., DeepCough2D and DeepCough3D, have been investigated. These methods have been deployed in a multi-platform prototype web-app 'CoughDetect'. Covid-19 recognition results rates achieved a promising AUC (Area Under Curve) of [Formula: see text] 98 . 80 % ± 0 . 83 % , sensitivity of [Formula: see text] 96 . 43 % ± 1 . 85 % , and specificity of [Formula: see text] 96 . 20 % ± 1 . 74 % and average AUC of [Formula: see text] 81 . 08 % ± 5 . 05 % for the recognition of three severity levels. Our proposed web tool as a point-of-need primary diagnostic test for Covid-19 facilitates the rapid detection of the infection. We believe it has the potential to significantly hamper the Covid-19 pandemic across the world.

A New Spectrum Driven Index for the Assessment of ECG Signal Quality.

In this paper, a new simple index has been introduced for the assessment of electrocardiography (ECG) signal quality. In the proposed method, first, the initial spectrum of the ECG is derived by applying synchrosqueezed wavelet transform (SSWT). Then, the main frequency rhythm of heart rate with maximum-energy embedded in the spectrum of the ECG signal is reconstructed using time-frequency ridge estimation algorithm. The ridge is subjected to the inverse SSW and SSW subsequently to reconstruct a clear spectrum corresponding to the main heart rhythm. Subtracting it from the initial spectrum, the resulting differential spectrum is converted to a single time-series by simply summing all the energy levels at each time-point. It has been shown that the derived time-series is proportional to the quality of ECG signal in terms of preserving its physiological features. The results of this research provide a profound basis for signal quality assessment of both ECG and photoplethysmography (PPG) signals under various noisy conditions and abnormal heart rate.

Recognition of Patient Groups with Sleep Related Disorders using Bio-signal Processing and Deep Learning.

Accurately diagnosing sleep disorders is essential for clinical assessments and treatments. Polysomnography (PSG) has long been used for detection of various sleep disorders. In this research, electrocardiography (ECG) and electromayography (EMG) have been used for recognition of breathing and movement-related sleep disorders. Bio-signal processing has been performed by extracting EMG features exploiting entropy and statistical moments, in addition to developing an iterative pulse peak detection algorithm using synchrosqueezed wavelet transform (SSWT) for reliable extraction of heart rate and breathing-related features from ECG. A deep learning framework has been designed to incorporate EMG and ECG features. The framework has been used to classify four groups: healthy subjects, patients with obstructive sleep apnea (OSA), patients with restless leg syndrome (RLS) and patients with both OSA and RLS. The proposed deep learning framework produced a mean accuracy of 72% and weighted F1 score of 0.57 across subjects for our formulated four-class problem.

Estimation of respiratory rate from motion contaminated photoplethysmography signals incorporating accelerometry.

Estimation of respiratory rate (RR) from photoplethysmography (PPG) signals has important applications in the healthcare sector, from assisting doctors onwards to monitoring patients in their own homes. The problem is still very challenging, particularly during the motion for large segments of data, where results from different methods often do not agree. The authors aim to propose a new technique which performs motion reduction from PPG signals with the help of simultaneous acceleration signals where the PPG and accelerometer sensors need to be embedded in the same sensor unit. This method also reconstructs motion corrupted PPG signals in the Hilbert domain. An auto-regressive (AR) based technique has been used to estimate the RR from reconstructed PPGs. The proposed method has provided promising results for the estimation of RRs and their variations from PPG signals corrupted with motion artefact. The proposed platform is able to contribute to continuous in-hospital and home-based monitoring of patients using PPG signals under various conditions such as rest and motion states.

Validation of Instantaneous Respiratory Rate Using Reflectance PPG from Different Body Positions.

Respiratory rate (RR) is a key parameter used in healthcare for monitoring and predicting patient deterioration. However, continuous and automatic estimation of this parameter from wearable sensors is still a challenging task. Various methods have been proposed to estimate RR from wearable sensors using windowed segments of the data; e.g., often using a minimum of 32 s. Little research has been reported in the literature concerning the instantaneous detection of respiratory rate from such sources. In this paper, we develop and evaluate a method to estimate instantaneous respiratory rate (IRR) from body-worn reflectance photoplethysmography (PPG) sensors. The proposed method relies on a nonlinear time-frequency representation, termed the wavelet synchrosqueezed transform (WSST). We apply the latter to derived modulations of the PPG that arise from the act of breathing.We validate the proposed algorithm using (i) a custom device with a PPG probe placed on various body positions and (ii) a commercial wrist-worn device (WaveletHealth Inc., Mountain View, CA, USA). Comparator reference data were obtained via a thermocouple placed under the nostrils, providing ground-truth information concerning respiration cycles. Tracking instantaneous frequencies was performed in the joint time-frequency spectrum of the (4 Hz re-sampled) respiratory-induced modulation using the WSST, from data obtained from 10 healthy subjects. The estimated instantaneous respiratory rates have shown to be highly correlated with breath-by-breath variations derived from the reference signals. The proposed method produced more accurate results compared to averaged RR obtained using 32 s windows investigated with overlap between successive windows of (i) zero and (ii) 28 s. For a set of five healthy subjects, the averaged similarity between reference RR and instantaneous RR, given by the longest common subsequence (LCSS) algorithm, was calculated as 0.69; this compares with averaged similarity of 0.49 using 32 s windows with 28 s overlap between successive windows. The results provide insight into estimation of IRR and show that upper body positions produced PPG signals from which a better respiration signal was extracted than for other body locations.

A Review on Accelerometry-Based Gait Analysis and Emerging Clinical Applications.

Gait analysis continues to be an important technique for many clinical applications to diagnose and monitor certain diseases. Many mental and physical abnormalities cause measurable differences in a person's gait. Gait analysis has applications in sport, computer games, physical rehabilitation, clinical assessment, surveillance, human recognition, modeling, and many other fields. There are established methods using various sensors for gait analysis, of which accelerometers are one of the most often employed. Accelerometer sensors are generally more user friendly and less invasive. In this paper, we review research regarding accelerometer sensors used for gait analysis with particular focus on clinical applications. We provide a brief introduction to accelerometer theory followed by other popular sensing technologies. Commonly used gait phases and parameters are enumerated. The details of selecting the papers for review are provided. We also review several gait analysis software. Then we provide an extensive report of accelerometry-based gait analysis systems and applications, with additional emphasis on trunk accelerometry. We conclude this review with future research directions.

Towards Photoplethysmography-Based Estimation of Instantaneous Heart Rate During Physical Activity.

OBJECTIVE: Recently numerous methods have been proposed for estimating average heart rate using photoplethysmography (PPG) during physical activity, overcoming the significant interference that motion causes in PPG traces. We propose a new algorithm framework for extracting instantaneous heart rate from wearable PPG and Electrocardiogram (ECG) signals to provide an estimate of heart rate variability during exercise. METHODS: For ECG signals, we propose a new spectral masking approach which modifies a particle filter tracking algorithm, and for PPG signals constrains the instantaneous frequency obtained from the Hilbert transform to a region of interest around a candidate heart rate measure. Performance is verified using accelerometry and wearable ECG and PPG data from subjects while biking and running on a treadmill. RESULTS: Instantaneous heart rate provides more information than average heart rate alone. The instantaneous heart rate can be extracted during motion to an accuracy of 1.75 beats per min (bpm) from PPG signals and 0.27 bpm from ECG signals. CONCLUSION: Estimates of instantaneous heart rate can now be generated from PPG signals during motion. These estimates can provide more information on the human body during exercise. SIGNIFICANCE: Instantaneous heart rate provides a direct measure of vagal nerve and sympathetic nervous system activity and is of substantial use in a number of analyzes and applications. Previously it has not been possible to estimate instantaneous heart rate from wrist wearable PPG signals.

Skill acquisition and stress adaptations following laparoscopic surgery training and detraining in novice surgeons.

BACKGROUND: Surgical training and practice is stressful, but adaptive changes in the stress circuitry (e.g. perceptual, physiological, hormonal, neural) could support skill development. This work examined skill acquisition and stress adaptations in novice surgeons during laparoscopic surgery (LS) training and detraining. METHODS: Twelve medical students were assessed for skill performance after 2 h (BASE), 5 h (MID) and 8 h (POST) of LS training in weeks 1-3, and then after 4 weeks of no training (RETEST). The stress outcomes included state anxiety, perceived stress and workload, heart rate (HR), heart rate variability (HRV), and salivary testosterone and cortisol concentrations. Functional near-infrared spectroscopy was used to assess cortical oxygenation change, as a marker of prefrontal cortex (PFC) activity. RESULTS: Skill performance improved in every session from BASE (p < 0.01), with corresponding decreases in state anxiety, stress, workload, low- and high-frequency HRV in the MID, POST and/or RETEST sessions (p < 0.05). Left and right PFC were symmetrically activated within each testing session (p < 0.01). The stress and workload measures predicted skill performance and changes over time (p < 0.05), with state anxiety, mean HR and the HRV measures also showing some predictive potential (p < 0.10). CONCLUSIONS: A 3-week LS training programme promoted stress-related adaptations likely to directly, or indirectly, support the acquisition of new surgical skills, and many outcomes were retained after a 4-week period without further LS training. These results have implications for medical training and education (e.g. distributed training for skill development and maintenance, stress resource and management training) and highlighted possible areas for new research (e.g. longitudinal stress and skill profiling).

Gait Analysis From a Single Ear-Worn Sensor: Reliability and Clinical Evaluation for Orthopaedic Patients.

Objective assessment of detailed gait patterns after orthopaedic surgery is important for post-surgical follow-up and rehabilitation. The purpose of this paper is to assess the use of a single ear-worn sensor for clinical gait analysis. A reliability measure is devised for indicating the confidence level of the estimated gait events, allowing it to be used in free-walking environments and for facilitating clinical assessment of orthopaedic patients after surgery. Patient groups prior to or following anterior cruciate ligament (ACL) reconstruction and knee replacement were recruited to assess the proposed method. The ability of the sensor for detailed longitudinal analysis is demonstrated with a group of patients after lower limb reconstruction by considering parameters such as temporal and force-related gait asymmetry derived from gait events. The results suggest that the ear-worn sensor can be used for objective gait assessments of orthopaedic patients without the requirement and expense of an elaborate laboratory setup for gait analysis. It significantly simplifies the monitoring protocol and opens the possibilities for home-based remote patient assessment.

Estimation of heart rate from foot worn photoplethysmography sensors during fast bike exercise.

This paper presents a new method for estimating the average heart rate from a foot/ankle worn photoplethysmography (PPG) sensor during fast bike activity. Placing the PPG sensor on the lower half of the body allows more energy to be collected from energy harvesting in order to give a power autonomous sensor node, but comes at the cost of introducing significant motion interference into the PPG trace. We present a normalised least mean square adaptive filter and short-time Fourier transform based algorithm for estimating heart rate in the presence of this motion contamination. Results from 8 subjects show the new algorithm has an average error of 9 beats-per-minute when compared to an ECG gold standard.

Assessing functional mobility after lower limb reconstruction: a psychometric evaluation of a sensor-based mobility score.

OBJECTIVE: To develop and validate a robust, objective mobility assessment tool, Hamlyn Mobility Score (HMS), using a wearable motion sensor. BACKGROUND: Advances in reconstructive techniques allow more limbs to be salvaged. However, evidence demonstrating superior long-term outcomes compared with amputation is unavailable. Lack of access to quality regular functional mobility status may be preventing patients and health care staff from optimizing rehabilitation programs and evaluating the reconstructive services. METHODS: In this prospective cohort study, 20 patients undergoing lower limb reconstruction and 10 age-matched controls were recruited. All subjects completed the HMS activity protocol twice under different instructors at 3 months postoperatively, and again at 6 months, while wearing an ear-worn accelerometer. Demographic and clinical data were also collected including a short-form health survey (SF-36). HMS parameters included standard test metrics and additional kinematic features extracted from accelerometer data. A psychometric evaluation was conducted to ascertain reliability and validity. RESULTS: The HMS demonstrated excellent reliability (intraclass correlation coefficient >0.90, P < 0.001) and internal consistency (Cronbach α = 0.897). Concurrent validity was demonstrated by correlation between HMS and SF-36 scores (Spearman ρ = 0.666, P = 0.005). Significant HMS differences between healthy subjects and patients, stratified according to fracture severity, were shown (Kruskal-Wallis nonparametric 1-way analysis of variance, χ = 21.5, P < 0.001). The HMS was 50% more responsive to change than SF-36 (effect size: 1.49 vs 0.99). CONCLUSIONS: The HMS shows satisfactory reliability and validity and may provide a platform to support adaptable, personalized rehabilitation and enhanced service evaluation to facilitate optimal patient outcomes.

Gait parameter estimation from a miniaturized ear-worn sensor using singular spectrum analysis and longest common subsequence.

This paper presents a new approach to gait analysis and parameter estimation from a single miniaturized ear-worn sensor embedded with a triaxial accelerometer. Singular spectrum analysis combined with the longest common subsequence algorithm has been used as a basis for gait parameter estimation. It incorporates information from all axes of the accelerometer to estimate parameters including swing, stance, and stride times. Rather than only using local features of the raw signals, the periodicity of the signals is also taken into account. The hypotheses tested by this study include: 1) how accurate is the ear-worn sensor in terms of gait parameter extraction compared to the use of an instrumented treadmill; 2) does the ear-worn sensor provide a feasible option for assessment and quantification of gait pattern changes. Key gait events for normal subjects such as heel contact and toe off are validated with a high-speed camera, as well as a force-plate instrumented treadmill. Ten healthy adults walked for 20 min on a treadmill with an increasing incline of 2% every 2 min. The upper and lower limits of the absolute errors using 95% confidence intervals for swing, stance, and stride times were obtained as 35.5 ±3.99 ms, 36.9 ±3.84 ms, and 17.9 ±2.29 ms, respectively.

Regional coherence evaluation in mild cognitive impairment and Alzheimer's disease based on adaptively extracted magnetoencephalogram rhythms.

This study assesses the connectivity alterations caused by Alzheimer's disease (AD) and mild cognitive impairment (MCI) in magnetoencephalogram (MEG) background activity. Moreover, a novel methodology to adaptively extract brain rhythms from the MEG is introduced. This methodology relies on the ability of empirical mode decomposition to isolate local signal oscillations and constrained blind source separation to extract the activity that jointly represents a subset of channels. Inter-regional MEG connectivity was analysed for 36 AD, 18 MCI and 26 control subjects in δ, θ, α and ß bands over left and right central, anterior, lateral and posterior regions with magnitude squared coherence-c(f). For the sake of comparison, c(f) was calculated from the original MEG channels and from the adaptively extracted rhythms. The results indicated that AD and MCI cause slight alterations in the MEG connectivity. Computed from the extracted rhythms, c(f) distinguished AD and MCI subjects from controls with 69.4% and 77.3% accuracies, respectively, in a full leave-one-out cross-validation evaluation. These values were higher than those obtained without the proposed extraction methodology.

A new spatiotemporal filtering method for single-trial estimation of correlated ERP subcomponents.

A novel spatiotemporal filtering method for single trial estimation of event-related potential (ERP) subcomponents is proposed here. Unlike some previous works in ERP estimation [1], , the proposed method is able to estimate temporally correlated ERP subcomponents such as P3a and P3b. A new cost function is, therefore, defined which can deflate one of the correlated subcomponents. The method is applied to both simulated and real data and has shown to perform very well even in low signal-to-noise ratio situations. In addition, the method is compared to spatial principal component analysis and its superiority has been confirmed by using simulated signals. The approach can be especially useful in mental fatigue analysis where the relative variability of P300 subcomponents is the key factor in detecting the level of fatigue.

Blind separation and localization of correlated P300 subcomponents from single trial recordings using extended PARAFAC2 tensor model.

A novel mathematical model based on multi-way data construction and analysis with the goal of simultaneously separating and localizing the brain sources specially the subcomponents of event related potentials (ERPs) is introduced. We represent multi-channel EEG data using a third-order tensor with modes: space (channels), time samples, and number of segments. Then, a multi-way technique, in particular, generalized version of PARAFAC2 method, is developed to blindly separate and localize mutually/temporally correlated P3a and P3b sources as subcomponents of P300 signal. In this paper the non-orthogonality of the ERP subcomponents is defined within the tensor model. In order to obtain essentially unique estimation of the signal components one parametric and one structural constraint are defined and imposed. The method is applied to both simulated and real data and has been shown to perform very well even in low signal to noise ratio situations. In addition, the method is compared with spatial principal component analysis (sPCA) and its superiority is demonstrated by using simulated signals.