Ambient assisted living for frail people through human activity recognition: state-of-the-art, challenges and future directions.

Ambient Assisted Living is a concept that focuses on using technology to support and enhance the quality of life and well-being of frail or elderly individuals in both indoor and outdoor environments. It aims at empowering individuals to maintain their independence and autonomy while ensuring their safety and providing assistance when needed. Human Activity Recognition is widely regarded as the most popular methodology within the field of Ambient Assisted Living. Human Activity Recognition involves automatically detecting and classifying the activities performed by individuals using sensor-based systems. Researchers have employed various methodologies, utilizing wearable and/or non-wearable sensors, and employing algorithms ranging from simple threshold-based techniques to more advanced deep learning approaches. In this review, literature from the past decade is critically examined, specifically exploring the technological aspects of Human Activity Recognition in Ambient Assisted Living. An exhaustive analysis of the methodologies adopted, highlighting their strengths and weaknesses is provided. Finally, challenges encountered in the field of Human Activity Recognition for Ambient Assisted Living are thoroughly discussed. These challenges encompass issues related to data collection, model training, real-time performance, generalizability, and user acceptance. Miniaturization, unobtrusiveness, energy harvesting and communication efficiency will be the crucial factors for new wearable solutions.

Recurrent Network Solutions for Human Posture Recognition Based on Kinect Skeletal Data.

Ambient Assisted Living (AAL) systems are designed to provide unobtrusive and user-friendly support in daily life and can be used for monitoring frail people based on various types of sensors, including wearables and cameras. Although cameras can be perceived as intrusive in terms of privacy, low-cost RGB-D devices (i.e., Kinect V2) that extract skeletal data can partially overcome these limits. In addition, deep learning-based algorithms, such as Recurrent Neural Networks (RNNs), can be trained on skeletal tracking data to automatically identify different human postures in the AAL domain. In this study, we investigate the performance of two RNN models (2BLSTM and 3BGRU) in identifying daily living postures and potentially dangerous situations in a home monitoring system, based on 3D skeletal data acquired with Kinect V2. We tested the RNN models with two different feature sets: one consisting of eight human-crafted kinematic features selected by a genetic algorithm, and another consisting of 52 ego-centric 3D coordinates of each considered skeleton joint, plus the subject's distance from the Kinect V2. To improve the generalization ability of the 3BGRU model, we also applied a data augmentation method to balance the training dataset. With this last solution we reached an accuracy of 88%, the best we achieved so far.

Corrective saccades in acute vestibular neuritis: studying the role of prediction with automated passively induced head impulses.

When the demands for visual stabilization during head rotations overwhelm the ability of the vestibuloocular reflex (VOR) to produce compensatory eye movements, the brain produces corrective saccades that bring gaze toward the fixation target, even without visual cues (covert saccades). What triggers covert saccades and what might be the role of prediction in their generation are unknown. We studied 14 subjects with acute vestibular neuritis. To minimize variability of the stimulus, head impulses were imposed with a motorized torque generator with the subject on a bite bar. Predictable and unpredictable (timing, amplitude, direction) stimuli were compared. Distributions of covert corrective saccade latencies were analyzed with a "LATER" (linear approach to threshold with ergodic rate) approach. On the affected side, VOR gain was higher (0.47 ± 0.28 vs. 0.39 ± 0.22, P ≪ 0.001) with predictable than unpredictable head impulses, and gaze error at the end of the head movement was less (5.4 ± 3.3° vs. 6.9 ± 3.3°, P ≪ 0.001). Analyzing trials with covert saccades, gaze error at saccade end was significantly less with predictable than unpredictable head impulses (4.2 ± 2.8° vs. 5.5 ± 3.2°, P ≪ 0.001). Furthermore, covert corrective saccades occurred earlier with predictable than unpredictable head impulses (140 ± 37 vs. 153 ± 37 ms, P ≪ 0.001). Using a LATER analysis with reciprobit plots, we were able to divide covert corrective saccades into two classes, early and late, with a break point in the range of 88-98 ms. We hypothesized two rise-to-threshold decision mechanisms for triggering early and late covert corrective saccades, with the first being most engaged when stimuli are predictable.NEW & NOTEWORTHY We successfully used a LATER (linear approach to threshold with ergodic rate) analysis of the latencies of corrective saccades in patients with acute vestibular neuritis. We found two types of covert saccades: early (<90 ms) and late (>90 ms) covert saccades. Predictability led to an increase in VOR gain and a decrease in saccade latency.

Neural Networks for Automatic Posture Recognition in Ambient-Assisted Living.

Human Action Recognition (HAR) is a rapidly evolving field impacting numerous domains, among which is Ambient Assisted Living (AAL). In such a context, the aim of HAR is meeting the needs of frail individuals, whether elderly and/or disabled and promoting autonomous, safe and secure living. To this goal, we propose a monitoring system detecting dangerous situations by classifying human postures through Artificial Intelligence (AI) solutions. The developed algorithm works on a set of features computed from the skeleton data provided by four Kinect One systems simultaneously recording the scene from different angles and identifying the posture of the subject in an ecological context within each recorded frame. Here, we compare the recognition abilities of Multi-Layer Perceptron (MLP) and Long-Short Term Memory (LSTM) Sequence networks. Starting from the set of previously selected features we performed a further feature selection based on an SVM algorithm for the optimization of the MLP network and used a genetic algorithm for selecting the features for the LSTM sequence model. We then optimized the architecture and hyperparameters of both models before comparing their performances. The best MLP model (3 hidden layers and a Softmax output layer) achieved 78.4%, while the best LSTM (2 bidirectional LSTM layers, 2 dropout and a fully connected layer) reached 85.7%. The analysis of the performances on individual classes highlights the better suitability of the LSTM approach.

Membrane Resonance in Pyramidal and GABAergic Neurons of the Mouse Perirhinal Cortex.

The perirhinal cortex (PRC) is a polymodal associative region of the temporal lobe that works as a gateway between cortical areas and hippocampus. In recent years, an increasing interest arose in the role played by the PRC in learning and memory processes, such as object recognition memory, in contrast with certain forms of hippocampus-dependent spatial and episodic memory. The integrative properties of the PRC should provide all necessary resources to select and enhance the information to be propagated to and from the hippocampus. Among these properties, we explore in this paper the ability of the PRC neurons to amplify the output voltage to current input at selected frequencies, known as membrane resonance. Within cerebral circuits the resonance of a neuron operates as a filter toward inputs signals at certain frequencies to coordinate network activity in the brain by affecting the rate of neuronal firing and the precision of spike timing. Furthermore, the ability of the PRC neurons to resonate could have a fundamental role in generating subthreshold oscillations and in the selection of cortical inputs directed to the hippocampus. Here, performing whole-cell patch-clamp recordings from perirhinal pyramidal neurons and GABAergic interneurons of GAD67-GFP+ mice, we found, for the first time, that the majority of PRC neurons are resonant at their resting potential, with a resonance frequency of 0.5-1.5 Hz at 23°C and of 1.5-2.8 Hz at 36°C. In the presence of ZD7288 (blocker of HCN channels) resonance was abolished in both pyramidal neurons and interneurons, suggesting that Ih current is critically involved in resonance generation. Otherwise, application of TTx (voltage-dependent Na+ channel blocker) attenuates the resonance in pyramidal neurons but not in interneurons, suggesting that only in pyramidal neurons the persistent sodium current has an amplifying effect. These experimental results have also been confirmed by a computational model. From a functional point of view, the resonance in the PRC would affect the reverberating activity between neocortex and hippocampus, especially during slow wave sleep, and could be involved in the redistribution and strengthening of memory representation in cortical regions.

Skeleton data pre-processing for human pose recognition using Neural Network.

Automatic monitoring of daily living activities can greatly improve the possibility of living autonomously for frail individuals. Pose recognition based on skeleton tracking data is promising for identifying dangerous situations and trigger external intervention or other alarms, while avoiding privacy issues and the need for patient compliance. Here we present the benefits of pre-processing Kinect-recorded skeleton data to limit the several errors produced by the system when the subject is not in ideal tracking conditions. The accuracy of our two hidden layers MLP classifier improved from about 82% to over 92% in recognizing actors in four different poses: standing, sitting, lying and dangerous sitting.

Automatic Pose Recognition for Monitoring Dangerous Situations in Ambient-Assisted Living.

Continuous monitoring of frail individuals for detecting dangerous situations during their daily living at home can be a powerful tool toward their inclusion in the society by allowing living independently while safely. To this goal we developed a pose recognition system tailored to disabled students living in college dorms and based on skeleton tracking through four Kinect One devices independently recording the inhabitant with different viewpoints, while preserving the individual's privacy. The system is intended to classify each data frame and provide the classification result to a further decision-making algorithm, which may trigger an alarm based on the classified pose and the location of the subject with respect to the furniture in the room. An extensive dataset was recorded on 12 individuals moving in a mockup room and undertaking four poses to be recognized: standing, sitting, lying down, and "dangerous sitting." The latter consists of the subject slumped in a chair with his/her head lying forward or backward as if unconscious. Each skeleton frame was labeled and represented using 10 discriminative features: three skeletal joint vertical coordinates and seven relative and absolute angles describing articular joint positions and body segment orientation. In order to classify the pose of the subject in each skeleton frame we built a two hidden layers multi-layer perceptron neural network with a "SoftMax" output layer, which we trained on the data from 10 of the 12 subjects (495,728 frames), with the data from the two remaining subjects representing the test set (106,802 frames). The system achieved very promising results, with an average accuracy of 83.9% (ranging 82.7 and 94.3% in each of the four classes). Our work proves the usefulness of human pose recognition based on machine learning in the field of safety monitoring in assisted living conditions.

The integration of multisensory motion stimuli is impaired in vestibular migraine patients.

BACKGROUND: Vestibular migraine (VM) is a relatively recently acknowledged vestibular syndrome with a very relevant prevalence of about 10% among patients complaining of vertigo. The diagnostic criteria for VM have been recently published by the Bárány Society, and they are now included in the latest version of the International Classification of Headache Disorders, yet there is no instrumental test that supports the diagnosis of VM. OBJECTIVE: In the hypothesis that the integration of different vestibular stimuli is functionally impaired in VM, we tested whether the combination of abrupt vestibular stimuli and full-field, moving visual stimuli would challenge vestibular migraine patients more than controls and other non-vestibular migraineurs. METHODS: In three clinical centers, we compared the performance in the functional head impulse test (fHIT) without and with an optokinetic stimulus rotating in the frontal plane in a group of 44 controls (Ctrl), a group of 42 patients with migraine (not vestibular migraine, MnoV), a group of 39 patients with vestibular migraine (VM) and a group of 15 patients with vestibular neuritis (VN). RESULTS: The optokinetic stimulation reduced the percentage of correct answers (%CA) in all groups, and in about 33% of the patients with migraine, in as many as 87% of VM patients and 60% of VN patients, this reduction was larger than expected from controls' data. CONCLUSIONS: The comparison of the fHIT results without and with optokinetic stimulation unveils a functional vestibular impairment in VM that is not as large as the one detectable in VN, and that, in contrast with all the other patient groups, mainly impairs the capability to integrate different vestibular stimuli.

Restoring the High-Frequency Dynamic Visual Acuity with a Vestibular Implant Prototype in Humans.

INTRODUCTION: The vestibular implant could become a clinically useful device in the near future. This study investigated the feasibility of restoring the high-frequency dynamic visual acuity (DVA) with a vestibular implant, using the functional Head Impulse Test (fHIT). METHODS: A 72-year-old female, with bilateral vestibulopathy and fitted with a modified cochlear implant incorporating three vestibular electrodes (MED-EL, Innsbruck, Austria), was available for this study. Electrical stimulation was delivered with the electrode close to the lateral ampullary nerve in the left ear. The high-frequency DVA in the horizontal plane was tested with the fHIT. After training, the patient underwent six trials of fHIT, each with a different setting of the vestibular implant: (1) System OFF before stimulation; (2) System ON, baseline stimulation; (3) System ON, reversed stimulation; (4) System ON, positive stimulation; (5) System OFF, without delay after stimulation offset; and (6) System OFF, 25 min delay after stimulation offset. The percentage of correct fHIT scores for right and left head impulses were compared between trials. RESULTS: Vestibular implant stimulation improved the high-frequency DVA compared to no stimulation. This improvement was significant for "System ON, baseline stimulation" (p = 0.02) and "System ON, positive stimulation" (p < 0.001). fHIT scores changed from 19 to 44% (no stimulation) to maximum 75-94% (System ON, positive stimulation). CONCLUSION: The vestibular implant seems capable of improving the high-frequency DVA. This functional benefit of the vestibular implant illustrates again the feasibility of this device for clinical use in the near future.

Non-linearity in gaze holding: Experimental results and possible mechanisms.

Cerebellar impairment may cause deficits in horizontal gaze holding, leading to centrifugal gaze-evoked nystagmus during fixation of eccentric targets. Healthy individuals show a weak drift leading to physiological nystagmus only at large gaze angles. These drifts are due to imperfect memory of the neural circuitry generating the eye position signals by integration of velocity signals. The cerebellum plays a crucial role in reducing the "leakiness" of this neural integrator. This neural integrator has been traditionally modeled as a first order low-pass filter, implying a linear relation between drift velocity and eye eccentricity. Evidences of a non-linear behavior, however, can be found in the literature. In a recent series of papers we showed that the eye drift velocity (V) can be descriptively modeled as a tangent function of gaze eccentricity (P) with the following equation: V=k2/k1tan(k1P). Notably, the two parameters have distinct roles: k1 regulates the rate of compression of the tangent, exclusively determining the non-linearity; k2 is a pure scaling factor. This descriptive model robustly fitted the data of healthy individuals both at baseline (n=50) and following transient cerebellar impairment induced by controlled amounts of alcohol [blood alcohol content 0.06% (n=15) and 0.1% (n=15)] and of patients with chronic cerebellar impairment of various origin (n=20). Interestingly, alcohol selectively changed the scaling factor k2, evidencing that an overall, transient cerebellar impairment does not impact the non-linear behavior. Patients with cerebellar degeneration, on the other hand, showed a change in both k1 and k2, implying a role of the cerebellum in limiting the range of eye positions where the non-linearity becomes relevant. Non-linearity has been reported in literature for both the neural integrator and the eye plant. While previous models using a neural network attempted to reproduce the non-linear behavior of the brainstem, we propose a block diagram reproducing the observed PV tangent relation by introducing a position dependency in the parameters of the cerebellar feedback loop.

The functional head impulse test: Comparing gain and percentage of correct answers.

OBJECTIVES: The video head impulse test (vHIT) provides as output a gain value that summarizes the behavior of the vestibulo-ocular reflex as the ratio of a measure of eye movement to the corresponding measure of head movement and is not directly informative of the functional effectiveness of the motor response. The functional HIT (fHIT) is based on the ability to recognize the orientation of a Landolt C optotype that briefly appears on a computer screen during passive head impulses imposed by the examiner over a range of head accelerations; accordingly fHIT is a functional measurement of the vestibular-ocular reflex since it measures the capability to keep clear vision and to read during head movement. METHODS: We compared the results of the fHIT with those of the vHIT and the results of the Dizziness Handicap Inventory (DHI) questionnaire in a group of 27 vestibular neuritis patients recorded acutely and at 3-months follow-up. RESULTS: Both the vHIT and fHIT exams correctly classified all patients as abnormal on the affected side when tested in the acute phase. After a 3-month follow-up, both were able to show that compensation phenomena had occurred. Otherwise the data from the two techniques were not correlated. More specifically, the fHIT detected more abnormalities than the vHIT, for head rotation toward the healthy side, both in the acute phase and after 3 months, and for head rotation toward the affected side after 3 months. The asymmetry indices, that compare the performance of the healthy to the affected side, also were larger for the fHIT than for the vHIT both at onset and after 3 months. There was no significant correlation between the different vHIT and fHIT parameters and indices, or with the DHI values after 3 months. CONCLUSIONS: The fHIT data are able to detect a difference between the healthy and the affected side in the acute phase, and they show an improvement after 3 months. fHIT detects more abnormalities than vHIT, but both these techniques lack a correlation with the DHI score.

Understanding the rotational vestibular ocular reflex: From differential equations to Laplace transforms.

The description of a biological system using a mathematical model is the most effective means to specify the functioning of a quantitative hypothesis, providing at the same time the ability to make predictions that can be further tested experimentally. The Vestibulo-Ocular Reflex (VOR), and more generally the ocular motor control system has been one of the first biological systems to be modeled mathematically and, thanks to contributions from anatomy, biology, biochemistry and information technology it is now the best understood sensory-motor system in humans. Basic science has made it possible to write the differential equations describing the functioning of the semicircular canals, of the otoliths, and of the ocular motor plant at different scales: from models explaining neurotransmitter behavior, to cell membranes and ionic currents, to individual neurons and entire populations, to those describing muscle contractions and eye movements. The differential equations are frequently represented in terms of Laplace transforms and provide a description of the input-output behavior of the system being considered as a function of frequency. Here we will review the input-output behavior of the rotational VOR to exemplify its mathematical modeling as a linear time-invariant dynamic system being stimulated by head rotations and producing eye movements as an output.

Functional Head Impulse Test in Professional Athletes: Sport-Specific Normative Values and Implication for Sport-Related Concussion.

Dizziness, slow visual tracking, or blurred vision following active head (or body) movements are among the most common symptoms reported following sport-related concussion, often related to concurrent dysfunctions of the vestibular system. In some cases, symptoms persist even if bedside and auxiliary standard vestibular tests are unremarkable. New functional tests have been developed in recent years to objectify neurological alterations that are not captured by standard tests. The functional head impulse test (fHIT) requires the patient to recognize an optotype that is briefly flashed during head rotations with various angular accelerations (2,001-6,000 deg/s2) and assesses the proportion if correct answers (pca). 268 active professional athletes (23.70 ± 5.32y) from six different sports were tested using fHIT. Pca were analyzed both pooling head acceleration in the range of 2,001-6,000 deg/s2 and computing a single pca value for each 1,000 deg/s2 bin in the range 2,001-8,000 deg/s2. No significant difference (p = 0.159) was found between responses to head impulses in the plane of horizontal (pca: 0.977) and vertical semicircular canals (pca: 0.97). The sport practiced had a major effect on the outcome of the fHIT. Handball players achieved a better performance (p < 0.001) than the whole athlete group, irrespective of the direction of head impulses. The pca achieved by athletes practicing snowboard, bob and skeleton were instead significantly below those of the whole athlete group (p < 0.001) but only when vertical head impulses were tested. Overall, pca declined with increasing head acceleration. The decline was particularly evident in the range not included in the standard fHIT exam, i.e., 6,001-8,000 deg/s2 for horizontal and 5,001-8,000 deg/s2 for vertical head impulses. When vertical head impulses were tested, athletes practicing snowboard, bob and skeleton (non-ball sports) showed, beside the lower overall pca, also a steeper decline as a function of vertical head acceleration. The findings suggest that: (1) functional VOR testing can help understanding sport-specific VOR requirements; (2) the fHIT is able to detect and objectify subtle, sport-specific changes of functional VOR performance; (3) if sport-specific normative values are used, the fHIT test procedure needs to be optimized, starting from the highest acceleration to minimize the number of head impulses.

Vestibulo-Ocular Responses and Dynamic Visual Acuity During Horizontal Rotation and Translation.

Dynamic visual acuity (DVA) provides an overall functional measure of visual stabilization performance that depends on the vestibulo-ocular reflex (VOR), but also on other processes, including catch-up saccades and likely visual motion processing. Capturing the efficiency of gaze stabilization against head movement as a whole, it is potentially valuable in the clinical context where assessment of overall patient performance provides an important indication of factors impacting patient participation and quality of life. DVA during head rotation (rDVA) has been assessed previously, but to our knowledge, DVA during horizontal translation (tDVA) has not been measured. tDVA can provide a valuable measure of how otolith, rather than canal, function impacts visual acuity. In addition, comparison of DVA during rotation and translation can shed light on whether common factors are limiting DVA performance in both cases. We therefore measured and compared DVA during both passive head rotations (head impulse test) and translations in the same set of healthy subjects (n = 7). In addition to DVA, we computed average VOR gain and retinal slip within and across subjects. We observed that during translation, VOR gain was reduced (VOR during rotation, mean ± SD: position gain = 1.05 ± 0.04, velocity gain = 0.97 ± 0.07; VOR during translation, mean ± SD: position gain = 0.21 ± 0.08, velocity gain = 0.51 ± 0.16), retinal slip was increased, and tDVA was worse than during rotation (average rDVA = 0.32 ± 0.15 logMAR; average tDVA = 0.56 ± 0.09 logMAR, p = 0.02). This suggests that reduced VOR gain leads to worse tDVA, as expected. We conclude with speculation about non-oculomotor factors that could vary across individuals and affect performance similarly during both rotation and translation.

The functional head impulse test: preliminary data.

The functional head impulse test is a new test of vestibular function based on the ability to recognize the orientation of a Landolt C optotype that briefly appears on a computer screen during passive head impulses imposed by the examiner over a range of head accelerations. Here, we compare its results with those of the video head impulse test on a population of vestibular neuritis patients recorded acutely and after 3 months from symptoms onset. The preliminary results presented here show that while both tests are able to identify the affected labyrinth and to show a recovery of vestibular functionality at 3 months, the two tests are not redundant, but complementary.