Deep Learning System for User Identification Using Sensors on Doorknobs.

Door access control systems are important to protect the security and integrity of physical spaces. Accuracy and speed are important factors that govern their performance. In this paper, we investigate a novel approach to identify users by measuring patterns of their interactions with a doorknob via an embedded accelerometer and gyroscope and by applying deep-learning-based algorithms to these measurements. Our identification results obtained from 47 users show an accuracy of 90.2%. When the sex of the user is used as an input feature, the accuracy is 89.8% in the case of male individuals and 97.0% in the case of female individuals. We study how the accuracy is affected by the sample duration, finding that is its possible to identify users using a sample of 0.5 s with an accuracy of 68.5%. Our results demonstrate the feasibility of using patterns of motor activity to provide access control, thus extending with it the set of alternatives to be considered for behavioral biometrics.

The Improved Biometric Identification of Keystroke Dynamics Based on Deep Learning Approaches.

In an era marked by escalating concerns about digital security, biometric identification methods have gained paramount importance. Despite the increasing adoption of biometric techniques, keystroke dynamics analysis remains a less explored yet promising avenue. This study highlights the untapped potential of keystroke dynamics, emphasizing its non-intrusive nature and distinctiveness. While keystroke dynamics analysis has not achieved widespread usage, ongoing research indicates its viability as a reliable biometric identifier. This research builds upon the existing foundation by proposing an innovative deep-learning methodology for keystroke dynamics-based identification. Leveraging open research datasets, our approach surpasses previously reported results, showcasing the effectiveness of deep learning in extracting intricate patterns from typing behaviors. This article contributes to the advancement of biometric identification, shedding light on the untapped potential of keystroke dynamics and demonstrating the efficacy of deep learning in enhancing the precision and reliability of identification systems.

MeshID: Few-Shot Finger Gesture Based User Identification Using Orthogonal Signal Interference.

Radio frequency (RF) technology has been applied to enable advanced behavioral sensing in human-computer interaction. Due to its device-free sensing capability and wide availability on Internet of Things devices. Enabling finger gesture-based identification with high accuracy can be challenging due to low RF signal resolution and user heterogeneity. In this paper, we propose MeshID, a novel RF-based user identification scheme that enables identification through finger gestures with high accuracy. MeshID significantly improves the sensing sensitivity on RF signal interference, and hence is able to extract subtle individual biometrics through velocity distribution profiling (VDP) features from less-distinct finger motions such as drawing digits in the air. We design an efficient few-shot model retraining framework based on first component reverse module, achieving high model robustness and performance in a complex environment. We conduct comprehensive real-world experiments and the results show that MeshID achieves a user identification accuracy of 95.17% on average in three indoor environments. The results indicate that MeshID outperforms the state-of-the-art in identification performance with less cost.

XSiteTraj: A cross-site user trajectory dataset.

With the development of mobile networks, social networking plays an increasingly important role in people's daily life. User identification, which aims to match user cross-site accounts, has been becoming an important issue for user supervision and recommendation system design in social networks. At present, many different user identification methods have emerged, such as DPLink, HFUL, etc. However, compared with the continuous development of user identification methods, the open-source work of datasets is very slow, and the datasets of most of the work are not public. The shortage of datasets has greatly hindered the development of this research field. At present, the academic urgently needs a large-scale social network user linkage dataset. In this paper, we publicize a new social network user linkage dataset, XSiteTraj v1.0 [2]. This dataset has good spatio-temporal coverage, including more than 27,000 users and more than one million check-in records from all over the world crawled from Facebook, Foursquare, and Twitter. Our dataset labels the identical users from different social websites, and each check-in record includes a timestamp, point of interest (PoI), and latitude and longitude of PoI. Through our dataset, we can conduct research on user behaviour habits and apply the dataset to the experiments and evaluation of social network user identification and other algorithms.

The Empty-Nest Power User Management Based on Data Mining Technology.

With the aging of the social population structure, the number of empty-nesters is also increasing. Therefore, it is necessary to manage empty-nesters with data mining technology. This paper proposed an empty-nest power user identification and power consumption management method based on data mining. Firstly, an empty-nest user identification algorithm based on weighted random forest was proposed. Compared with similar algorithms, the results indicate that the performance of the algorithm is the best, and the identification accuracy of empty-nest users is 74.2%. Then a method for analyzing the electricity consumption behavior of empty-nest users based on fusion clustering index adaptive cosine K-means was proposed, which can adaptively select the optimal number of clusters. Compared with similar algorithms, the algorithm has the shortest running time, the smallest Sum of the Squared Error (SSE), and the largest mean distance between clusters (MDC), which are 3.4281 s, 31.6591 and 13.9513, respectively. Finally, an anomaly detection model with an Auto-regressive Integrated Moving Average (ARIMA) algorithm and an isolated forest algorithm was established. The case analysis shows that the recognition accuracy of abnormal electricity consumption for empty-nest users was 86%. The results indicate that the model can effectively detect the abnormal behavior of empty-nest power users and help the power department to better serve empty-nest users.

Activity-Free User Identification Using Wearables Based on Vision Techniques.

In order to achieve the promise of smart spaces where the environment acts to fulfill the needs of users in an unobtrusive and personalized manner, it is necessary to provide means for a seamless and continuous identification of users to know who indeed is interacting with the system and to whom the smart services are to be provided. In this paper, we propose a new approach capable of performing activity-free identification of users based on hand and arm motion patterns obtained from an wrist-worn inertial measurement unit (IMU). Our approach is not constrained to particular types of movements, gestures, or activities, thus, allowing users to perform freely and unconstrained their daily routine while the user identification takes place. We evaluate our approach based on IMU data collected from 23 people performing their daily routines unconstrained. Our results indicate that our approach is able to perform activity-free user identification with an accuracy of 0.9485 for 23 users without requiring any direct input or specific action from users. Furthermore, our evaluation provides evidence regarding the robustness of our approach in various different configurations.

Explaining One-Dimensional Convolutional Models in Human Activity Recognition and Biometric Identification Tasks.

Due to wearables' popularity, human activity recognition (HAR) plays a significant role in people's routines. Many deep learning (DL) approaches have studied HAR to classify human activities. Previous studies employ two HAR validation approaches: subject-dependent (SD) and subject-independent (SI). Using accelerometer data, this paper shows how to generate visual explanations about the trained models' decision making on both HAR and biometric user identification (BUI) tasks and the correlation between them. We adapted gradient-weighted class activation mapping (grad-CAM) to one-dimensional convolutional neural networks (CNN) architectures to produce visual explanations of HAR and BUI models. Our proposed networks achieved 0.978 and 0.755 accuracy, employing both SD and SI. The proposed BUI network achieved 0.937 average accuracy. We demonstrate that HAR's high performance with SD comes not only from physical activity learning but also from learning an individual's signature, as in BUI models. Our experiments show that CNN focuses on larger signal sections in BUI, while HAR focuses on smaller signal segments. We also use the grad-CAM technique to identify database bias problems, such as signal discontinuities. Combining explainable techniques with deep learning can help models design, avoid results overestimation, find bias problems, and improve generalization capability.

A Multiple Salient Features-Based User Identification across Social Media.

Identifying users across social media has practical applications in many research areas, such as user behavior prediction, commercial recommendation systems, and information retrieval. In this paper, we propose a multiple salient features-based user identification across social media (MSF-UI), which extracts and fuses the rich redundant features contained in user display name, network topology, and published content. According to the differences between users' different features, a multi-module calculation method is used to obtain the similarity between various redundant features. Finally, the bidirectional stable marriage matching algorithm is used for user identification across social media. Experimental results show that: (1) Compared with single-attribute features, the multi-dimensional information generated by users is integrated to optimize the universality of user identification; (2) Compared with baseline methods such as ranking-based cross-matching (RCM) and random forest confirmation algorithm based on stable marriage matching (RFCA-SMM), this method can effectively improve precision rate, recall rate, and comprehensive evaluation index (F1).

Deep Residual Network for Smartwatch-Based User Identification through Complex Hand Movements.

Wearable technology has advanced significantly and is now used in various entertainment and business contexts. Authentication methods could be trustworthy, transparent, and non-intrusive to guarantee that users can engage in online communications without consequences. An authentication system on a security framework starts with a process for identifying the user to ensure that the user is permitted. Establishing and verifying an individual's appearance usually requires a lot of effort. Recent years have seen an increase in the usage of activity-based user identification systems to identify individuals. Despite this, there has not been much research into how complex hand movements can be used to determine the identity of an individual. This research used a one-dimensional residual network with squeeze-and-excitation (SE) configurations called the 1D-ResNet-SE model to investigate hand movements and user identification. According to the findings, the SE modules have enhanced the one-dimensional residual network's identification ability. As a deep learning model, the proposed methodology is capable of effectively identifying features from the input smartwatch sensor and could be utilized as an end-to-end model to clarify the modeling process. The 1D-ResNet-SE identification model is superior to the other models. Hand movement assessment based on deep learning is an effective technique to identify smartwatch users.

Stack LSTM-Based User Identification Using Smart Shoes with Accelerometer Data.

In this study, we propose a long short-term memory (LSTM)-based user identification method using accelerometer data from smart shoes. In general, for the user identification with human walking data, we require a pre-processing stage in order to divide human walking data into individual steps. Next, user identification can be made with divided step data. In these approaches, when there exist partial data that cannot complete a single step, it is difficult to apply those data to the classification. Considering these facts, in this study, we present a stack LSTM-based user identification method for smart-shoes data. Rather than using a complicated analysis method, we designed an LSTM network for user identification with accelerometer data of smart shoes. In order to learn partial data, the LSTM network was trained using walking data with random sizes and random locations. Then, the identification can be made without any additional analysis such as step division. In the experiments, user walking data with 10 m were used. The experimental results show that the average recognition rate was about 93.41%, 97.19%, and 98.26% by using walking data of 2.6, 3.9, and 5.2 s, respectively. With the experimental results, we show that the proposed method can classify users effectively.

Recognition System Using Fusion Normalization Based on Morphological Features of Post-Exercise ECG for Intelligent Biometrics.

Although biometrics systems using an electrocardiogram (ECG) have been actively researched, there is a characteristic that the morphological features of the ECG signal are measured differently depending on the measurement environment. In general, post-exercise ECG is not matched with the morphological features of the pre-exercise ECG because of the temporary tachycardia. This can degrade the user recognition performance. Although normalization studies have been conducted to match the post- and pre-exercise ECG, limitations related to the distortion of the P wave, QRS complexes, and T wave, which are morphological features, often arise. In this paper, we propose a method for matching pre- and post-exercise ECG cycles based on time and frequency fusion normalization in consideration of morphological features and classifying users with high performance by an optimized system. One cycle of post-exercise ECG is expanded by linear interpolation and filtered with an optimized frequency through the fusion normalization method. The fusion normalization method aims to match one post-exercise ECG cycle to one pre-exercise ECG cycle. The experimental results show that the average similarity between the pre- and post-exercise states improves by 25.6% after normalization, for 30 ECG cycles. Additionally, the normalization algorithm improves the maximum user recognition performance from 96.4 to 98%.

Can Ensemble Deep Learning Identify People by Their Gait Using Data Collected from Multi-Modal Sensors in Their Insole?

Gait is a characteristic that has been utilized for identifying individuals. As human gait information is now able to be captured by several types of devices, many studies have proposed biometric identification methods using gait information. As research continues, the performance of this technology in terms of identification accuracy has been improved by gathering information from multi-modal sensors. However, in past studies, gait information was collected using ancillary devices while the identification accuracy was not high enough for biometric identification. In this study, we propose a deep learning-based biometric model to identify people by their gait information collected through a wearable device, namely an insole. The identification accuracy of the proposed model when utilizing multi-modal sensing is over 99%.

Compressed-Domain ECG-Based Biometric User Identification Using Compressive Analysis.

Nowadays, user identification plays a more and more important role for authorized machine access and remote personal data usage. For reasons of privacy and convenience, biometrics-based user identification, such as iris, fingerprint, and face ID, has become mainstream methods in our daily lives. However, most of the biometric methods can be easily imitated or artificially cracked. New types of biometrics, such as electrocardiography (ECG), are based on physiological signals rather than traditional biological traits. Recently, compressive sensing (CS) technology that combines both sampling and compression has been widely applied to reduce the power of data acquisition and transmission. However, prior CS-based frameworks suffer from high reconstruction overhead and cannot directly align compressed ECG signals. In this paper, in order to solve the above two problems, we propose a compressed alignment-aided compressive analysis (CA-CA) algorithm for ECG-based biometric user identification. With CA-CA, it can avoid reconstruction and extract information directly from CS-based compressed ECG signals to reduce overall complexity and power. Besides, CA-CA can also align the compressed ECG signals in the eigenspace-domain, which can further enhance the precision of identifications and reduce the total training time. The experimental result shows that our proposed algorithm has a 94.16% accuracy based on a public database of 22 people.

BioMove: Biometric User Identification from Human Kinesiological Movements for Virtual Reality Systems.

Virtual reality (VR) has advanced rapidly and is used for many entertainment and business purposes. The need for secure, transparent and non-intrusive identification mechanisms is important to facilitate users' safe participation and secure experience. People are kinesiologically unique, having individual behavioral and movement characteristics, which can be leveraged and used in security sensitive VR applications to compensate for users' inability to detect potential observational attackers in the physical world. Additionally, such method of identification using a user's kinesiological data is valuable in common scenarios where multiple users simultaneously participate in a VR environment. In this paper, we present a user study (n = 15) where our participants performed a series of controlled tasks that require physical movements (such as grabbing, rotating and dropping) that could be decomposed into unique kinesiological patterns while we monitored and captured their hand, head and eye gaze data within the VR environment. We present an analysis of the data and show that these data can be used as a biometric discriminant of high confidence using machine learning classification methods such as kNN or SVM, thereby adding a layer of security in terms of identification or dynamically adapting the VR environment to the users' preferences. We also performed a whitebox penetration testing with 12 attackers, some of whom were physically similar to the participants. We could obtain an average identification confidence value of 0.98 from the actual participants' test data after the initial study and also a trained model classification accuracy of 98.6%. Penetration testing indicated all attackers resulted in confidence values of less than 50% (<50%), although physically similar attackers had higher confidence values. These findings can help the design and development of secure VR systems.

Biometric From Surface Electromyogram (sEMG): Feasibility of User Verification and Identification Based on Gesture Recognition.

Electrical biosignals are favored as biometric traits due to their hidden nature and allowing for liveness detection. This study explored the feasibility of surface electromyogram (sEMG), the electrical manifestation of muscle activities, as a biometric trait. The accurate gesture recognition from sEMG provided a unique advantage over two traditional electrical biosignal traits, electrocardiogram (ECG), and electroencephalogram (EEG), enabling users to customize their own gesture codes. The performance of 16 static wrist and hand gestures was systematically investigated in two identity management modes: verification and identification. The results showed that for a single fixed gesture, using only 0.8-second data, the averaged equal error rate (EER) for verification was 3.5%, and the averaged rank-1 for identification was 90.3%, both comparable to the reported performance of ECG and EEG. The function of customizing gesture code could further improve the verification performance to 1.1% EER. This work demonstrated the potential and effectiveness of sEMG as a biometric trait in user verification and identification, beneficial for the design of future biometric systems.

User Identification from Gait Analysis Using Multi-Modal Sensors in Smart Insole.

Recent studies indicate that individuals can be identified by their gait pattern. A number of sensors including vision, acceleration, and pressure have been used to capture humans' gait patterns, and a number of methods have been developed to recognize individuals from their gait pattern data. This study proposes a novel method of identifying individuals using null-space linear discriminant analysis on humans' gait pattern data. The gait pattern data consists of time series pressure and acceleration data measured from multi-modal sensors in a smart insole used while walking. We compare the identification accuracies from three sensing modalities, which are acceleration, pressure, and both in combination. Experimental results show that the proposed multi-modal features identify 14 participants with high accuracy over 95% from their gait pattern data of walking.

ADLAuth: Passive Authentication Based on Activity of Daily Living Using Heterogeneous Sensing in Smart Cities.

The Internet of Things is a rapidly growing paradigm for smart cities that provides a way of communication, identification, and sensing capabilities among physically distributed devices. With the evolution of the Internet of Things (IoTs), user dependence on smart systems and services, such as smart appliances, smartphone, security, and healthcare applications, has been increased. This demands secure authentication mechanisms to preserve the users' privacy when interacting with smart devices. This paper proposes a heterogeneous framework "ADLAuth" for passive and implicit authentication of the user using either a smartphone's built-in sensor or wearable sensors by analyzing the physical activity patterns of the users. Multiclass machine learning algorithms are applied to users' identity verification. Analyses are performed on three different datasets of heterogeneous sensors for a diverse number of activities. A series of experiments have been performed to test the effectiveness of the proposed framework. The results demonstrate the better performance of the proposed scheme compared to existing work for user authentication.

Deep Arm/Ear-ECG Image Learning for Highly Wearable Biometric Human Identification.

In this study, to advance smart health applications which have increasing security/privacy requirements, we propose a novel highly wearable ECG-based user identification system, empowered by both non-standard convenient ECG lead configurations and deep learning techniques. Specifically, to achieve a super wearability, we suggest situating all the ECG electrodes on the left upper-arm, or behind the ears, and successfully obtain weak but distinguishable ECG waveforms. Afterwards, to identify individuals from weak ECG, we further present a two-stage framework, including ECG imaging and deep feature learning/identification. In the former stage, the ECG heartbeats are projected to a 2D state space, to reveal heartbeats' trajectory behaviors and produce 2D images by a split-then-hit method. In the second stage, a convolutional neural network is introduced to automatically learn the intricate patterns directly from the ECG image representations without heavy feature engineering, and then perform user identification. Experimental results on two acquired datasets using our wearable prototype, show a promising identification rate of 98.4% (single-arm-ECG) and 91.1% (ear-ECG), respectively. To the best of our knowledge, it is the first study on the feasibility of using single-arm-ECG/ear-ECG for user identification purpose, which is expected to contribute to pervasive ECG-based user identification in smart health applications.

Multi-User Identification-Based Eye-Tracking Algorithm Using Position Estimation.

This paper proposes a new multi-user eye-tracking algorithm using position estimation. Conventional eye-tracking algorithms are typically suitable only for a single user, and thereby cannot be used for a multi-user system. Even though they can be used to track the eyes of multiple users, their detection accuracy is low and they cannot identify multiple users individually. The proposed algorithm solves these problems and enhances the detection accuracy. Specifically, the proposed algorithm adopts a classifier to detect faces for the red, green, and blue (RGB) and depth images. Then, it calculates features based on the histogram of the oriented gradient for the detected facial region to identify multiple users, and selects the template that best matches the users from a pre-determined face database. Finally, the proposed algorithm extracts the final eye positions based on anatomical proportions. Simulation results show that the proposed algorithm improved the average F1 score by up to 0.490, compared with benchmark algorithms.

Interaction Detection with Depth Sensing and Body Tracking Cameras in Physical Rehabilitation.

INTRODUCTION: This article is part of the Focus Theme of Methods of Information in Medicine on "Methodologies, Models and Algorithms for Patients Rehabilitation". OBJECTIVES: This paper presents a camera based method for identifying the patient and detecting interactions between the patient and the therapist during therapy. Detecting interactions helps to discriminate between active and passive motion of the patient as well as to estimate the accuracy of the skeletal data. METHODS: Continuous face recognition is used to detect, recognize and track the patient with other people in the scene (e.g. the therapist, or a clinician). We use a method based on local binary patterns (LBP). After identifying users in the scene we identify interactions between the patient and other people. We use a depth map/point cloud for estimating the distance between two people. Our method uses the association of depth regions to user identities and computes the minimal distance between the regions. RESULTS: Our results show state-of-the-art performance of real-time face recognition using low-resolution images that is sufficient to use in adaptive systems. Our proposed approach for detecting interactions shows 91.9% overall recognition accuracy what is sufficient for applications in the context of serious games. We also discuss limitations of the proposed method as well as general limitations of using depth cameras for serious games. CONCLUSIONS: We introduced a new method for frame-by-frame automated identification of the patient and labeling reliable sequences of the patient's data recorded during rehabilitation (games). Our method improves automated rehabilitation systems by detecting the identity of the patient as well as of the therapist and by detecting the distance between both over time.