Multidimensional Deep Ultraviolet (DUV) Synapses Based on Organic/Perovskite Semiconductor Heterojunction Transistors for Antispoofing Facial Recognition Systems.

Reliably discerning real human faces from fake ones, known as antispoofing, is crucial for facial recognition systems. While neuromorphic systems offer integrated sensing-memory-processing functions, they still struggle with efficient antispoofing techniques. Here we introduce a neuromorphic facial recognition system incorporating multidimensional deep ultraviolet (DUV) optoelectronic synapses to address these challenges. To overcome the complexity and high cost of producing DUV synapses using traditional wide-bandgap semiconductors, we developed a low-temperature (≤70 °C) solution process for fabricating DUV synapses based on PEA2PbBr4/C8-BTBT heterojunction field-effect transistors. This method enables the large-scale (4-in.), uniform, and transparent production of DUV synapses. These devices respond to both DUV and visible light, showing multidimensional features. Leveraging the unique ability of the multidimensional DUV synapse (MDUVS) to discriminate real human skin from artificial materials, we have achieved robust neuromorphic facial recognition with antispoofing capability, successfully identifying genuine human faces with an accuracy exceeding 92%.

Algorithm of face anti-spoofing based on pseudo-negative features generation.

Introduction: Despite advancements in face anti-spoofing technology, attackers continue to pose challenges with their evolving deceptive methods. This is primarily due to the increased complexity of their attacks, coupled with a diversity in presentation modes, acquisition devices, and prosthetic materials. Furthermore, the scarcity of negative sample data exacerbates the situation by causing domain shift issues and impeding robust generalization. Hence, there is a pressing need for more effective cross-domain approaches to bolster the model's capability to generalize across different scenarios. Methods: This method improves the effectiveness of face anti-spoofing systems by analyzing pseudo-negative sample features, expanding the training dataset, and boosting cross-domain generalization. By generating pseudo-negative features with a new algorithm and aligning these features with the use of KL divergence loss, we enrich the negative sample dataset, aiding the training of a more robust feature classifier and broadening the range of attacks that the system can defend against. Results: Through experiments on four public datasets (MSU-MFSD, OULU-NPU, Replay-Attack, and CASIA-FASD), we assess the model's performance within and across datasets by controlling variables. Our method delivers positive results in multiple experiments, including those conducted on smaller datasets. Discussion: Through controlled experiments, we demonstrate the effectiveness of our method. Furthermore, our approach consistently yields favorable results in both intra-dataset and cross-dataset evaluations, thereby highlighting its excellent generalization capabilities. The superior performance on small datasets further underscores our method's remarkable ability to handle unseen data beyond the training set.

Face anti-spoofing with cross-stage relation enhancement and spoof material perception.

Face Anti-Spoofing (FAS) seeks to protect face recognition systems from spoofing attacks, which is applied extensively in scenarios such as access control, electronic payment, and security surveillance systems. Face anti-spoofing requires the integration of local details and global semantic information. Existing CNN-based methods rely on small stride or image patch-based feature extraction structures, which struggle to capture spatial and cross-layer feature correlations effectively. Meanwhile, Transformer-based methods have limitations in extracting discriminative detailed features. To address the aforementioned issues, we introduce a multi-stage CNN-Transformer-based framework, which extracts local features through the convolutional layer and long-distance feature relationships via self-attention. Based on this, we proposed a cross-attention multi-stage feature fusion, employing semantically high-stage features to query task-relevant features in low-stage features for further cross-stage feature fusion. To enhance the discrimination of local features for subtle differences, we design pixel-wise material classification supervision and add a auxiliary branch in the intermediate layers of the model. Moreover, to address the limitations of a single acquisition environment and scarcity of acquisition devices in the existing Near-Infrared dataset, we create a large-scale Near-Infrared Face Anti-Spoofing dataset with 380k pictures of 1040 identities. The proposed method could achieve the state-of-the-art in OULU-NPU and our proposed Near-Infrared dataset at just 1.3GFlops and 3.2M parameter numbers, which demonstrate the effective of the proposed method.

Gaussian-Filtered High-Frequency-Feature Trained Optimized BiLSTM Network for Spoofed-Speech Classification.

Voice-controlled devices are in demand due to their hands-free controls. However, using voice-controlled devices in sensitive scenarios like smartphone applications and financial transactions requires protection against fraudulent attacks referred to as "speech spoofing". The algorithms used in spoof attacks are practically unknown; hence, further analysis and development of spoof-detection models for improving spoof classification are required. A study of the spoofed-speech spectrum suggests that high-frequency features are able to discriminate genuine speech from spoofed speech well. Typically, linear or triangular filter banks are used to obtain high-frequency features. However, a Gaussian filter can extract more global information than a triangular filter. In addition, MFCC features are preferable among other speech features because of their lower covariance. Therefore, in this study, the use of a Gaussian filter is proposed for the extraction of inverted MFCC (iMFCC) features, providing high-frequency features. Complementary features are integrated with iMFCC to strengthen the features that aid in the discrimination of spoof speech. Deep learning has been proven to be efficient in classification applications, but the selection of its hyper-parameters and architecture is crucial and directly affects performance. Therefore, a Bayesian algorithm is used to optimize the BiLSTM network. Thus, in this study, we build a high-frequency-based optimized BiLSTM network to classify the spoofed-speech signal, and we present an extensive investigation using the ASVSpoof 2017 dataset. The optimized BiLSTM model is successfully trained with the least epoch and achieved a 99.58% validation accuracy. The proposed algorithm achieved a 6.58% EER on the evaluation dataset, with a relative improvement of 78% on a baseline spoof-identification system.

A survey on face presentation attack detection mechanisms: hitherto and future perspectives.

The advances in human face recognition (FR) systems have recorded sublime success for automatic and secured authentication in diverse domains. Although the traditional methods have been overshadowed by face recognition counterpart during this progress, computer vision gains rapid traction, and the modern accomplishments address problems with real-world complexity. However, security threats in FR-based systems are a growing concern that offers a new-fangled track to the research community. In particular, recent past has witnessed ample instances of spoofing attacks where imposter breaches security of the system with an artifact of human face to circumvent the sensor module. Therefore, presentation attack detection (PAD) capabilities are instilled in the system for discriminating genuine and fake traits and anticipation of their impact on the overall behavior of the FR-based systems. To scrutinize exhaustively the current state-of-the-art efforts, provide insights, and identify potential research directions on face PAD mechanisms, this systematic study presents a review of face anti-spoofing techniques that use computational approaches. The study includes advancements in face PAD mechanisms ranging from traditional hardware-based solutions to up-to-date handcrafted features or deep learning-based approaches. We also present an analytical overview of face artifacts, performance protocols, and benchmark face anti-spoofing datasets. In addition, we perform analysis of the twelve recent state-of-the-art (SOTA) face PAD techniques on a common platform using identical dataset (i.e., REPLAY-ATTACK) and performance protocols (i.e., HTER and ACA). Our overall analysis investigates that despite prevalent face PAD mechanisms demonstrate potential performance, there exist some crucial issues that requisite a futuristic attention. Our analysis put forward a number of open issues such as; limited generalization to unknown attacks, inadequacy of face datasets for DL-models, training models with new fakes, efficient DL-enabled face PAD with smaller datasets, and limited discrimination of handcrafted features. Furthermore, the COVID-19 pandemic is an additional challenge to the existing face-based recognition systems, and hence to the PAD methods. Our motive is to present a complete reference of studies in this field and orient researchers to promising directions.

Multi-Domain Feature Alignment for Face Anti-Spoofing.

Face anti-spoofing is critical for enhancing the robustness of face recognition systems against presentation attacks. Existing methods predominantly rely on binary classification tasks. Recently, methods based on domain generalization have yielded promising results. However, due to distribution discrepancies between various domains, the differences in the feature space related to the domain considerably hinder the generalization of features from unfamiliar domains. In this work, we propose a multi-domain feature alignment framework (MADG) that addresses poor generalization when multiple source domains are distributed in the scattered feature space. Specifically, an adversarial learning process is designed to narrow the differences between domains, achieving the effect of aligning the features of multiple sources, thus resulting in multi-domain alignment. Moreover, to further improve the effectiveness of our proposed framework, we incorporate multi-directional triplet loss to achieve a higher degree of separation in the feature space between fake and real faces. To evaluate the performance of our method, we conducted extensive experiments on several public datasets. The results demonstrate that our proposed approach outperforms current state-of-the-art methods, thereby validating its effectiveness in face anti-spoofing.

DEFAEK: Domain Effective Fast Adaptive Network for Face Anti-Spoofing.

Existing deep learning based face anti-spoofing (FAS) or deepfake detection approaches usually rely on large-scale datasets and powerful networks with significant amount of parameters to achieve satisfactory performance. However, these make them resource-heavy and unsuitable for handheld devices. Moreover, they are limited by the types of spoof in the dataset they train on and require considerable training time. To produce a robust FAS model, they need large datasets covering the widest variety of predefined presentation attacks possible. Testing on new or unseen attacks or environments generally results in poor performance. Ideally, the FAS model should learn discriminative features that can generalize well even on unseen spoof types. In this paper, we propose a fast learning approach called Domain Effective Fast Adaptive nEt-worK (DEFAEK), a face anti-spoofing approach based on the optimization-based meta-learning paradigm that effectively and quickly adapts to new tasks. DEFAEK treats differences in an environment as domains and simulates multiple domain shifts during training. To further improve the effectiveness and efficiency of meta-learning, we adopt the metric learning in the inner loop update with careful sample selection. With extensive experiments on the challenging CelebA-Spoof and FaceForensics++ datasets, the evaluation results show that DEFAEK can learn cues independent of the environment with good generalization capability. In addition, the resulting model is lightweight following the design principle of modern lightweight network architecture and still generalizes well on unseen classes. In addition, we also demonstrate our model's capabilities by comparing the numbers of parameters, FLOPS, and model performance with other state-of-the-art methods.

Anti-spoofing-enabled Contactless Attendance Monitoring System in the COVID-19 Pandemic.

The Global Novel Coronavirus Disease-2019 (COVID-19) pandemic has forced social distancing norms that have been followed worldwide. Thus, traditional biometric-based attendance marking systems are replaced with contactless attendance marking schemes. However, there are limitations of manufacturing cost, spoofing attacks, and security vulnerabilities. Thus, the paper proposes a contactless camera-based attendance system with the equipped functionalities of anti-spoofing. The proposed scheme can detect liveliness, so fake attendance marking is eliminated. The proposed scheme is also scalable and cost-effective, with generic solutions adaptable to schools, colleges, or other places where attendance is required. The system also eliminates the limitation of one-entry by multiple face-marking systems that allow simultaneous attendance marking. In performance analysis, parameters like image precision, storage cost, retrieval latency, and analysis of the anti-spoofing module is presented against existing schemes. An accuracy of 95.85% is reported for the model, with a significant improvement of 33.52% in storage cost through the Firebase database, which outperforms existing state-of-the-art schemes.

Can Hierarchical Transformers Learn Facial Geometry?

Human faces are a core part of our identity and expression, and thus, understanding facial geometry is key to capturing this information. Automated systems that seek to make use of this information must have a way of modeling facial features in a way that makes them accessible. Hierarchical, multi-level architectures have the capability of capturing the different resolutions of representation involved. In this work, we propose using a hierarchical transformer architecture as a means of capturing a robust representation of facial geometry. We further demonstrate the versatility of our approach by using this transformer as a backbone to support three facial representation problems: face anti-spoofing, facial expression representation, and deepfake detection. The combination of effective fine-grained details alongside global attention representations makes this architecture an excellent candidate for these facial representation problems. We conduct numerous experiments first showcasing the ability of our approach to address common issues in facial modeling (pose, occlusions, and background variation) and capture facial symmetry, then demonstrating its effectiveness on three supplemental tasks.

Deep Learning Based One-Class Detection System for Fake Faces Generated by GAN Network.

Recently, the dangers associated with face generation technology have been attracting much attention in image processing and forensic science. The current face anti-spoofing methods based on Generative Adversarial Networks (GANs) suffer from defects such as overfitting and generalization problems. This paper proposes a new generation method using a one-class classification model to judge the authenticity of facial images for the purpose of realizing a method to generate a model that is as compatible as possible with other datasets and new data, rather than strongly depending on the dataset used for training. The method proposed in this paper has the following features: (a) we adopted various filter enhancement methods as basic pseudo-image generation methods for data enhancement; (b) an improved Multi-Channel Convolutional Neural Network (MCCNN) was adopted as the main network, making it possible to accept multiple preprocessed data individually, obtain feature maps, and extract attention maps; (c) as a first ingenuity in training the main network, we augmented the data using weakly supervised learning methods to add attention cropping and dropping to the data; (d) as a second ingenuity in training the main network, we trained it in two steps. In the first step, we used a binary classification loss function to ensure that known fake facial features generated by known GAN networks were filtered out. In the second step, we used a one-class classification loss function to deal with the various types of GAN networks or unknown fake face generation methods. We compared our proposed method with four recent methods. Our experiments demonstrate that the proposed method improves cross-domain detection efficiency while maintaining source-domain accuracy. These studies show one possible direction for improving the correct answer rate in judging facial image authenticity, thereby making a great contribution both academically and practically.

Multichannel Acoustic Spectroscopy of the Human Body for Inviolable Biometric Authentication.

Specific features of the human body, such as fingerprint, iris, and face, are extensively used in biometric authentication. Conversely, the internal structure and material features of the body have not been explored extensively in biometrics. Bioacoustics technology is suitable for extracting information about the internal structure and biological and material characteristics of the human body. Herein, we report a biometric authentication method that enables multichannel bioacoustic signal acquisition with a systematic approach to study the effects of selectively distilled frequency features, increasing the number of sensing channels with respect to multiple fingers. The accuracy of identity recognition according to the number of sensing channels and the number of selectively chosen frequency features was evaluated using exhaustive combination searches and forward-feature selection. The technique was applied to test the accuracy of machine learning classification using 5,232 datasets from 54 subjects. By optimizing the scanning frequency and sensing channels, our method achieved an accuracy of 99.62%, which is comparable to existing biometric methods. Overall, the proposed biometric method not only provides an unbreakable, inviolable biometric but also can be applied anywhere in the body and can substantially broaden the use of biometrics by enabling continuous identity recognition on various body parts for biometric identity authentication.

Enriching Facial Anti-Spoofing Datasets via an Effective Face Swapping Framework.

In the era of rapid development of the Internet of things, deep learning, and communication technologies, social media has become an indispensable element. However, while enjoying the convenience brought by technological innovation, people are also facing the negative impact brought by them. Taking the users' portraits of multimedia systems as examples, with the maturity of deep facial forgery technologies, personal portraits are facing malicious tampering and forgery, which pose a potential threat to personal privacy security and social impact. At present, the deep forgery detection methods are learning-based methods, which depend on the data to a certain extent. Enriching facial anti-spoofing datasets is an effective method to solve the above problem. Therefore, we propose an effective face swapping framework based on StyleGAN. We utilize the feature pyramid network to extract facial features and map them to the latent space of StyleGAN. In order to realize the transformation of identity, we explore the representation of identity information and propose an adaptive identity editing module. We design a simple and effective post-processing process to improve the authenticity of the images. Experiments show that our proposed method can effectively complete face swapping and provide high-quality data for deep forgery detection to ensure the security of multimedia systems.

A Slowly Varying Spoofing Algorithm on Loosely Coupled GNSS/IMU Avoiding Multiple Anti-Spoofing Techniques.

When satellite navigation terminal sensors encounter malicious signal spoofing or interference, if attention is not paid to improving their anti-spoofing ability, the performance of the sensors will be seriously affected. The global navigation satellite system (GNSS) spoofing has gradually become a research hotspot of the jammer because of its great harm and high concealment. In the face of more and more sensors coupling GNSS and inertial measurement unit (IMU) to varying degrees and configuring a variety of anti-spoofing techniques to effectively detect spoofing, even if the spoofer intends to gradually pull the positioning results, if the spoofing strategy is unreasonable, the parameters of the coupled filter output and spoofing observation measurement will lose their rationality, which will lead to the spoofing being detected. To solve the above problems, in order to effectively counter the non-cooperative target sensors of assembling loosely coupled GNSS/IMU using GNSS spoofing, based on the analysis of the influence mechanism of spoofing on the positioning of loosely coupled GNSS/IMU, a slowly varying spoofing algorithm to avoid loosely coupled GNSS/IMU with multiple anti-spoofing techniques is proposed in this paper, and a measurement deviation determination method to avoid multiple anti-spoofing techniques is proposed, which can gradually pull the positioning results of the coupled system and successfully avoid the detection of anti-spoofing techniques of innovation sequence monitoring and a rationality check on parameters. Simulation experimental results show that the proposed algorithm gradually changes the positioning of loosely coupled GNSS/IMU, the north and east displacements achieve the purpose of spoofing, and error with expected offset is -0.2 m and 2.3 m, respectively. Down displacement also basically achieves the purpose of spoofing, and error with the expected offset is 13.2 m. At the same time, the spoofer avoids the detection of multiple anti-spoofing techniques, does not trigger the system alarm, and realizes the purpose of spoofing; thus, the effectiveness and high concealment of the spoofing algorithm are verified.

Face Biometric Spoof Detection Method Using a Remote Photoplethysmography Signal.

Spoofing attacks in face recognition systems are easy because faces are always exposed. Various remote photoplethysmography-based methods to detect face spoofing have been developed. However, they are vulnerable to replay attacks. In this study, we propose a remote photoplethysmography-based face recognition spoofing detection method that minimizes the susceptibility to certain database dependencies and high-quality replay attacks without additional devices. The proposed method has the following advantages. First, because only an RGB camera is used to detect spoofing attacks, the proposed method is highly usable in various mobile environments. Second, solutions are incorporated in the method to obviate new attack scenarios that have not been previously dealt with. In this study, we propose a remote photoplethysmography-based face recognition spoofing detection method that improves susceptibility to certain database dependencies and high-quality replay attack, which are the limitations of previous methods without additional devices. In the experiment, we also verified the cut-off attack scenario in the jaw and cheek area where the proposed method can be counter-attacked. By using the time series feature and the frequency feature of the remote photoplethysmography signal, it was confirmed that the accuracy of spoof detection was 99.7424%.

Fingerprint Presentation Attack Detection Utilizing Spatio-Temporal Features.

This paper presents a novel mechanism for fingerprint dynamic presentation attack detection. We utilize five spatio-temporal feature extractors to efficiently eliminate and mitigate different presentation attack species. The feature extractors are selected such that the fingerprint ridge/valley pattern is consolidated with the temporal variations within the pattern in fingerprint videos. An SVM classification scheme, with a second degree polynomial kernel, is used in our presentation attack detection subsystem to classify bona fide and attack presentations. The experiment protocol and evaluation are conducted following the ISO/IEC 30107-3:2017 standard. Our proposed approach demonstrates efficient capability of detecting presentation attacks with significantly low BPCER where BPCER is 1.11% for an optical sensor and 3.89% for a thermal sensor at 5% APCER for both.

Convolutional Neural Network Approach for Multispectral Facial Presentation Attack Detection in Automated Border Control Systems.

Automated border control systems are the first critical infrastructure point when crossing a border country. Crossing border lines for unauthorized passengers is a high security risk to any country. This paper presents a multispectral analysis of presentation attack detection for facial biometrics using the learned features from a convolutional neural network. Three sensors are considered to design and develop a new database that is composed of visible (VIS), near-infrared (NIR), and thermal images. Most studies are based on laboratory or ideal conditions-controlled environments. However, in a real scenario, a subject's situation is completely modified due to diverse physiological conditions, such as stress, temperature changes, sweating, and increased blood pressure. For this reason, the added value of this study is that this database was acquired in situ. The attacks considered were printed, masked, and displayed images. In addition, five classifiers were used to detect the presentation attack. Note that thermal sensors provide better performance than other solutions. The results present better outputs when all sensors are used together, regardless of whether classifier or feature-level fusion is considered. Finally, classifiers such as KNN or SVM show high performance and low computational level.

A Survey on Anti-Spoofing Methods for Facial Recognition with RGB Cameras of Generic Consumer Devices.

The widespread deployment of facial recognition-based biometric systems has made facial presentation attack detection (face anti-spoofing) an increasingly critical issue. This survey thoroughly investigates facial Presentation Attack Detection (PAD) methods that only require RGB cameras of generic consumer devices over the past two decades. We present an attack scenario-oriented typology of the existing facial PAD methods, and we provide a review of over 50 of the most influenced facial PAD methods over the past two decades till today and their related issues. We adopt a comprehensive presentation of the reviewed facial PAD methods following the proposed typology and in chronological order. By doing so, we depict the main challenges, evolutions and current trends in the field of facial PAD and provide insights on its future research. From an experimental point of view, this survey paper provides a summarized overview of the available public databases and an extensive comparison of the results reported in PAD-reviewed papers.

Bio-Inspired Presentation Attack Detection for Face Biometrics.

Today, face biometric systems are becoming widely accepted as a standard method for identity authentication in many security settings. For example, their deployment in automated border control gates plays a crucial role in accurate document authentication and reduced traveler flow rates in congested border zones. The proliferation of such systems is further spurred by the advent of portable devices. On the one hand, modern smartphone and tablet cameras have in-built user authentication applications while on the other hand, their displays are being consistently exploited for face spoofing. Similar to biometric systems of other physiological biometric identifiers, face biometric systems have their own unique set of potential vulnerabilities. In this work, these vulnerabilities (presentation attacks) are being explored via a biologically-inspired presentation attack detection model which is termed "BIOPAD." Our model employs Gabor features in a feedforward hierarchical structure of layers that progressively process and train from visual information of people's faces, along with their presentation attacks, in the visible and near-infrared spectral regions. BIOPAD's performance is directly compared with other popular biologically-inspired layered models such as the "Hierarchical Model And X" (HMAX) that applies similar handcrafted features, and Convolutional Neural Networks (CNN) that discover low-level features through stochastic descent training. BIOPAD shows superior performance to both HMAX and CNN in all of the three presentation attack databases examined and these results were consistent in two different classifiers (Support Vector Machine and k-nearest neighbor). In certain cases, our findings have shown that BIOPAD can produce authentication rates with 99% accuracy. Finally, we further introduce a new presentation attack database with visible and near-infrared information for direct comparisons. Overall, BIOPAD's operation, which is to fuse information from different spectral bands at both feature and score levels for the purpose of face presentation attack detection, has never been attempted before with a biologically-inspired algorithm. Obtained detection rates are promising and confirm that near-infrared visual information significantly assists in overcoming presentation attacks.

The Effectiveness of Depth Data in Liveness Face Authentication Using 3D Sensor Cameras.

Even though biometric technology increases the security of systems that use it, they are prone to spoof attacks where attempts of fraudulent biometrics are used. To overcome these risks, techniques on detecting liveness of the biometric measure are employed. For example, in systems that utilise face authentication as biometrics, a liveness is assured using an estimation of blood flow, or analysis of quality of the face image. Liveness assurance of the face using real depth technique is rarely used in biometric devices and in the literature, even with the availability of depth datasets. Therefore, this technique of employing 3D cameras for liveness of face authentication is underexplored for its vulnerabilities to spoofing attacks. This research reviews the literature on this aspect and then evaluates the liveness detection to suggest solutions that account for the weaknesses found in detecting spoofing attacks. We conduct a proof-of-concept study to assess the liveness detection of 3D cameras in three devices, where the results show that having more flexibility resulted in achieving a higher rate in detecting spoofing attacks. Nonetheless, it was found that selecting a wide depth range of the 3D camera is important for anti-spoofing security recognition systems such as surveillance cameras used in airports. Therefore, to utilise the depth information and implement techniques that detect faces regardless of the distance, a 3D camera with long maximum depth range (e.g., 20 m) and high resolution stereo cameras could be selected, which can have a positive impact on accuracy.

Spoofing Detection and Mitigation in a Multi-correlator GPS Receiver Based on the Maximum Likelihood Principle.

As a structural interference, spoofing is difficult to detect by the target receiver while the advent of a repeater makes the implementation of spoofing much easier. Most existing anti-spoofing methods are merely capable of detecting the spoofing, i.e., they cannot effectively remove counterfeit signals. Therefore, based on the similarities between multipath and spoofing, the feasibility of applying multipath mitigation methods to anti-spoofing is first analyzed in this paper. We then propose a novel algorithm based on maximum likelihood (ML) estimation to resolve this problem. The tracking channels with multi-correlators are constructed and a set of corresponding steps of detecting and removing the counterfeit signals is designed to ensure that the receiver locks the authentic signals in the presence of spoofing. Finally, the spoofing is successfully executed with a software receiver and the saved intermediate frequency (IF) signals, on this basis, the effectiveness of the proposed algorithm is verified by experiments.