Decoupling visual and identity features for adversarial palm-vein image attack.

Palm-vein has been widely used for biometric recognition due to its resistance to theft and forgery. However, with the emergence of adversarial attacks, most existing palm-vein recognition methods are vulnerable to adversarial image attacks, and to the best of our knowledge, there is still no study specifically focusing on palm-vein image attacks. In this paper, we propose an adversarial palm-vein image attack network that generates highly similar adversarial palm-vein images to the original samples, but with altered palm-identities. Unlike most existing generator-oriented methods that directly learn image features via concatenated convolutional layers, our proposed network first maps palm-vein images into multi-scale high-dimensional shallow representation, and then develops attention-based dual-path feature learning modules to extensively exploit diverse palm-vein-specific features. After that, we design visual-consistency and identity-aware loss functions to specially decouple the visual and identity features to reconstruct the adversarial palm-vein images. By doing this, the visual characteristics of palm-vein images can be largely preserved while the identity information is removed in the adversarial palm-vein images, such that high-aggressive adversarial palm-vein samples can be obtained. Extensive white-box and black-box attack experiments conducted on three widely used databases clearly show the effectiveness of the proposed network.

Model and Method for Providing Resilience to Resource-Constrained AI-System.

Artificial intelligence technologies are becoming increasingly prevalent in resource-constrained, safety-critical embedded systems. Numerous methods exist to enhance the resilience of AI systems against disruptive influences. However, when resources are limited, ensuring cost-effective resilience becomes crucial. A promising approach for reducing the resource consumption of AI systems during test-time involves applying the concepts and methods of dynamic neural networks. Nevertheless, the resilience of dynamic neural networks against various disturbances remains underexplored. This paper proposes a model architecture and training method that integrate dynamic neural networks with a focus on resilience. Compared to conventional training methods, the proposed approach yields a 24% increase in the resilience of convolutional networks and a 19.7% increase in the resilience of visual transformers under fault injections. Additionally, it results in a 16.9% increase in the resilience of convolutional network ResNet-110 and a 21.6% increase in the resilience of visual transformer DeiT-S under adversarial attacks, while saving more than 30% of computational resources. Meta-training the neural network model improves resilience to task changes by an average of 22%, while achieving the same level of resource savings.

Quantum-inspired analysis of neural network vulnerabilities: the role of conjugate variables in system attacks.

Neural networks demonstrate vulnerability to small, non-random perturbations, emerging as adversarial attacks. Such attacks, born from the gradient of the loss function relative to the input, are discerned as input conjugates, revealing a systemic fragility within the network structure. Intriguingly, a mathematical congruence manifests between this mechanism and the quantum physics' uncertainty principle, casting light on a hitherto unanticipated interdisciplinarity. This inherent susceptibility within neural network systems is generally intrinsic, highlighting not only the innate vulnerability of these networks, but also suggesting potential advancements in the interdisciplinary area for understanding these black-box networks.

Adversarial Robustness Enhancement for Deep Learning-Based Soft Sensors: An Adversarial Training Strategy Using Historical Gradients and Domain Adaptation.

Despite their high prediction accuracy, deep learning-based soft sensor (DLSS) models face challenges related to adversarial robustness against malicious adversarial attacks, which hinder their widespread deployment and safe application. Although adversarial training is the primary method for enhancing adversarial robustness, existing adversarial-training-based defense methods often struggle with accurately estimating transfer gradients and avoiding adversarial robust overfitting. To address these issues, we propose a novel adversarial training approach, namely domain-adaptive adversarial training (DAAT). DAAT comprises two stages: historical gradient-based adversarial attack (HGAA) and domain-adaptive training. In the first stage, HGAA incorporates historical gradient information into the iterative process of generating adversarial samples. It considers gradient similarity between iterative steps to stabilize the updating direction, resulting in improved transfer gradient estimation and stronger adversarial samples. In the second stage, a soft sensor domain-adaptive training model is developed to learn common features from adversarial and original samples through domain-adaptive training, thereby avoiding excessive leaning toward either side and enhancing the adversarial robustness of DLSS without robust overfitting. To demonstrate the effectiveness of DAAT, a DLSS model for crystal quality variables in silicon single-crystal growth manufacturing processes is used as a case study. Through DAAT, the DLSS achieves a balance between defense against adversarial samples and prediction accuracy on normal samples to some extent, offering an effective approach for enhancing the adversarial robustness of DLSS.

Removing Adversarial Noise in X-ray Images via Total Variation Minimization and Patch-Based Regularization for Robust Deep Learning-based Diagnosis.

Deep learning has significantly advanced the field of radiology-based disease diagnosis, offering enhanced accuracy and efficiency in detecting various medical conditions through the analysis of complex medical images such as X-rays. This technology's ability to discern subtle patterns and anomalies has proven invaluable for swift and accurate disease identification. The relevance of deep learning in radiology has been particularly highlighted during the COVID-19 pandemic, where rapid and accurate diagnosis is crucial for effective treatment and containment. However, recent research has uncovered vulnerabilities in deep learning models when exposed to adversarial attacks, leading to incorrect predictions. In response to this critical challenge, we introduce a novel approach that leverages total variation minimization to combat adversarial noise within X-ray images effectively. Our focus narrows to COVID-19 diagnosis as a case study, where we initially construct a classification model through transfer learning designed to accurately classify lung X-ray images encompassing no pneumonia, COVID-19 pneumonia, and non-COVID pneumonia cases. Subsequently, we extensively evaluated the model's susceptibility to targeted and un-targeted adversarial attacks by employing the fast gradient sign gradient (FGSM) method. Our findings reveal a substantial reduction in the model's performance, with the average accuracy plummeting from 95.56 to 19.83% under adversarial conditions. However, the experimental results demonstrate the exceptional efficacy of the proposed denoising approach in enhancing the performance of diagnosis models when applied to adversarial examples. Post-denoising, the model exhibits a remarkable accuracy improvement, surging from 19.83 to 88.23% on adversarial images. These promising outcomes underscore the potential of denoising techniques to fortify the resilience and reliability of AI-based COVID-19 diagnostic systems, laying the foundation for their successful deployment in clinical settings.

HyGloadAttack: Hard-label black-box textual adversarial attacks via hybrid optimization.

Hard-label black-box textual adversarial attacks present a highly challenging task due to the discrete and non-differentiable nature of text data and the lack of direct access to the model's predictions. Research in this issue is still in its early stages, and the performance and efficiency of existing methods has potential for improvement. For instance, exchange-based and gradient-based attacks may become trapped in local optima and require excessive queries, hindering the generation of adversarial examples with high semantic similarity and low perturbation under limited query conditions. To address these issues, we propose a novel framework called HyGloadAttack (adversarial Attacks via Hybrid optimization and Global random initialization) for crafting high-quality adversarial examples. HyGloadAttack utilizes a perturbation matrix in the word embedding space to find nearby adversarial examples after global initialization and selects synonyms that maximize similarity while maintaining adversarial properties. Furthermore, we introduce a gradient-based quick search method to accelerate the search process of optimization. Extensive experiments on five datasets of text classification and natural language inference, as well as two real APIs, demonstrate the significant superiority of our proposed HyGloadAttack method over state-of-the-art baseline methods.

FDAA: A feature distribution-aware transferable adversarial attack method.

In recent years, the research on transferable feature-level adversarial attack has become a hot spot due to attacking unknown deep neural networks successfully. But the following problems limit its transferability. Existing feature disruption methods often focus on computing feature weights precisely, while overlooking the noise influence of feature maps, which results in disturbing non-critical features. Meanwhile, geometric augmentation algorithms are used to enhance image diversity but compromise information integrity, which hamper models from capturing comprehensive features. Furthermore, current feature perturbation could not pay attention to the density distribution of object-relevant key features, which mainly concentrate in salient region and fewer in the most distributed background region, and get limited transferability. To tackle these challenges, a feature distribution-aware transferable adversarial attack method, called FDAA, is proposed to implement distinct strategies for different image regions in the paper. A novel Aggregated Feature Map Attack (AFMA) is presented to significantly denoise feature maps, and an input transformation strategy, called Smixup, is introduced to help feature disruption algorithms to capture comprehensive features. Extensive experiments demonstrate that scheme proposed achieves better transferability with an average success rate of 78.6% on adversarially trained models.

Reidentification of Participants in Shared Clinical Data Sets: Experimental Study.

BACKGROUND: Large curated data sets are required to leverage speech-based tools in health care. These are costly to produce, resulting in increased interest in data sharing. As speech can potentially identify speakers (ie, voiceprints), sharing recordings raises privacy concerns. This is especially relevant when working with patient data protected under the Health Insurance Portability and Accountability Act. OBJECTIVE: We aimed to determine the reidentification risk for speech recordings, without reference to demographics or metadata, in clinical data sets considering both the size of the search space (ie, the number of comparisons that must be considered when reidentifying) and the nature of the speech recording (ie, the type of speech task). METHODS: Using a state-of-the-art speaker identification model, we modeled an adversarial attack scenario in which an adversary uses a large data set of identified speech (hereafter, the known set) to reidentify as many unknown speakers in a shared data set (hereafter, the unknown set) as possible. We first considered the effect of search space size by attempting reidentification with various sizes of known and unknown sets using VoxCeleb, a data set with recordings of natural, connected speech from >7000 healthy speakers. We then repeated these tests with different types of recordings in each set to examine whether the nature of a speech recording influences reidentification risk. For these tests, we used our clinical data set composed of recordings of elicited speech tasks from 941 speakers. RESULTS: We found that the risk was inversely related to the number of comparisons an adversary must consider (ie, the search space), with a positive linear correlation between the number of false acceptances (FAs) and the number of comparisons (r=0.69; P<.001). The true acceptances (TAs) stayed relatively stable, and the ratio between FAs and TAs rose from 0.02 at 1 × 105 comparisons to 1.41 at 6 × 106 comparisons, with a near 1:1 ratio at the midpoint of 3 × 106 comparisons. In effect, risk was high for a small search space but dropped as the search space grew. We also found that the nature of a speech recording influenced reidentification risk, with nonconnected speech (eg, vowel prolongation: FA/TA=98.5; alternating motion rate: FA/TA=8) being harder to identify than connected speech (eg, sentence repetition: FA/TA=0.54) in cross-task conditions. The inverse was mostly true in within-task conditions, with the FA/TA ratio for vowel prolongation and alternating motion rate dropping to 0.39 and 1.17, respectively. CONCLUSIONS: Our findings suggest that speaker identification models can be used to reidentify participants in specific circumstances, but in practice, the reidentification risk appears small. The variation in risk due to search space size and type of speech task provides actionable recommendations to further increase participant privacy and considerations for policy regarding public release of speech recordings.

Adversarial attacks and adversarial training for burn image segmentation based on deep learning.

Deep learning has been widely applied in the fields of image classification and segmentation, while adversarial attacks can impact the model's results in image segmentation and classification. Especially in medical images, due to constraints from factors like shooting angles, environmental lighting, and diverse photography devices, medical images typically contain various forms of noise. In order to address the impact of these physically meaningful disturbances on existing deep learning models in the application of burn image segmentation, we simulate attack methods inspired by natural phenomena and propose an adversarial training approach specifically designed for burn image segmentation. The method is tested on our burn dataset. Through the defensive training using our approach, the segmentation accuracy of adversarial samples, initially at 54%, is elevated to 82.19%, exhibiting a 1.97% improvement compared to conventional adversarial training methods, while substantially reducing the training time. Ablation experiments validate the effectiveness of individual losses, and we assess and compare training results with different adversarial samples using various metrics.

Implications of Minimum Description Length for Adversarial Attack in Natural Language Processing.

Investigating causality to establish novel criteria for training robust natural language processing (NLP) models is an active research area. However, current methods face various challenges such as the difficulties in identifying keyword lexicons and obtaining data from multiple labeled environments. In this paper, we study the problem of robust NLP from a complementary but different angle: we treat the behavior of an attack model as a complex causal mechanism and quantify its algorithmic information using the minimum description length (MDL) framework. Specifically, we use masked language modeling (MLM) to measure the "amount of effort" needed to transform from the original text to the altered text. Based on that, we develop techniques for judging whether a specified set of tokens has been altered by the attack, even in the absence of the original text data.

Model-data-driven adversarial active learning for brain tumor segmentation.

Active learning (AL) attempts to select informative samples in a dataset to minimize the number of required labels while maximizing the performance of the model. Current AL in segmentation tasks is limited to the expansion of popular classification-based methods including entropy, MC-dropout, etc. Meanwhile, most applications in the medical field are simply migrations that fail to consider the nature of medical images, such as high class imbalance, high domain difference, and data scarcity. In this study, we address these challenges and propose a novel AL framework for medical image segmentation task. Our approach introduces a pseudo-label-based filter addressing excessive blank patches in medical abnormalities segmentation tasks, e.g., lesions, and tumors, used before the AL selection. This filter helps reduce resource usage and allows the model to focus on selecting more informative samples. For the sample selection, we propose a novel query strategy that combines both model impact and data stability by employing adversarial attack. Furthermore, we harness the adversarial samples generated during the query process to enhance the robustness of the model. The experimental results verify our framework's effectiveness over various state-of-the-art methods. Our proposed method only needs less than 14% annotated patches in 3D brain MRI multiple sclerosis (MS) segmentation tasks and 20% for Low-Grade Glioma (LGG) tumor segmentation to achieve competitive results with full supervision. These promising outcomes not only improve performance but alleviate the time burden associated with expert annotation, thereby facilitating further advancements in the field of medical image segmentation. Our code is available at https://github.com/HelenMa9998/adversarial_active_learning.

Blinding and blurring the multi-object tracker with adversarial perturbations.

Adversarial attack reveals a potential imperfection in deep models that they are susceptible to being tricked by imperceptible perturbations added to images. Recent deep multi-object trackers combine the functionalities of detection and association, rendering attacks on either the detector or the association component an effective means of deception. Existing attacks focus on increasing the frequency of ID switching, which greatly damages tracking stability, but is not enough to make the tracker completely ineffective. To fully explore the potential of adversarial attacks, we propose Blind-Blur Attack (BBA), a novel attack method based on spatio-temporal motion information to fool multi-object trackers. Specifically, a simple but efficient perturbation generator is trained with the blind-blur loss, simultaneously making the target invisible to the tracker and letting the background be regarded as moving targets. We take TraDeS as our main research tracker, and verify our attack method on other excellent algorithms (i.e., CenterTrack, FairMOT, and ByteTrack) on MOT-Challenge benchmark datasets (i.e., MOT16, MOT17, and MOT20). BBA attack reduced the MOTA of TraDeS and ByteTrack from 69.1 and 80.3 to -238.1 and -357.0, respectively, indicating that it is an efficient method with a high degrees of transferability.

HAG-NET: Hiding Data and Adversarial Attacking with Generative Adversarial Network.

Recent studies on watermarking techniques based on image carriers have demonstrated new approaches that combine adversarial perturbations against steganalysis with embedding distortions. However, while these methods successfully counter convolutional neural network-based steganalysis, they do not adequately protect the data of the carrier itself. Recognizing the high sensitivity of Deep Neural Networks (DNNs) to small perturbations, we propose HAG-NET, a method based on image carriers, which is jointly trained by the encoder, decoder, and attacker. In this paper, the encoder generates Adversarial Steganographic Examples (ASEs) that are adversarial to the target classification network, thereby providing protection for the carrier data. Additionally, the decoder can recover secret data from ASEs. The experimental results demonstrate that ASEs produced by HAG-NET achieve an average success rate of over 99% on both the MNIST and CIFAR-10 datasets. ASEs generated with the attacker exhibit greater robustness in terms of attack ability, with an average increase of about 3.32%. Furthermore, our method, when compared with other generative stego examples under similar perturbation strength, contains significantly more information according to image information entropy measurements.

Defense against adversarial attacks based on color space transformation.

Deep Learning algorithms have achieved state-of-the-art performance in various important tasks. However, recent studies have found that an elaborate perturbation may cause a network to misclassify, which is known as an adversarial attack. Based on current research, it is suggested that adversarial examples cannot be eliminated completely. Consequently, it is always possible to determine an attack that is effective against a defense model. We render existing adversarial examples invalid by altering the classification boundaries. Meanwhile, for valid adversarial examples generated against the defense model, the adversarial perturbations are increased so that they can be distinguished by the human eye. This paper proposes a method for implementing the abovementioned concepts through color space transformation. Experiments on CIFAR-10, CIFAR-100, and Mini-ImageNet demonstrate the effectiveness and versatility of our defense method. To the best of our knowledge, this is the first defense model based on the amplification of adversarial perturbations.

Temporal shuffling for defending deep action recognition models against adversarial attacks.

Recently, video-based action recognition methods using convolutional neural networks (CNNs) achieve remarkable recognition performance. However, there is still lack of understanding about the generalization mechanism of action recognition models. In this paper, we suggest that action recognition models rely on the motion information less than expected, and thus they are robust to randomization of frame orders. Furthermore, we find that motion monotonicity remaining after randomization also contributes to such robustness. Based on this observation, we develop a novel defense method using temporal shuffling of input videos against adversarial attacks for action recognition models. Another observation enabling our defense method is that adversarial perturbations on videos are sensitive to temporal destruction. To the best of our knowledge, this is the first attempt to design a defense method without additional training for 3D CNN-based video action recognition models.

Black-box attacks on dynamic graphs via adversarial topology perturbations.

Research and analysis of attacks on dynamic graph is beneficial for information systems to investigate vulnerabilities and strength abilities in resisting malicious attacks. Existing attacks on dynamic graphs mainly focus on rewiring original graph structures, which are often infeasible in real-world scenarios. To address this issue, we adopt a novel strategy by injecting both fake nodes and links to attack dynamic graphs. Based on that, we present the first study on attacking dynamic graphs via adversarial topology perturbations in a restricted black-box setting, in which downstream graph learning tasks are unknown. Specifically, we first divide dynamic graph structure perturbations into three sub-tasks and transform them as a sequential decision making process. Then, we propose a hierarchical reinforcement learning based black-box attack (HRBBA) framework to model three sub-tasks as attack policies. In addition, an imperceptible perturbation constraint to guarantee the concealment of attacks is incorporated into HRBBA. Finally, HRBBA is optimized based on the actor-critic process. Extensive experiments on four real-world dynamic graphs show that the performance of diverse dynamic graph learning methods (victim methods) on tasks like link prediction, node classification and network clustering can be substantially degraded under HRBBA attack.

Local imperceptible adversarial attacks against human pose estimation networks.

Deep neural networks are vulnerable to attacks from adversarial inputs. Corresponding attack research on human pose estimation (HPE), particularly for body joint detection, has been largely unexplored. Transferring classification-based attack methods to body joint regression tasks is not straightforward. Another issue is that the attack effectiveness and imperceptibility contradict each other. To solve these issues, we propose local imperceptible attacks on HPE networks. In particular, we reformulate imperceptible attacks on body joint regression into a constrained maximum allowable attack. Furthermore, we approximate the solution using iterative gradient-based strength refinement and greedy-based pixel selection. Our method crafts effective perceptual adversarial attacks that consider both human perception and attack effectiveness. We conducted a series of imperceptible attacks against state-of-the-art HPE methods, including HigherHRNet, DEKR, and ViTPose. The experimental results demonstrate that the proposed method achieves excellent imperceptibility while maintaining attack effectiveness by significantly reducing the number of perturbed pixels. Approximately 4% of the pixels can achieve sufficient attacks on HPE.

Attention distraction with gradient sharpening for multi-task adversarial attack.

The advancement of deep learning has resulted in significant improvements on various visual tasks. However, deep neural networks (DNNs) have been found to be vulnerable to well-designed adversarial examples, which can easily deceive DNNs by adding visually imperceptible perturbations to original clean data. Prior research on adversarial attack methods mainly focused on single-task settings, i.e., generating adversarial examples to fool networks with a specific task. However, real-world artificial intelligence systems often require solving multiple tasks simultaneously. In such multi-task situations, the single-task adversarial attacks will have poor attack performance on the unrelated tasks. To address this issue, the generation of multi-task adversarial examples should leverage the generalization knowledge among multiple tasks and reduce the impact of task-specific information during the generation process. In this study, we propose a multi-task adversarial attack method to generate adversarial examples from a multi-task learning network by applying attention distraction with gradient sharpening. Specifically, we first attack the attention heat maps, which contain more generalization information than feature representations, by distracting the attention on the attack regions. Additionally, we use gradient-based adversarial example-generating schemes and propose to sharpen the gradients so that the gradients with multi-task information rather than only task-specific information can make a greater impact. Experimental results on the NYUD-V2 and PASCAL datasets demonstrate that the proposed method can improve the generalization ability of adversarial examples among multiple tasks and achieve better attack performance.

Adversarial attacks and defenses using feature-space stochasticity.

Recent studies in deep neural networks have shown that injecting random noise in the input layer of the networks contributes towards ℓp-norm-bounded adversarial perturbations. However, to defend against unrestricted adversarial examples, most of which are not ℓp-norm-bounded in the input layer, such input-layer random noise may not be sufficient. In the first part of this study, we generated a novel class of unrestricted adversarial examples termed feature-space adversarial examples. These examples are far from the original data in the input space but adjacent to the original data in a hidden-layer feature space and far again in the output layer. In the second part of this study, we empirically showed that while injecting random noise in the input layer was unable to defend these feature-space adversarial examples, they were defended by injecting random noise in the hidden layer. These results highlight the novel benefit of stochasticity in higher layers, in that it is useful for defending against these feature-space adversarial examples, a class of unrestricted adversarial examples.

Adv-BDPM: Adversarial attack based on Boundary Diffusion Probability Model.

Deep neural networks have become increasingly significant in our daily lives due to their remarkable performance. The issue of adversarial examples, which are responsible for the vulnerability problem of deep neural networks, has attracted the attention of researchers in the study of robustness of these networks. To address the issues caused by the restricted diversity and precision of adversarial perturbations in neural networks, we introduce a novel technique called Adversarial Boundary Diffusion Probability Modeling (Adv-BDPM). This approach combines boundary analysis and diffusion probability modeling. First, we combined the denoising diffusion probability model with the boundary loss to design the boundary diffusion probability model, which can generate corresponding boundary perturbations for a specific neural network. Then, through the iterative process of boundary perturbations and its corresponding orthogonal perturbations, we proposed a decision boundary search algorithm to generate adversarial samples. The comparison experiments with black-box attacks in ImageNet demonstrate that Adv-BDPM has better attack success rate and perturbation precision. The comparison experiments with white-box attacks in CIFAR-10 and CIFAR-100 demonstrate that Adv-BDPM has better attack success rate, attack diversity for the same sample, and can effectively defend against adversarial training with shorter running time.