To see the other types of publications on this topic, follow the link: Adversarial Attacker.
Journal articles on the topic 'Adversarial Attacker'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Adversarial Attacker.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Park, Sanglee, and Jungmin So. "On the Effectiveness of Adversarial Training in Defending against Adversarial Example Attacks for Image Classification." Applied Sciences 10, no. 22 (2020): 8079. http://dx.doi.org/10.3390/app10228079.
Full textAbstract:
State-of-the-art neural network models are actively used in various fields, but it is well-known that they are vulnerable to adversarial example attacks. Throughout the efforts to make the models robust against adversarial example attacks, it has been found to be a very difficult task. While many defense approaches were shown to be not effective, adversarial training remains as one of the promising methods. In adversarial training, the training data are augmented by “adversarial” samples generated using an attack algorithm. If the attacker uses a similar attack algorithm to generate adversarial examples, the adversarially trained network can be quite robust to the attack. However, there are numerous ways of creating adversarial examples, and the defender does not know what algorithm the attacker may use. A natural question is: Can we use adversarial training to train a model robust to multiple types of attack? Previous work have shown that, when a network is trained with adversarial examples generated from multiple attack methods, the network is still vulnerable to white-box attacks where the attacker has complete access to the model parameters. In this paper, we study this question in the context of black-box attacks, which can be a more realistic assumption for practical applications. Experiments with the MNIST dataset show that adversarially training a network with an attack method helps defending against that particular attack method, but has limited effect for other attack methods. In addition, even if the defender trains a network with multiple types of adversarial examples and the attacker attacks with one of the methods, the network could lose accuracy to the attack if the attacker uses a different data augmentation strategy on the target network. These results show that it is very difficult to make a robust network using adversarial training, even for black-box settings where the attacker has restricted information on the target network.
2
Rosenberg, Ishai, Asaf Shabtai, Yuval Elovici, and Lior Rokach. "Adversarial Machine Learning Attacks and Defense Methods in the Cyber Security Domain." ACM Computing Surveys 54, no. 5 (2021): 1–36. http://dx.doi.org/10.1145/3453158.
Full textAbstract:
In recent years, machine learning algorithms, and more specifically deep learning algorithms, have been widely used in many fields, including cyber security. However, machine learning systems are vulnerable to adversarial attacks, and this limits the application of machine learning, especially in non-stationary, adversarial environments, such as the cyber security domain, where actual adversaries (e.g., malware developers) exist. This article comprehensively summarizes the latest research on adversarial attacks against security solutions based on machine learning techniques and illuminates the risks they pose. First, the adversarial attack methods are characterized based on their stage of occurrence, and the attacker’ s goals and capabilities. Then, we categorize the applications of adversarial attack and defense methods in the cyber security domain. Finally, we highlight some characteristics identified in recent research and discuss the impact of recent advancements in other adversarial learning domains on future research directions in the cyber security domain. To the best of our knowledge, this work is the first to discuss the unique challenges of implementing end-to-end adversarial attacks in the cyber security domain, map them in a unified taxonomy, and use the taxonomy to highlight future research directions.
3
Sutanto, Richard Evan, and Sukho Lee. "Real-Time Adversarial Attack Detection with Deep Image Prior Initialized as a High-Level Representation Based Blurring Network." Electronics 10, no. 1 (2020): 52. http://dx.doi.org/10.3390/electronics10010052.
Full textAbstract:
Several recent studies have shown that artificial intelligence (AI) systems can malfunction due to intentionally manipulated data coming through normal channels. Such kinds of manipulated data are called adversarial examples. Adversarial examples can pose a major threat to an AI-led society when an attacker uses them as means to attack an AI system, which is called an adversarial attack. Therefore, major IT companies such as Google are now studying ways to build AI systems which are robust against adversarial attacks by developing effective defense methods. However, one of the reasons why it is difficult to establish an effective defense system is due to the fact that it is difficult to know in advance what kind of adversarial attack method the opponent is using. Therefore, in this paper, we propose a method to detect the adversarial noise without knowledge of the kind of adversarial noise used by the attacker. For this end, we propose a blurring network that is trained only with normal images and also use it as an initial condition of the Deep Image Prior (DIP) network. This is in contrast to other neural network based detection methods, which require the use of many adversarial noisy images for the training of the neural network. Experimental results indicate the validity of the proposed method.
4
Yang, Puyudi, Jianbo Chen, Cho-Jui Hsieh, Jane-Ling Wang, and Michael Jordan. "ML-LOO: Detecting Adversarial Examples with Feature Attribution." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (2020): 6639–47. http://dx.doi.org/10.1609/aaai.v34i04.6140.
Full textAbstract:
Deep neural networks obtain state-of-the-art performance on a series of tasks. However, they are easily fooled by adding a small adversarial perturbation to the input. The perturbation is often imperceptible to humans on image data. We observe a significant difference in feature attributions between adversarially crafted examples and original examples. Based on this observation, we introduce a new framework to detect adversarial examples through thresholding a scale estimate of feature attribution scores. Furthermore, we extend our method to include multi-layer feature attributions in order to tackle attacks that have mixed confidence levels. As demonstrated in extensive experiments, our method achieves superior performances in distinguishing adversarial examples from popular attack methods on a variety of real data sets compared to state-of-the-art detection methods. In particular, our method is able to detect adversarial examples of mixed confidence levels, and transfer between different attacking methods. We also show that our method achieves competitive performance even when the attacker has complete access to the detector.
5
Chen, Yiding, and Xiaojin Zhu. "Optimal Attack against Autoregressive Models by Manipulating the Environment." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (2020): 3545–52. http://dx.doi.org/10.1609/aaai.v34i04.5760.
Full textAbstract:
We describe an optimal adversarial attack formulation against autoregressive time series forecast using Linear Quadratic Regulator (LQR). In this threat model, the environment evolves according to a dynamical system; an autoregressive model observes the current environment state and predicts its future values; an attacker has the ability to modify the environment state in order to manipulate future autoregressive forecasts. The attacker's goal is to force autoregressive forecasts into tracking a target trajectory while minimizing its attack expenditure. In the white-box setting where the attacker knows the environment and forecast models, we present the optimal attack using LQR for linear models, and Model Predictive Control (MPC) for nonlinear models. In the black-box setting, we combine system identification and MPC. Experiments demonstrate the effectiveness of our attacks.
6
Zhang, Chaoning, Philipp Benz, Tooba Imtiaz, and In-So Kweon. "CD-UAP: Class Discriminative Universal Adversarial Perturbation." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (2020): 6754–61. http://dx.doi.org/10.1609/aaai.v34i04.6154.
Full textAbstract:
A single universal adversarial perturbation (UAP) can be added to all natural images to change most of their predicted class labels. It is of high practical relevance for an attacker to have flexible control over the targeted classes to be attacked, however, the existing UAP method attacks samples from all classes. In this work, we propose a new universal attack method to generate a single perturbation that fools a target network to misclassify only a chosen group of classes, while having limited influence on the remaining classes. Since the proposed attack generates a universal adversarial perturbation that is discriminative to targeted and non-targeted classes, we term it class discriminative universal adversarial perturbation (CD-UAP). We propose one simple yet effective algorithm framework, under which we design and compare various loss function configurations tailored for the class discriminative universal attack. The proposed approach has been evaluated with extensive experiments on various benchmark datasets. Additionally, our proposed approach achieves state-of-the-art performance for the original task of UAP attacking all classes, which demonstrates the effectiveness of our approach.
7
Butts, Jonathan, Mason Rice, and Sujeet Shenoi. "An Adversarial Model for Expressing Attacks on Control Protocols." Journal of Defense Modeling and Simulation: Applications, Methodology, Technology 9, no. 3 (2012): 243–55. http://dx.doi.org/10.1177/1548512911449409.
Full textAbstract:
In this paper we present a model for expressing attacks on control protocols that involve the exchange of messages. Attacks are modeled using the notion of an attacker who can block and/or fabricate messages. These two attack mechanisms cover a variety of scenarios ranging from power grid failures to cyber attacks on oil pipelines. The model provides a method to syntactically express communication systems and attacks, which supports the development of attack and defense strategies. For demonstration purposes, an attack instance is modeled that shows how a targeted messaging attack can result in the rupture of a gas pipeline.
8
Saha, Aniruddha, Akshayvarun Subramanya, and Hamed Pirsiavash. "Hidden Trigger Backdoor Attacks." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 07 (2020): 11957–65. http://dx.doi.org/10.1609/aaai.v34i07.6871.
Full textAbstract:
With the success of deep learning algorithms in various domains, studying adversarial attacks to secure deep models in real world applications has become an important research topic. Backdoor attacks are a form of adversarial attacks on deep networks where the attacker provides poisoned data to the victim to train the model with, and then activates the attack by showing a specific small trigger pattern at the test time. Most state-of-the-art backdoor attacks either provide mislabeled poisoning data that is possible to identify by visual inspection, reveal the trigger in the poisoned data, or use noise to hide the trigger. We propose a novel form of backdoor attack where poisoned data look natural with correct labels and also more importantly, the attacker hides the trigger in the poisoned data and keeps the trigger secret until the test time. We perform an extensive study on various image classification settings and show that our attack can fool the model by pasting the trigger at random locations on unseen images although the model performs well on clean data. We also show that our proposed attack cannot be easily defended using a state-of-the-art defense algorithm for backdoor attacks.
9
Chhabra, Anshuman, Abhishek Roy, and Prasant Mohapatra. "Suspicion-Free Adversarial Attacks on Clustering Algorithms." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (2020): 3625–32. http://dx.doi.org/10.1609/aaai.v34i04.5770.
Full textAbstract:
Clustering algorithms are used in a large number of applications and play an important role in modern machine learning– yet, adversarial attacks on clustering algorithms seem to be broadly overlooked unlike supervised learning. In this paper, we seek to bridge this gap by proposing a black-box adversarial attack for clustering models for linearly separable clusters. Our attack works by perturbing a single sample close to the decision boundary, which leads to the misclustering of multiple unperturbed samples, named spill-over adversarial samples. We theoretically show the existence of such adversarial samples for the K-Means clustering. Our attack is especially strong as (1) we ensure the perturbed sample is not an outlier, hence not detectable, and (2) the exact metric used for clustering is not known to the attacker. We theoretically justify that the attack can indeed be successful without the knowledge of the true metric. We conclude by providing empirical results on a number of datasets, and clustering algorithms. To the best of our knowledge, this is the first work that generates spill-over adversarial samples without the knowledge of the true metric ensuring that the perturbed sample is not an outlier, and theoretically proves the above.
10
Dankwa, Stephen, and Lu Yang. "Securing IoT Devices: A Robust and Efficient Deep Learning with a Mixed Batch Adversarial Generation Process for CAPTCHA Security Verification." Electronics 10, no. 15 (2021): 1798. http://dx.doi.org/10.3390/electronics10151798.
Full textAbstract:
The Internet of Things environment (e.g., smart phones, smart televisions, and smart watches) ensures that the end user experience is easy, by connecting lives on web services via the internet. Integrating Internet of Things devices poses ethical risks related to data security, privacy, reliability and management, data mining, and knowledge exchange. An adversarial machine learning attack is a good practice to adopt, to strengthen the security of text-based CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart), to withstand against malicious attacks from computer hackers, to protect Internet of Things devices and the end user’s privacy. The goal of this current study is to perform security vulnerability verification on adversarial text-based CAPTCHA, based on attacker–defender scenarios. Therefore, this study proposed computation-efficient deep learning with a mixed batch adversarial generation process model, which attempted to break the transferability attack, and mitigate the problem of catastrophic forgetting in the context of adversarial attack defense. After performing K-fold cross-validation, experimental results showed that the proposed defense model achieved mean accuracies in the range of 82–84% among three gradient-based adversarial attack datasets.
11
Yang, Runze, and Teng Long. "Derivative-free optimization adversarial attacks for graph convolutional networks." PeerJ Computer Science 7 (August 24, 2021): e693. http://dx.doi.org/10.7717/peerj-cs.693.
Full textAbstract:
In recent years, graph convolutional networks (GCNs) have emerged rapidly due to their excellent performance in graph data processing. However, recent researches show that GCNs are vulnerable to adversarial attacks. An attacker can maliciously modify edges or nodes of the graph to mislead the model’s classification of the target nodes, or even cause a degradation of the model’s overall classification performance. In this paper, we first propose a black-box adversarial attack framework based on derivative-free optimization (DFO) to generate graph adversarial examples without using gradient and apply advanced DFO algorithms conveniently. Second, we implement a direct attack algorithm (DFDA) using the Nevergrad library based on the framework. Additionally, we overcome the problem of large search space by redesigning the perturbation vector using constraint size. Finally, we conducted a series of experiments on different datasets and parameters. The results show that DFDA outperforms Nettack in most cases, and it can achieve an average attack success rate of more than 95% on the Cora dataset when perturbing at most eight edges. This demonstrates that our framework can fully exploit the potential of DFO methods in node classification adversarial attacks.
12
Xu, Guangquan, Guofeng Feng, Litao Jiao, Meiqi Feng, Xi Zheng, and Jian Liu. "FNet: A Two-Stream Model for Detecting Adversarial Attacks against 5G-Based Deep Learning Services." Security and Communication Networks 2021 (September 6, 2021): 1–10. http://dx.doi.org/10.1155/2021/5395705.
Full textAbstract:
With the extensive application of artificial intelligence technology in 5G and Beyond Fifth Generation (B5G) networks, it has become a common trend for artificial intelligence to integrate into modern communication networks. Deep learning is a subset of machine learning and has recently led to significant improvements in many fields. In particular, many 5G-based services use deep learning technology to provide better services. Although deep learning is powerful, it is still vulnerable when faced with 5G-based deep learning services. Because of the nonlinearity of deep learning algorithms, slight perturbation input by the attacker will result in big changes in the output. Although many researchers have proposed methods against adversarial attacks, these methods are not always effective against powerful attacks such as CW. In this paper, we propose a new two-stream network which includes RGB stream and spatial rich model (SRM) noise stream to discover the difference between adversarial examples and clean examples. The RGB stream uses raw data to capture subtle differences in adversarial samples. The SRM noise stream uses the SRM filters to get noise features. We regard the noise features as additional evidence for adversarial detection. Then, we adopt bilinear pooling to fuse the RGB features and the SRM features. Finally, the final features are input into the decision network to decide whether the image is adversarial or not. Experimental results show that our proposed method can accurately detect adversarial examples. Even with powerful attacks, we can still achieve a detection rate of 91.3%. Moreover, our method has good transferability to generalize to other adversaries.
13
Du, Xiaohu, Jie Yu, Zibo Yi, et al. "A Hybrid Adversarial Attack for Different Application Scenarios." Applied Sciences 10, no. 10 (2020): 3559. http://dx.doi.org/10.3390/app10103559.
Full textAbstract:
Adversarial attack against natural language has been a hot topic in the field of artificial intelligence security in recent years. It is mainly to study the methods and implementation of generating adversarial examples. The purpose is to better deal with the vulnerability and security of deep learning systems. According to whether the attacker understands the deep learning model structure, the adversarial attack is divided into black-box attack and white-box attack. In this paper, we propose a hybrid adversarial attack for different application scenarios. Firstly, we propose a novel black-box attack method of generating adversarial examples to trick the word-level sentiment classifier, which is based on differential evolution (DE) algorithm to generate semantically and syntactically similar adversarial examples. Compared with existing genetic algorithm based adversarial attacks, our algorithm can achieve a higher attack success rate while maintaining a lower word replacement rate. At the 10% word substitution threshold, we have increased the attack success rate from 58.5% to 63%. Secondly, when we understand the model architecture and parameters, etc., we propose a white-box attack with gradient-based perturbation against the same sentiment classifier. In this attack, we use a Euclidean distance and cosine distance combined metric to find the most semantically and syntactically similar substitution, and we introduce the coefficient of variation (CV) factor to control the dispersion of the modified words in the adversarial examples. More dispersed modifications can increase human imperceptibility and text readability. Compared with the existing global attack, our attack can increase the attack success rate and make modification positions in generated examples more dispersed. We’ve increased the global search success rate from 75.8% to 85.8%. Finally, we can deal with different application scenarios by using these two attack methods, that is, whether we understand the internal structure and parameters of the model, we can all generate good adversarial examples.
14
Chang, Heng, Yu Rong, Tingyang Xu, et al. "A Restricted Black-Box Adversarial Framework Towards Attacking Graph Embedding Models." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (2020): 3389–96. http://dx.doi.org/10.1609/aaai.v34i04.5741.
Full textAbstract:
With the great success of graph embedding model on both academic and industry area, the robustness of graph embedding against adversarial attack inevitably becomes a central problem in graph learning domain. Regardless of the fruitful progress, most of the current works perform the attack in a white-box fashion: they need to access the model predictions and labels to construct their adversarial loss. However, the inaccessibility of model predictions in real systems makes the white-box attack impractical to real graph learning system. This paper promotes current frameworks in a more general and flexible sense – we demand to attack various kinds of graph embedding model with black-box driven. To this end, we begin by investigating the theoretical connections between graph signal processing and graph embedding models in a principled way and formulate the graph embedding model as a general graph signal process with corresponding graph filter. As such, a generalized adversarial attacker: GF-Attack is constructed by the graph filter and feature matrix. Instead of accessing any knowledge of the target classifiers used in graph embedding, GF-Attack performs the attack only on the graph filter in a black-box attack fashion. To validate the generalization of GF-Attack, we construct the attacker on four popular graph embedding models. Extensive experimental results validate the effectiveness of our attacker on several benchmark datasets. Particularly by using our attack, even small graph perturbations like one-edge flip is able to consistently make a strong attack in performance to different graph embedding models.
15
Papadopoulos, Pavlos, Oliver Thornewill von Essen, Nikolaos Pitropakis, Christos Chrysoulas, Alexios Mylonas, and William J. Buchanan. "Launching Adversarial Attacks against Network Intrusion Detection Systems for IoT." Journal of Cybersecurity and Privacy 1, no. 2 (2021): 252–73. http://dx.doi.org/10.3390/jcp1020014.
Full textAbstract:
As the internet continues to be populated with new devices and emerging technologies, the attack surface grows exponentially. Technology is shifting towards a profit-driven Internet of Things market where security is an afterthought. Traditional defending approaches are no longer sufficient to detect both known and unknown attacks to high accuracy. Machine learning intrusion detection systems have proven their success in identifying unknown attacks with high precision. Nevertheless, machine learning models are also vulnerable to attacks. Adversarial examples can be used to evaluate the robustness of a designed model before it is deployed. Further, using adversarial examples is critical to creating a robust model designed for an adversarial environment. Our work evaluates both traditional machine learning and deep learning models’ robustness using the Bot-IoT dataset. Our methodology included two main approaches. First, label poisoning, used to cause incorrect classification by the model. Second, the fast gradient sign method, used to evade detection measures. The experiments demonstrated that an attacker could manipulate or circumvent detection with significant probability.
16
Tu, Chun-Chen, Paishun Ting, Pin-Yu Chen, et al. "AutoZOOM: Autoencoder-Based Zeroth Order Optimization Method for Attacking Black-Box Neural Networks." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 742–49. http://dx.doi.org/10.1609/aaai.v33i01.3301742.
Full textAbstract:
Recent studies have shown that adversarial examples in state-of-the-art image classifiers trained by deep neural networks (DNN) can be easily generated when the target model is transparent to an attacker, known as the white-box setting. However, when attacking a deployed machine learning service, one can only acquire the input-output correspondences of the target model; this is the so-called black-box attack setting. The major drawback of existing black-box attacks is the need for excessive model queries, which may give a false sense of model robustness due to inefficient query designs. To bridge this gap, we propose a generic framework for query-efficient blackbox attacks. Our framework, AutoZOOM, which is short for Autoencoder-based Zeroth Order Optimization Method, has two novel building blocks towards efficient black-box attacks: (i) an adaptive random gradient estimation strategy to balance query counts and distortion, and (ii) an autoencoder that is either trained offline with unlabeled data or a bilinear resizing operation for attack acceleration. Experimental results suggest that, by applying AutoZOOM to a state-of-the-art black-box attack (ZOO), a significant reduction in model queries can be achieved without sacrificing the attack success rate and the visual quality of the resulting adversarial examples. In particular, when compared to the standard ZOO method, AutoZOOM can consistently reduce the mean query counts in finding successful adversarial examples (or reaching the same distortion level) by at least 93% on MNIST, CIFAR-10 and ImageNet datasets, leading to novel insights on adversarial robustness.
17
Miller, David, Yujia Wang, and George Kesidis. "When Not to Classify: Anomaly Detection of Attacks (ADA) on DNN Classifiers at Test Time." Neural Computation 31, no. 8 (2019): 1624–70. http://dx.doi.org/10.1162/neco_a_01209.
Full textAbstract:
A significant threat to the recent, wide deployment of machine learning–based systems, including deep neural networks (DNNs), is adversarial learning attacks. The main focus here is on evasion attacks against DNN-based classifiers at test time. While much work has focused on devising attacks that make small perturbations to a test pattern (e.g., an image) that induce a change in the classifier's decision, until recently there has been a relative paucity of work defending against such attacks. Some works robustify the classifier to make correct decisions on perturbed patterns. This is an important objective for some applications and for natural adversary scenarios. However, we analyze the possible digital evasion attack mechanisms and show that in some important cases, when the pattern (image) has been attacked, correctly classifying it has no utility---when the image to be attacked is (even arbitrarily) selected from the attacker's cache and when the sole recipient of the classifier's decision is the attacker. Moreover, in some application domains and scenarios, it is highly actionable to detect the attack irrespective of correctly classifying in the face of it (with classification still performed if no attack is detected). We hypothesize that adversarial perturbations are machine detectable even if they are small. We propose a purely unsupervised anomaly detector (AD) that, unlike previous works, (1) models the joint density of a deep layer using highly suitable null hypothesis density models (matched in particular to the nonnegative support for rectified linear unit (ReLU) layers); (2) exploits multiple DNN layers; and (3) leverages a source and destination class concept, source class uncertainty, the class confusion matrix, and DNN weight information in constructing a novel decision statistic grounded in the Kullback-Leibler divergence. Tested on MNIST and CIFAR image databases under three prominent attack strategies, our approach outperforms previous detection methods, achieving strong receiver operating characteristic area under the curve detection accuracy on two attacks and better accuracy than recently reported for a variety of methods on the strongest (CW) attack. We also evaluate a fully white box attack on our system and demonstrate that our method can be leveraged to strong effect in detecting reverse engineering attacks. Finally, we evaluate other important performance measures such as classification accuracy versus true detection rate and multiple measures versus attack strength.
18
Lee, Sun Woo, Sok Joon Lee, and Dong Hoon Lee. "Attack on Vehicular Platooning and Mitigation Strategy: A Survey." Applied Mechanics and Materials 865 (June 2017): 423–28. http://dx.doi.org/10.4028/www.scientific.net/amm.865.423.
Full textAbstract:
Through vehicular platooning, a group of autonomous vehicles move together under the same control law with maintaining constant inter-vehicle distance and velocity. Owing to many advantages in the aspect of economy, environment, and safety, platoon has been developed for AHS (Automated Highway System). But there is little study of platoon in adversarial environment. Since vehicle safety is directly related to a passenger’s life, the in-depth study of adversarial platooning is of crucial importance. In this paper, we present that an attacker in platoon can cause serious accident just by slightly modifying control law and also discuss the control system designed for mitigating the damage of accident caused by the attacker.
19
Yang, Gaoming, Mingwei Li, Xianjing Fang, Ji Zhang, and Xingzhu Liang. "Generating adversarial examples without specifying a target model." PeerJ Computer Science 7 (September 13, 2021): e702. http://dx.doi.org/10.7717/peerj-cs.702.
Full textAbstract:
Adversarial examples are regarded as a security threat to deep learning models, and there are many ways to generate them. However, most existing methods require the query authority of the target during their work. In a more practical situation, the attacker will be easily detected because of too many queries, and this problem is especially obvious under the black-box setting. To solve the problem, we propose the Attack Without a Target Model (AWTM). Our algorithm does not specify any target model in generating adversarial examples, so it does not need to query the target. Experimental results show that it achieved a maximum attack success rate of 81.78% in the MNIST data set and 87.99% in the CIFAR-10 data set. In addition, it has a low time cost because it is a GAN-based method.
20
Shirazi, Hossein, Bruhadeshwar Bezawada, Indrakshi Ray, and Chuck Anderson. "Directed adversarial sampling attacks on phishing detection." Journal of Computer Security 29, no. 1 (2021): 1–23. http://dx.doi.org/10.3233/jcs-191411.
Full textAbstract:
Phishing websites trick honest users into believing that they interact with a legitimate website and capture sensitive information, such as user names, passwords, credit card numbers, and other personal information. Machine learning is a promising technique to distinguish between phishing and legitimate websites. However, machine learning approaches are susceptible to adversarial learning attacks where a phishing sample can bypass classifiers. Our experiments on publicly available datasets reveal that the phishing detection mechanisms are vulnerable to adversarial learning attacks. We investigate the robustness of machine learning-based phishing detection in the face of adversarial learning attacks. We propose a practical approach to simulate such attacks by generating adversarial samples through direct feature manipulation. To enhance the sample’s success probability, we describe a clustering approach that guides an attacker to select the best possible phishing samples that can bypass the classifier by appearing as legitimate samples. We define the notion of vulnerability level for each dataset that measures the number of features that can be manipulated and the cost for such manipulation. Further, we clustered phishing samples and showed that some clusters of samples are more likely to exhibit higher vulnerability levels than others. This helps an adversary identify the best candidates of phishing samples to generate adversarial samples at a lower cost. Our finding can be used to refine the dataset and develop better learning models to compensate for the weak samples in the training dataset.
21
Tondi, Benedetta, Neri Merhav, and Mauro Barni. "Detection Games under Fully Active Adversaries." Entropy 21, no. 1 (2018): 23. http://dx.doi.org/10.3390/e21010023.
Full textAbstract:
We study a binary hypothesis testing problem in which a defender must decide whether a test sequence has been drawn from a given memoryless source P 0 , while an attacker strives to impede the correct detection. With respect to previous works, the adversarial setup addressed in this paper considers an attacker who is active under both hypotheses, namely, a fully active attacker, as opposed to a partially active attacker who is active under one hypothesis only. In the fully active setup, the attacker distorts sequences drawn both from P 0 and from an alternative memoryless source P 1 , up to a certain distortion level, which is possibly different under the two hypotheses, to maximize the confusion in distinguishing between the two sources, i.e., to induce both false positive and false negative errors at the detector, also referred to as the defender. We model the defender–attacker interaction as a game and study two versions of this game, the Neyman–Pearson game and the Bayesian game. Our main result is in the characterization of an attack strategy that is asymptotically both dominant (i.e., optimal no matter what the defender’s strategy is) and universal, i.e., independent of P 0 and P 1 . From the analysis of the equilibrium payoff, we also derive the best achievable performance of the defender, by relaxing the requirement on the exponential decay rate of the false positive error probability in the Neyman–Pearson setup and the tradeoff between the error exponents in the Bayesian setup. Such analysis permits characterizing the conditions for the distinguishability of the two sources given the distortion levels.
22
Chen, Lili, Zhen Wang, Fenghua Li, Yunchuan Guo, and Kui Geng. "A Stackelberg Security Game for Adversarial Outbreak Detection in the Internet of Things." Sensors 20, no. 3 (2020): 804. http://dx.doi.org/10.3390/s20030804.
Full textAbstract:
With limited computing resources and a lack of physical lines of defense, the Internet of Things (IoT) has become a focus of cyberattacks. In recent years, outbreak propagation attacks against the IoT have occurred frequently, and these attacks are often strategical. In order to detect the outbreak propagation as soon as possible, t embedded Intrusion Detection Systems (IDSs) are widely deployed in the IoT. This paper tackles the problem of outbreak detection in adversarial environment in the IoT. A dynamic scheduling strategy based on specific IDSs monitoring of IoT devices is proposed to avoid strategic attacks. Firstly, we formulate the interaction between the defender and attacker as a Stackelberg game in which the defender first chooses a set of device nodes to activate, and then the attacker selects one seed (one device node) to spread the worms. This yields an extremely complex bilevel optimization problem. Our approach is to build a modified Column Generation framework for computing the optimal strategy effectively. The optimal response of the defender’s problem is expressed as mixed-integer linear programming (MILPs). It is proved that the solution of the defender’s optimal response is a NP-hard problem. Moreover, the optimal response of defenders is improved by an approximate algorithm--a greedy algorithm. Finally, the proposed scheme is tested on some randomly generated instances. The experimental results show that the scheme is effective for monitoring optimal scheduling.
23
Zhao, Jinxiong, Xun Zhang, Fuqiang Di, et al. "Exploring the Optimum Proactive Defense Strategy for the Power Systems from an Attack Perspective." Security and Communication Networks 2021 (February 12, 2021): 1–14. http://dx.doi.org/10.1155/2021/6699108.
Full textAbstract:
Proactive defense is one of the most promising approaches to enhance cyber-security in the power systems, while how to balance its costs and benefits has not been fully studied. This paper proposes a novel method to model cyber adversarial behaviors as attackers contending for the defenders’ benefit based on the game theory. We firstly calculate the final benefit of the hackers and defenders in different states on the basis of the constructed models and then predict the possible attack behavior and evaluate the best defense strategy for the power systems. Based on a real power system subnet, we analyze 27 attack models with our method, and the result shows that the optimal strategy of the attacker is to launch a small-scale attack. Correspondingly, the optimal strategy of the defender is to conduct partial-defense.
24
Pal, Soham, Yash Gupta, Aditya Shukla, Aditya Kanade, Shirish Shevade, and Vinod Ganapathy. "ActiveThief: Model Extraction Using Active Learning and Unannotated Public Data." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 01 (2020): 865–72. http://dx.doi.org/10.1609/aaai.v34i01.5432.
Full textAbstract:
Machine learning models are increasingly being deployed in practice. Machine Learning as a Service (MLaaS) providers expose such models to queries by third-party developers through application programming interfaces (APIs). Prior work has developed model extraction attacks, in which an attacker extracts an approximation of an MLaaS model by making black-box queries to it. We design ActiveThief – a model extraction framework for deep neural networks that makes use of active learning techniques and unannotated public datasets to perform model extraction. It does not expect strong domain knowledge or access to annotated data on the part of the attacker. We demonstrate that (1) it is possible to use ActiveThief to extract deep classifiers trained on a variety of datasets from image and text domains, while querying the model with as few as 10-30% of samples from public datasets, (2) the resulting model exhibits a higher transferability success rate of adversarial examples than prior work, and (3) the attack evades detection by the state-of-the-art model extraction detection method, PRADA.
25
Tong, Liang, Aron Laszka, Chao Yan, Ning Zhang, and Yevgeniy Vorobeychik. "Finding Needles in a Moving Haystack: Prioritizing Alerts with Adversarial Reinforcement Learning." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 01 (2020): 946–53. http://dx.doi.org/10.1609/aaai.v34i01.5442.
Full textAbstract:
Detection of malicious behavior is a fundamental problem in security. One of the major challenges in using detection systems in practice is in dealing with an overwhelming number of alerts that are triggered by normal behavior (the so-called false positives), obscuring alerts resulting from actual malicious activities. We introduce a novel approach for computing a policy for prioritizing alerts using adversarial reinforcement learning. Our approach assumes that the attacker knows the full state of the detection system and the defender's alert prioritization policy, and will dynamically choose an optimal attack. The first step of our approach is to capture the interaction between the defender and attacker in a game theoretic model. To tackle the computational complexity of solving this game to obtain a dynamic stochastic alert prioritization policy, we propose an adversarial reinforcement learning framework. In this framework, we use neural reinforcement learning to compute best response policies for both the defender and the adversary to an arbitrary stochastic policy of the other. We then use these in a double-oracle framework to obtain an approximate equilibrium of the game, which in turn yields a robust stochastic policy for the defender. We use case studies in network intrusion and fraud detection to demonstrate that our approach is effective in creating robust alert prioritization policies.1
26
Niu, L., Y. Song, J. Chu, and S. Li. "ANALYSIS OF THE ATTACKER AND DEFENDER GAN MODELS FOR THE INDOOR NAVIGATION NETWORK." International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLIII-B4-2021 (June 30, 2021): 237–42. http://dx.doi.org/10.5194/isprs-archives-xliii-b4-2021-237-2021.
Full textAbstract:
Abstract. Evacuation research relies heavily on the efficiency analysis of the study navigation networks, and this principle also applies to indoor scenarios. One crucial type of these scenarios is the attacker and defender topic, which discusses the paralyzing and recovering operations for a specific indoor navigation network. Our approach is to apply the Generative-Adversarial-Neural network (GAN) model to optimize both reduction and increase operations for a specific indoor navigation network. In other words, the proposed model utilizes GAN both in the attacking behavior efficiency analysis and the recovering behavior efficiency analysis. To this purpose, we design a black box of training the generative model and adversarial model to construct the hidden neural networks to mimic the human selection of choosing the critical nodes in the studying navigation networks. The experiment shows that the proposed model could alleviate the selection of nodes that significantly influence network transportation efficiency. Therefore, we could apply this model to disaster responding scenarios like fire evacuation and communication network recovery operations.
27
Wachter, Jasmin, Stefan Rass, and Sandra König. "Security from the Adversary’s Inertia–Controlling Convergence Speed When Playing Mixed Strategy Equilibria." Games 9, no. 3 (2018): 59. http://dx.doi.org/10.3390/g9030059.
Full textAbstract:
Game-theoretic models are a convenient tool to systematically analyze competitive situations. This makes them particularly handy in the field of security where a company or a critical infrastructure wants to defend against an attacker. When the optimal solution of the security game involves several pure strategies (i.e., the equilibrium is mixed), this may induce additional costs. Minimizing these costs can be done simultaneously with the original goal of minimizing the damage due to the attack. Existing models assume that the attacker instantly knows the action chosen by the defender (i.e., the pure strategy he is playing in the i-th round) but in real situations this may take some time. Such adversarial inertia can be exploited to gain security and save cost. To this end, we introduce the concept of information delay, which is defined as the time it takes an attacker to mount an attack. In this period it is assumed that the adversary has no information about the present state of the system, but only knows the last state before commencing the attack. Based on a Markov chain model we construct strategy policies that are cheaper in terms of maintenance (switching costs) when compared to classical approaches. The proposed approach yields slightly larger security risk but overall ensures a better performance. Furthermore, by reinvesting the saved costs in additional security measures it is possible to obtain even more security at the same overall cost.
28
Yang, Wenjie, Jian Weng, Weiqi Luo, and Anjia Yang. "Strongly Unforgeable Certificateless Signature Resisting Attacks from Malicious-But-Passive KGC." Security and Communication Networks 2017 (2017): 1–8. http://dx.doi.org/10.1155/2017/5704865.
Full textAbstract:
In digital signature, strong unforgeability requires that an attacker cannot forge a new signature on any previously signed/new messages, which is attractive in both theory and practice. Recently, a strongly unforgeable certificateless signature (CLS) scheme without random oracles was presented. In this paper, we firstly show that the scheme fails to achieve strong unforgeability by forging a new signature on a previously signed message under its adversarial model. Then, we point out that the scheme is also vulnerable to the malicious-but-passive key generation center (MKGC) attacks. Finally, we propose an improved strongly unforgeable CLS scheme in the standard model. The improved scheme not only meets the requirement of strong unforgeability but also withstands the MKGC attacks. To the best of our knowledge, we are the first to prove a CLS scheme to be strongly unforgeable against the MKGC attacks without using random oracles.
29
Roponen, Juho, and Ahti Salo. "Adversarial Risk Analysis for Enhancing Combat Simulation Models." Journal of Military Studies 6, no. 2 (2015): 82–103. http://dx.doi.org/10.1515/jms-2016-0200.
Full textAbstract:
Abstract Adversarial Risk Analysis (ARA) builds on statistical risk analysis and game theory to analyze decision situations involving two or more intelligent opponents who make decisions under uncertainty. During the past few years, the ARA approach-which is based on the explicit modelling of the decision making processes of a rational opponent-has been applied extensively in areas such as counterterrorism and corporate competition. In the context of military combat modelling, however, ARA has not been used systematically, even if there have been attempts to predict the opponent’s decisions based on wargaming, application of game theoretic equilibria, and the use of expert judgements. Against this backdrop, we argue that combining ARA with military combat modelling holds promise for enhancing the capabilities of combat modelling tools. We identify ways of combining ARA with combat modelling and give an illustrative example of how ARA can provide insights into a problem where the defender needs to estimate the utility gained from hiding its troop movements from the attacker. Even if the ARA approach can be challenging to apply, it can be instructive in that relevant assumptions about the resources, expectations and goals that guide the adversary’s decisions must be explicated.
30
Liu, Xu, Xiaoqiang Di, Jinqing Li, et al. "Allocating Limited Resources to Protect a Massive Number of Targets Using a Game Theoretic Model." Mathematical Problems in Engineering 2019 (March 13, 2019): 1–16. http://dx.doi.org/10.1155/2019/5475341.
Full textAbstract:
Resource allocation is the process of optimizing the rare resources. In the area of security, how to allocate limited resources to protect a massive number of targets is especially challenging. This paper addresses this resource allocation issue by constructing a game theoretic model. A defender and an attacker are players and the interaction is formulated as a trade-off between protecting targets and consuming resources. The action cost which is a necessary role of consuming resource is considered in the proposed model. Additionally, a bounded rational behavior model (quantal response: QR), which simulates a human attacker of the adversarial nature, is introduced to improve the proposed model. To validate the proposed model, we compare the different utility functions and resource allocation strategies. The comparison results suggest that the proposed resource allocation strategy performs better than others in the perspective of utility and resource effectiveness.
31
Meng, Sascha, Marcus Wiens, and Frank Schultmann. "Adversarial risks in the lab – An experimental study of framing-effects in attacker-defender games." Safety Science 120 (December 2019): 551–60. http://dx.doi.org/10.1016/j.ssci.2019.08.004.
Full text
32
Kalbantner, Jan, Konstantinos Markantonakis, Darren Hurley-Smith, Raja Naeem Akram, and Benjamin Semal. "P2PEdge: A Decentralised, Scalable P2P Architecture for Energy Trading in Real-Time." Energies 14, no. 3 (2021): 606. http://dx.doi.org/10.3390/en14030606.
Full textAbstract:
Current Peer-to-Peer (P2P) energy market models raise serious concerns regarding the confidentiality and integrity of energy consumption, trading and billing data. While Distributed Ledger Technology (DLT) systems (e.g., blockchain) have been proposed to enhance security, an attacker could damage other parts of the model, such as its infrastructure: an adversarial attacker could target the communication between entities by, e.g., eavesdropping or modifying data. The main goal of this paper is to propose a model for a decentralised P2P marketplace for trading energy, which addresses the problem of developing security and privacy-aware environments. Additionally, a Multi-Agent System (MAS) architecture is presented with a focus on security and sustainability. In order to propose a solution to DLT’s scalability issues (i.e., through transaction confirmation delays), off-chain state channels are considered for the energy negotiation and resolution processes. Additionally, a STRIDE (spoofing, tampering, repudiation, information disclosure, denial of service, elevation of privilege) security analysis is conducted within the context of the proposed model to identify potential vulnerabilities.
33
Gao, Xianfeng, Yu-an Tan, Hongwei Jiang, Quanxin Zhang, and Xiaohui Kuang. "Boosting Targeted Black-Box Attacks via Ensemble Substitute Training and Linear Augmentation." Applied Sciences 9, no. 11 (2019): 2286. http://dx.doi.org/10.3390/app9112286.
Full textAbstract:
These years, Deep Neural Networks (DNNs) have shown unprecedented performance in many areas. However, some recent studies revealed their vulnerability to small perturbations added on source inputs. Furthermore, we call the ways to generate these perturbations’ adversarial attacks, which contain two types, black-box and white-box attacks, according to the adversaries’ access to target models. In order to overcome the problem of black-box attackers’ unreachabilities to the internals of target DNN, many researchers put forward a series of strategies. Previous works include a method of training a local substitute model for the target black-box model via Jacobian-based augmentation and then use the substitute model to craft adversarial examples using white-box methods. In this work, we improve the dataset augmentation to make the substitute models better fit the decision boundary of the target model. Unlike the previous work that just performed the non-targeted attack, we make it first to generate targeted adversarial examples via training substitute models. Moreover, to boost the targeted attacks, we apply the idea of ensemble attacks to the substitute training. Experiments on MNIST and GTSRB, two common datasets for image classification, demonstrate our effectiveness and efficiency of boosting a targeted black-box attack, and we finally attack the MNIST and GTSRB classifiers with the success rates of 97.7% and 92.8%.
34
Jaiswal, Mimansa, and Emily Mower Provost. "Privacy Enhanced Multimodal Neural Representations for Emotion Recognition." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 05 (2020): 7985–93. http://dx.doi.org/10.1609/aaai.v34i05.6307.
Full textAbstract:
Many mobile applications and virtual conversational agents now aim to recognize and adapt to emotions. To enable this, data are transmitted from users' devices and stored on central servers. Yet, these data contain sensitive information that could be used by mobile applications without user's consent or, maliciously, by an eavesdropping adversary. In this work, we show how multimodal representations trained for a primary task, here emotion recognition, can unintentionally leak demographic information, which could override a selected opt-out option by the user. We analyze how this leakage differs in representations obtained from textual, acoustic, and multimodal data. We use an adversarial learning paradigm to unlearn the private information present in a representation and investigate the effect of varying the strength of the adversarial component on the primary task and on the privacy metric, defined here as the inability of an attacker to predict specific demographic information. We evaluate this paradigm on multiple datasets and show that we can improve the privacy metric while not significantly impacting the performance on the primary task. To the best of our knowledge, this is the first work to analyze how the privacy metric differs across modalities and how multiple privacy concerns can be tackled while still maintaining performance on emotion recognition.
35
Liu, Ninghao, Mengnan Du, Ruocheng Guo, Huan Liu, and Xia Hu. "Adversarial Attacks and Defenses." ACM SIGKDD Explorations Newsletter 23, no. 1 (2021): 86–99. http://dx.doi.org/10.1145/3468507.3468519.
Full textAbstract:
Despite the recent advances in a wide spectrum of applications, machine learning models, especially deep neural networks, have been shown to be vulnerable to adversarial attacks. Attackers add carefully-crafted perturbations to input, where the perturbations are almost imperceptible to humans, but can cause models to make wrong predictions. Techniques to protect models against adversarial input are called adversarial defense methods. Although many approaches have been proposed to study adversarial attacks and defenses in different scenarios, an intriguing and crucial challenge remains that how to really understand model vulnerability? Inspired by the saying that "if you know yourself and your enemy, you need not fear the battles", we may tackle the challenge above after interpreting machine learning models to open the black-boxes. The goal of model interpretation, or interpretable machine learning, is to extract human-understandable terms for the working mechanism of models. Recently, some approaches start incorporating interpretation into the exploration of adversarial attacks and defenses. Meanwhile, we also observe that many existing methods of adversarial attacks and defenses, although not explicitly claimed, can be understood from the perspective of interpretation. In this paper, we review recent work on adversarial attacks and defenses, particularly from the perspective of machine learning interpretation. We categorize interpretation into two types, feature-level interpretation, and model-level interpretation. For each type of interpretation, we elaborate on how it could be used for adversarial attacks and defenses. We then briefly illustrate additional correlations between interpretation and adversaries. Finally, we discuss the challenges and future directions for tackling adversary issues with interpretation.
36
Zheng, Tianhang, Changyou Chen, and Kui Ren. "Distributionally Adversarial Attack." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 2253–60. http://dx.doi.org/10.1609/aaai.v33i01.33012253.
Full textAbstract:
Recent work on adversarial attack has shown that Projected Gradient Descent (PGD) Adversary is a universal first-order adversary, and the classifier adversarially trained by PGD is robust against a wide range of first-order attacks. It is worth noting that the original objective of an attack/defense model relies on a data distribution p(x), typically in the form of risk maximization/minimization, e.g., max/min Ep(x) L(x) with p(x) some unknown data distribution and L(·) a loss function. However, since PGD generates attack samples independently for each data sample based on L(·), the procedure does not necessarily lead to good generalization in terms of risk optimization. In this paper, we achieve the goal by proposing distributionally adversarial attack (DAA), a framework to solve an optimal adversarial-data distribution, a perturbed distribution that satisfies the L∞ constraint but deviates from the original data distribution to increase the generalization risk maximally. Algorithmically, DAA performs optimization on the space of potential data distributions, which introduces direct dependency between all data points when generating adversarial samples. DAA is evaluated by attacking state-of-the-art defense models, including the adversarially-trained models provided by MIT MadryLab. Notably, DAA ranks the first place on MadryLab’s white-box leaderboards, reducing the accuracy of their secret MNIST model to 88.56% (with l∞ perturbations of ε = 0.3) and the accuracy of their secret CIFAR model to 44.71% (with l∞ perturbations of ε = 8.0). Code for the experiments is released on https://github.com/tianzheng4/Distributionally-Adversarial-Attack.
37
Sagar, Ramani, Rutvij Jhaveri, and Carlos Borrego. "Applications in Security and Evasions in Machine Learning: A Survey." Electronics 9, no. 1 (2020): 97. http://dx.doi.org/10.3390/electronics9010097.
Full textAbstract:
In recent years, machine learning (ML) has become an important part to yield security and privacy in various applications. ML is used to address serious issues such as real-time attack detection, data leakage vulnerability assessments and many more. ML extensively supports the demanding requirements of the current scenario of security and privacy across a range of areas such as real-time decision-making, big data processing, reduced cycle time for learning, cost-efficiency and error-free processing. Therefore, in this paper, we review the state of the art approaches where ML is applicable more effectively to fulfill current real-world requirements in security. We examine different security applications’ perspectives where ML models play an essential role and compare, with different possible dimensions, their accuracy results. By analyzing ML algorithms in security application it provides a blueprint for an interdisciplinary research area. Even with the use of current sophisticated technology and tools, attackers can evade the ML models by committing adversarial attacks. Therefore, requirements rise to assess the vulnerability in the ML models to cope up with the adversarial attacks at the time of development. Accordingly, as a supplement to this point, we also analyze the different types of adversarial attacks on the ML models. To give proper visualization of security properties, we have represented the threat model and defense strategies against adversarial attack methods. Moreover, we illustrate the adversarial attacks based on the attackers’ knowledge about the model and addressed the point of the model at which possible attacks may be committed. Finally, we also investigate different types of properties of the adversarial attacks.
38
Imam, Niddal H., and Vassilios G. Vassilakis. "A Survey of Attacks Against Twitter Spam Detectors in an Adversarial Environment." Robotics 8, no. 3 (2019): 50. http://dx.doi.org/10.3390/robotics8030050.
Full textAbstract:
Online Social Networks (OSNs), such as Facebook and Twitter, have become a very important part of many people’s daily lives. Unfortunately, the high popularity of these platforms makes them very attractive to spammers. Machine learning (ML) techniques have been widely used as a tool to address many cybersecurity application problems (such as spam and malware detection). However, most of the proposed approaches do not consider the presence of adversaries that target the defense mechanism itself. Adversaries can launch sophisticated attacks to undermine deployed spam detectors either during training or the prediction (test) phase. Not considering these adversarial activities at the design stage makes OSNs’ spam detectors vulnerable to a range of adversarial attacks. Thus, this paper surveys the attacks against Twitter spam detectors in an adversarial environment, and a general taxonomy of potential adversarial attacks is presented using common frameworks from the literature. Examples of adversarial activities on Twitter that were discovered after observing Arabic trending hashtags are discussed in detail. A new type of spam tweet (adversarial spam tweet), which can be used to undermine a deployed classifier, is examined. In addition, possible countermeasures that could increase the robustness of Twitter spam detectors to such attacks are investigated.
39
An, Bo, Eric Shieh, Milind Tambe, et al. "PROTECT -- A Deployed Game Theoretic System for Strategic Security Allocation for the United States Coast Guard." AI Magazine 33, no. 4 (2012): 96. http://dx.doi.org/10.1609/aimag.v33i4.2401.
Full textAbstract:
While three deployed applications of game theory for security have recently been reported, we as a community of agents and AI researchers remain in the early stages of these deployments; there is a continuing need to understand the core principles for innovative security applications of game theory. Towards that end, this paper presents PROTECT, a game-theoretic system deployed by the United States Coast Guard (USCG) in the port of Boston for scheduling their patrols. USCG has termed the deployment of PROTECT in Boston a success, and efforts are underway to test it in the port of New York, with the potential for nationwide deployment.PROTECT is premised on an attacker-defender Stackelberg game model and offers five key innovations. First, this system is a departure from the assumption of perfect adversary rationality noted in previous work, relying instead on a quantal response (QR) model of the adversary's behavior --- to the best of our knowledge, this is the first real-world deployment of the QR model. Second, to improve PROTECT's efficiency, we generate a compact representation of the defender's strategy space, exploiting equivalence and dominance. Third, we show how to practically model a real maritime patrolling problem as a Stackelberg game. Fourth, our experimental results illustrate that PROTECT's QR model more robustly handles real-world uncertainties than a perfect rationality model. Finally, in evaluating PROTECT, this paper for the first time provides real-world data: (i) comparison of human-generated vs PROTECT security schedules, and (ii) results from an Adversarial Perspective Team's (human mock attackers) analysis.
40
Park, Hosung, Gwonsang Ryu, and Daeseon Choi. "Partial Retraining Substitute Model for Query-Limited Black-Box Attacks." Applied Sciences 10, no. 20 (2020): 7168. http://dx.doi.org/10.3390/app10207168.
Full textAbstract:
Black-box attacks against deep neural network (DNN) classifiers are receiving increasing attention because they represent a more practical approach in the real world than white box attacks. In black-box environments, adversaries have limited knowledge regarding the target model. This makes it difficult to estimate gradients for crafting adversarial examples, such that powerful white-box algorithms cannot be directly applied to black-box attacks. Therefore, a well-known black-box attack strategy creates local DNNs, called substitute models, to emulate the target model. The adversaries then craft adversarial examples using the substitute models instead of the unknown target model. The substitute models repeat the query process and are trained by observing labels from the target model’s responses to queries. However, emulating a target model usually requires numerous queries because new DNNs are trained from the beginning. In this study, we propose a new training method for substitute models to minimize the number of queries. We consider the number of queries as an important factor for practical black-box attacks because real-world systems often restrict queries for security and financial purposes. To decrease the number of queries, the proposed method does not emulate the entire target model and only adjusts the partial classification boundary based on a current attack. Furthermore, it does not use queries in the pre-training phase and creates queries only in the retraining phase. The experimental results indicate that the proposed method is effective in terms of the number of queries and attack success ratio against MNIST, VGGFace2, and ImageNet classifiers in query-limited black-box environments. Further, we demonstrate a black-box attack against a commercial classifier, Google AutoML Vision.
41
Jiang, Yan, Guisheng Yin, Ye Yuan, and Qingan Da. "Project Gradient Descent Adversarial Attack against Multisource Remote Sensing Image Scene Classification." Security and Communication Networks 2021 (June 12, 2021): 1–13. http://dx.doi.org/10.1155/2021/6663028.
Full textAbstract:
Deep learning technology (a deeper and optimized network structure) and remote sensing imaging (i.e., the more multisource and the more multicategory remote sensing data) have developed rapidly. Although the deep convolutional neural network (CNN) has achieved state-of-the-art performance on remote sensing image (RSI) scene classification, the existence of adversarial attacks poses a potential security threat to the RSI scene classification task based on CNN. The corresponding adversarial samples can be generated by adding a small perturbation to the original images. Feeding the CNN-based classifier with the adversarial samples leads to the classifier misclassify with high confidence. To achieve a higher attack success rate against scene classification based on CNN, we introduce the projected gradient descent method to generate adversarial remote sensing images. Then, we select several mainstream CNN-based classifiers as the attacked models to demonstrate the effectiveness of our method. The experimental results show that our proposed method can dramatically reduce the classification accuracy under untargeted and targeted attacks. Furthermore, we also evaluate the quality of the generated adversarial images by visual and quantitative comparisons. The results show that our method can generate the imperceptible adversarial samples and has a stronger attack ability for the RSI scene classification.
42
Zhang, Jing, Shifei Shen, and Rui Yang. "The impacts of adaptive attacking and defending strategies on mitigation of intentional threats." Kybernetes 39, no. 5 (2010): 825–37. http://dx.doi.org/10.1108/03684921011043279.
Full textAbstract:
PurposeThe purpose of this paper is to focus on resource allocation and information disclosure policy for defending multiple targets against intentional attacks. The intentional attacks, like terrorism events, probably cause great losses and fatalities. Attackers and defenders usually make decisions based on incomplete information. Adaptive attacking and defending strategies are considered, to study how both sides make more effective decisions according to previous fights.Design/methodology/approachA stochastic game‐theoretic approach is proposed for modeling attacker‐defender conflicts. Attackers and defenders are supposed both to be strategic decision makers and partially aware of adversary's information. Adaptive strategies are compared with different inflexible strategies in a fortification‐patrol problem, where the fortification affects the security vulnerability of targets and the patrol indicates the defensive signal.FindingsThe result shows that the intentional risk would be elevated by adaptive attack strategies. An inflexible defending strategy probably fails when facing uncertainties of adversary. It is shown that the optimal response of defenders is to adjust defending strategies by learning from previous games and assessing behaviors of adversaries to minimize the expected loss.Originality/valueThis paper explores how adaptive strategies affect attacker‐defender conflicts. The key issue is defense allocation and information disclosure policy for mitigation of intentional threats. Attackers and defenders can adjust their strategies by learning from previous fights, and the strategic adjustment of both sides may be asynchronous.
43
Hu, Yongjin, Jin Tian, and Jun Ma. "A Novel Way to Generate Adversarial Network Traffic Samples against Network Traffic Classification." Wireless Communications and Mobile Computing 2021 (August 23, 2021): 1–12. http://dx.doi.org/10.1155/2021/7367107.
Full textAbstract:
Network traffic classification technologies could be used by attackers to implement network monitoring and then launch traffic analysis attacks or website fingerprint attacks. In order to prevent such attacks, a novel way to generate adversarial samples of network traffic from the perspective of the defender is proposed. By adding perturbation to the normal network traffic, a kind of adversarial network traffic is formed, which will cause misclassification when the attackers are implementing network traffic classification with deep convolutional neural networks (CNN) as a classification model. The paper uses the concept of adversarial samples in image recognition for reference to the field of network traffic classification and chooses several different methods to generate adversarial samples of network traffic. The experiment, in which the LeNet-5 CNN is selected as a classification model used by attackers and Vgg16 CNN is selected as the model to test the transferability of the adversarial network traffic generated, shows the effect of the adversarial network traffic samples.
44
Zhao, Chenxiao, P. Thomas Fletcher, Mixue Yu, Yaxin Peng, Guixu Zhang, and Chaomin Shen. "The Adversarial Attack and Detection under the Fisher Information Metric." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 5869–76. http://dx.doi.org/10.1609/aaai.v33i01.33015869.
Full textAbstract:
Many deep learning models are vulnerable to the adversarial attack, i.e., imperceptible but intentionally-designed perturbations to the input can cause incorrect output of the networks. In this paper, using information geometry, we provide a reasonable explanation for the vulnerability of deep learning models. By considering the data space as a non-linear space with the Fisher information metric induced from a neural network, we first propose an adversarial attack algorithm termed one-step spectral attack (OSSA). The method is described by a constrained quadratic form of the Fisher information matrix, where the optimal adversarial perturbation is given by the first eigenvector, and the vulnerability is reflected by the eigenvalues. The larger an eigenvalue is, the more vulnerable the model is to be attacked by the corresponding eigenvector. Taking advantage of the property, we also propose an adversarial detection method with the eigenvalues serving as characteristics. Both our attack and detection algorithms are numerically optimized to work efficiently on large datasets. Our evaluations show superior performance compared with other methods, implying that the Fisher information is a promising approach to investigate the adversarial attacks and defenses.
45
Kim, Yongsu, Hyoeun Kang, Naufal Suryanto, Harashta Tatimma Larasati, Afifatul Mukaroh, and Howon Kim. "Extended Spatially Localized Perturbation GAN (eSLP-GAN) for Robust Adversarial Camouflage Patches." Sensors 21, no. 16 (2021): 5323. http://dx.doi.org/10.3390/s21165323.
Full textAbstract:
Deep neural networks (DNNs), especially those used in computer vision, are highly vulnerable to adversarial attacks, such as adversarial perturbations and adversarial patches. Adversarial patches, often considered more appropriate for a real-world attack, are attached to the target object or its surroundings to deceive the target system. However, most previous research employed adversarial patches that are conspicuous to human vision, making them easy to identify and counter. Previously, the spatially localized perturbation GAN (SLP-GAN) was proposed, in which the perturbation was only added to the most representative area of the input images, creating a spatially localized adversarial camouflage patch that excels in terms of visual fidelity and is, therefore, difficult to detect by human vision. In this study, the use of the method called eSLP-GAN was extended to deceive classifiers and object detection systems. Specifically, the loss function was modified for greater compatibility with an object-detection model attack and to increase robustness in the real world. Furthermore, the applicability of the proposed method was tested on the CARLA simulator for a more authentic real-world attack scenario.
46
Haq, Ijaz Ul, Zahid Younas Khan, Arshad Ahmad, et al. "Evaluating and Enhancing the Robustness of Sustainable Neural Relationship Classifiers Using Query-Efficient Black-Box Adversarial Attacks." Sustainability 13, no. 11 (2021): 5892. http://dx.doi.org/10.3390/su13115892.
Full textAbstract:
Neural relation extraction (NRE) models are the backbone of various machine learning tasks, including knowledge base enrichment, information extraction, and document summarization. Despite the vast popularity of these models, their vulnerabilities remain unknown; this is of high concern given their growing use in security-sensitive applications such as question answering and machine translation in the aspects of sustainability. In this study, we demonstrate that NRE models are inherently vulnerable to adversarially crafted text that contains imperceptible modifications of the original but can mislead the target NRE model. Specifically, we propose a novel sustainable term frequency-inverse document frequency (TFIDF) based black-box adversarial attack to evaluate the robustness of state-of-the-art CNN, CGN, LSTM, and BERT-based models on two benchmark RE datasets. Compared with white-box adversarial attacks, black-box attacks impose further constraints on the query budget; thus, efficient black-box attacks remain an open problem. By applying TFIDF to the correctly classified sentences of each class label in the test set, the proposed query-efficient method achieves a reduction of up to 70% in the number of queries to the target model for identifying important text items. Based on these items, we design both character- and word-level perturbations to generate adversarial examples. The proposed attack successfully reduces the accuracy of six representative models from an average F1 score of 80% to below 20%. The generated adversarial examples were evaluated by humans and are considered semantically similar. Moreover, we discuss defense strategies that mitigate such attacks, and the potential countermeasures that could be deployed in order to improve sustainability of the proposed scheme.
47
Che, Zhaohui, Ali Borji, Guangtao Zhai, Suiyi Ling, Jing Li, and Patrick Le Callet. "A New Ensemble Adversarial Attack Powered by Long-Term Gradient Memories." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (2020): 3405–13. http://dx.doi.org/10.1609/aaai.v34i04.5743.
Full textAbstract:
Deep neural networks are vulnerable to adversarial attacks. More importantly, some adversarial examples crafted against an ensemble of pre-trained source models can transfer to other new target models, thus pose a security threat to black-box applications (when the attackers have no access to the target models). Despite adopting diverse architectures and parameters, source and target models often share similar decision boundaries. Therefore, if an adversary is capable of fooling several source models concurrently, it can potentially capture intrinsic transferable adversarial information that may allow it to fool a broad class of other black-box target models. Current ensemble attacks, however, only consider a limited number of source models to craft an adversary, and obtain poor transferability. In this paper, we propose a novel black-box attack, dubbed Serial-Mini-Batch-Ensemble-Attack (SMBEA). SMBEA divides a large number of pre-trained source models into several mini-batches. For each single batch, we design 3 new ensemble strategies to improve the intra-batch transferability. Besides, we propose a new algorithm that recursively accumulates the “long-term” gradient memories of the previous batch to the following batch. This way, the learned adversarial information can be preserved and the inter-batch transferability can be improved. Experiments indicate that our method outperforms state-of-the-art ensemble attacks over multiple pixel-to-pixel vision tasks including image translation and salient region prediction. Our method successfully fools two online black-box saliency prediction systems including DeepGaze-II (Kummerer 2017) and SALICON (Huang et al. 2017). Finally, we also contribute a new repository to promote the research on adversarial attack and defense over pixel-to-pixel tasks: https://github.com/CZHQuality/AAA-Pix2pix.
48
Shi, Zheyuan Ryan, Aaron Schlenker, Brian Hay, et al. "Draining the Water Hole: Mitigating Social Engineering Attacks with CyberTWEAK." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 08 (2020): 13363–68. http://dx.doi.org/10.1609/aaai.v34i08.7050.
Full textAbstract:
Cyber adversaries have increasingly leveraged social engineering attacks to breach large organizations and threaten the well-being of today's online users. One clever technique, the “watering hole” attack, compromises a legitimate website to execute drive-by download attacks by redirecting users to another malicious domain. We introduce a game-theoretic model that captures the salient aspects for an organization protecting itself from a watering hole attack by altering the environment information in web traffic so as to deceive the attackers. Our main contributions are (1) a novel Social Engineering Deception (SED) game model that features a continuous action set for the attacker, (2) an in-depth analysis of the SED model to identify computationally feasible real-world cases, and (3) the CyberTWEAK algorithm which solves for the optimal protection policy. To illustrate the potential use of our framework, we built a browser extension based on our algorithms which is now publicly available online. The CyberTWEAK extension will be vital to the continued development and deployment of countermeasures for social engineering.
49
Qureshi, Ayyaz Ul Haq, Hadi Larijani, Mehdi Yousefi, Ahsan Adeel, and Nhamoinesu Mtetwa. "An Adversarial Approach for Intrusion Detection Systems Using Jacobian Saliency Map Attacks (JSMA) Algorithm." Computers 9, no. 3 (2020): 58. http://dx.doi.org/10.3390/computers9030058.
Full textAbstract:
In today’s digital world, the information systems are revolutionizing the way we connect. As the people are trying to adopt and integrate intelligent systems into daily lives, the risks around cyberattacks on user-specific information have significantly grown. To ensure safe communication, the Intrusion Detection Systems (IDS) were developed often by using machine learning (ML) algorithms that have the unique ability to detect malware against network security violations. Recently, it was reported that the IDS are prone to carefully crafted perturbations known as adversaries. With the aim to understand the impact of such attacks, in this paper, we have proposed a novel random neural network-based adversarial intrusion detection system (RNN-ADV). The NSL-KDD dataset is utilized for training. For adversarial attack crafting, the Jacobian Saliency Map Attack (JSMA) algorithm is used, which identifies the feature which can cause maximum change to the benign samples with minimum added perturbation. To check the effectiveness of the proposed adversarial scheme, the results are compared with a deep neural network which indicates that RNN-ADV performs better in terms of accuracy, precision, recall, F1 score and training epochs.
50
Mao, Junjie, Bin Weng, Tianqiang Huang, Feng Ye, and Liqing Huang. "Research on Multimodality Face Antispoofing Model Based on Adversarial Attacks." Security and Communication Networks 2021 (August 9, 2021): 1–12. http://dx.doi.org/10.1155/2021/3670339.
Full textAbstract:
Face antispoofing detection aims to identify whether the user’s face identity information is legal. Multimodality models generally have high accuracy. However, the existing works of face antispoofing detection have the problem of insufficient research on the safety of the model itself. Therefore, the purpose of this paper is to explore the vulnerability of existing face antispoofing models, especially multimodality models, when resisting various types of attacks. In this paper, we firstly study the resistance ability of multimodality models when they encounter white-box attacks and black-box attacks from the perspective of adversarial examples. Then, we propose a new method that combines mixed adversarial training and differentiable high-frequency suppression modules to effectively improve model safety. Experimental results show that the accuracy of the multimodality face antispoofing model is reduced from over 90% to about 10% when it is attacked by adversarial examples. But, after applying the proposed defence method, the model can still maintain more than 90% accuracy on original examples, and the accuracy of the model can reach more than 80% on attack examples.