To see the other types of publications on this topic, follow the link: Network anomaly detection.
Dissertations / Theses on the topic 'Network anomaly detection'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 dissertations / theses for your research on the topic 'Network anomaly detection.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse dissertations / theses on a wide variety of disciplines and organise your bibliography correctly.
1
Mazel, Johan. "Unsupervised network anomaly detection." Thesis, Toulouse, INSA, 2011. http://www.theses.fr/2011ISAT0024/document.
Full textAbstract:
La détection d'anomalies est une tâche critique de l'administration des réseaux. L'apparition continue de nouvelles anomalies et la nature changeante du trafic réseau compliquent de fait la détection d'anomalies. Les méthodes existantes de détection d'anomalies s'appuient sur une connaissance préalable du trafic : soit via des signatures créées à partir d'anomalies connues, soit via un profil de normalité. Ces deux approches sont limitées : la première ne peut détecter les nouvelles anomalies et la seconde requiert une constante mise à jour de son profil de normalité. Ces deux aspects limitent de façon importante l'efficacité des méthodes de détection existantes.Nous présentons une approche non-supervisée qui permet de détecter et caractériser les anomalies réseaux de façon autonome. Notre approche utilise des techniques de partitionnement afin d'identifier les flux anormaux. Nous proposons également plusieurs techniques qui permettent de traiter les anomalies extraites pour faciliter la tâche des opérateurs. Nous évaluons les performances de notre système sur des traces de trafic réel issues de la base de trace MAWI. Les résultats obtenus mettent en évidence la possibilité de mettre en place des systèmes de détection d'anomalies autonomes et fonctionnant sans connaissance préalableAnomaly detection has become a vital component of any network in today’s Internet. Ranging from non-malicious unexpected events such as flash-crowds and failures, to network attacks such as denials-of-service and network scans, network traffic anomalies can have serious detrimental effects on the performance and integrity of the network. The continuous arising of new anomalies and attacks create a continuous challenge to cope with events that put the network integrity at risk. Moreover, the inner polymorphic nature of traffic caused, among other things, by a highly changing protocol landscape, complicates anomaly detection system's task. In fact, most network anomaly detection systems proposed so far employ knowledge-dependent techniques, using either misuse detection signature-based detection methods or anomaly detection relying on supervised-learning techniques. However, both approaches present major limitations: the former fails to detect and characterize unknown anomalies (letting the network unprotected for long periods) and the latter requires training over labeled normal traffic, which is a difficult and expensive stage that need to be updated on a regular basis to follow network traffic evolution. Such limitations impose a serious bottleneck to the previously presented problem.We introduce an unsupervised approach to detect and characterize network anomalies, without relying on signatures, statistical training, or labeled traffic, which represents a significant step towards the autonomy of networks. Unsupervised detection is accomplished by means of robust data-clustering techniques, combining Sub-Space clustering with Evidence Accumulation or Inter-Clustering Results Association, to blindly identify anomalies in traffic flows. Correlating the results of several unsupervised detections is also performed to improve detection robustness. The correlation results are further used along other anomaly characteristics to build an anomaly hierarchy in terms of dangerousness. Characterization is then achieved by building efficient filtering rules to describe a detected anomaly. The detection and characterization performances and sensitivities to parameters are evaluated over a substantial subset of the MAWI repository which contains real network traffic traces.Our work shows that unsupervised learning techniques allow anomaly detection systems to isolate anomalous traffic without any previous knowledge. We think that this contribution constitutes a great step towards autonomous network anomaly detection.This PhD thesis has been funded through the ECODE project by the European Commission under the Framework Programme 7. The goal of this project is to develop, implement, and validate experimentally a cognitive routing system that meet the challenges experienced by the Internet in terms of manageability and security, availability and accountability, as well as routing system scalability and quality. The concerned use case inside the ECODE project is network anomaly
2
Brauckhoff, Daniela. "Network traffic anomaly detection and evaluation." Aachen Shaker, 2010. http://d-nb.info/1001177746/04.
Full text
3
Udd, Robert. "Anomaly Detection in SCADA Network Traffic." Thesis, Linköpings universitet, Programvara och system, 2015. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-122680.
Full textAbstract:
Critical infrastructure provides us with the most important parts of modern society, electricity, water and transport. To increase efficiency and to meet new demands from the customer remote monitoring and control of the systems is necessary. This opens new ways for an attacker to reach the Supervisory Control And Data Acquisition (SCADA) systems that control and monitors the physical processes involved. This also increases the need for security features specially designed for these settings. Anomaly-based detection is a technique suitable for the more deterministic SCADA systems. This thesis uses a combination of two techniques to detect anomalies. The first technique is an automatic whitelist that learns the behavior of the network flows. The second technique utilizes the differences in arrival times of the network packets. A prototype anomaly detector has been developed in Bro. To analyze the IEC 60870-5-104 protocol a new parser for Bro was also developed. The resulting anomaly detector was able to achieve a high detection rate for three of the four different types of attacks evaluated. The studied methods of detection are promising when used in a highly deterministic setting, such as a SCADA system.
4
Kabore, Raogo. "Hybrid deep neural network anomaly detection system for SCADA networks." Thesis, Ecole nationale supérieure Mines-Télécom Atlantique Bretagne Pays de la Loire, 2020. http://www.theses.fr/2020IMTA0190.
Full textAbstract:
Les systèmes SCADA sont de plus en plus ciblés par les cyberattaques en raison de nombreuses vulnérabilités dans le matériel, les logiciels, les protocoles et la pile de communication. Ces systèmes utilisent aujourd'hui du matériel, des logiciels, des systèmes d'exploitation et des protocoles standard. De plus, les systèmes SCADA qui étaient auparavant isolés sont désormais interconnectés aux réseaux d'entreprise et à Internet, élargissant ainsi la surface d'attaque. Dans cette thèse, nous utilisons une approche deep learning pour proposer un réseau de neurones profonds hybride efficace pour la détection d'anomalies dans les systèmes SCADA. Les principales caractéristiques des données SCADA sont apprises de manière automatique et non supervisée, puis transmises à un classificateur supervisé afin de déterminer si ces données sont normales ou anormales, c'est-à-dire s'il y a une cyber-attaque ou non. Par la suite, en réponse au défi dû au temps d’entraînement élevé des modèles deep learning, nous avons proposé une approche distribuée de notre système de détection d'anomalies afin de réduire le temps d’entraînement de notre modèleSCADA systems are more and more targeted by cyber-attacks because of many vulnerabilities inhardware, software, protocols and the communication stack. Those systems nowadays use standard hardware, software, operating systems and protocols. Furthermore, SCADA systems which used to be air-gaped are now interconnected to corporate networks and to the Internet, widening the attack surface.In this thesis, we are using a deep learning approach to propose an efficient hybrid deep neural network for anomaly detection in SCADA systems. The salient features of SCADA data are automatically and unsupervisingly learnt, and then fed to a supervised classifier in order to dertermine if those data are normal or abnormal, i.e if there is a cyber-attack or not. Afterwards, as a response to the challenge caused by high training time of deep learning models, we proposed a distributed approach of our anomaly detection system in order lo lessen the training time of our model
5
Balupari, Ravindra. "Real-time network-based anomaly intrusion detection." Ohio : Ohio University, 2002. http://www.ohiolink.edu/etd/view.cgi?ohiou1174579398.
Full text
6
Patcha, Animesh. "Network Anomaly Detection with Incomplete Audit Data." Diss., Virginia Tech, 2006. http://hdl.handle.net/10919/28334.
Full textAbstract:
With the ever increasing deployment and usage of gigabit networks, traditional network anomaly detection based intrusion detection systems have not scaled accordingly. Most, if not all, systems deployed assume the availability of complete and clean data for the purpose of intrusion detection. We contend that this assumption is not valid. Factors like noise in the audit data, mobility of the nodes, and the large amount of data generated by the network make it difficult to build a normal traffic profile of the network for the purpose of anomaly detection.
From this perspective, the leitmotif of the research effort described in this dissertation is the design of a novel intrusion detection system that has the capability to detect intrusions with high accuracy even when complete audit data is not available. In this dissertation, we take a holistic approach to anomaly detection to address the threats posed by network based denial-of-service attacks by proposing improvements in every step of the intrusion detection process. At the data collection phase, we have implemented an adaptive sampling scheme that intelligently samples incoming network data to reduce the volume of traffic sampled, while maintaining the intrinsic characteristics of the network traffic. A Bloom filters based fast flow aggregation scheme is employed at the data pre-processing stage to further reduce the response time of the anomaly detection scheme. Lastly, this dissertation also proposes an expectation-maximization algorithm based anomaly detection scheme that uses the sampled audit data to detect intrusions in the incoming network traffic.Ph. D.
7
Salzwedel, Jason Paul. "Anomaly detection in a mobile data network." Master's thesis, Faculty of Science, 2019. http://hdl.handle.net/11427/31202.
Full textAbstract:
The dissertation investigated the creation of an anomaly detection approach to
identify anomalies in the SGW elements of a LTE network. Unsupervised techniques
were compared and used to identify and remove anomalies in the training data set.
This “cleaned” data set was then used to train an autoencoder in an semi-supervised
approach. The resultant autoencoder was able to indentify normal observations. A
subsequent data set was then analysed by the autoencoder. The resultant
reconstruction errors were then compared to the ground truth events to investigate
the effectiveness of the autoencoder’s anomaly detection capability.
8
Babaie, Tahereh Tara. "New Methods for Network Traffic Anomaly Detection." Thesis, The University of Sydney, 2014. http://hdl.handle.net/2123/12032.
Full textAbstract:
In this thesis we examine the efficacy of applying outlier detection techniques to understand the behaviour of anomalies in communication network traffic. We have identified several shortcomings. Our most finding is that known techniques either focus on characterizing the spatial or temporal behaviour of traffic but rarely both. For example DoS attacks are anomalies which violate temporal patterns while port scans violate the spatial equilibrium of network traffic. To address this observed weakness we have designed a new method for outlier detection based spectral decomposition of the Hankel matrix. The Hankel matrix is spatio-temporal correlation matrix and has been used in many other domains including climate data analysis and econometrics. Using our approach we can seamlessly integrate the discovery of both spatial and temporal anomalies. Comparison with other state of the art methods in the networks community confirms that our approach can discover both DoS and port scan attacks. The spectral decomposition of the Hankel matrix is closely tied to the problem of inference in Linear Dynamical Systems (LDS). We introduce a new problem, the Online Selective Anomaly Detection (OSAD) problem, to model the situation where the objective is to report new anomalies in the system and suppress know faults. For example, in the network setting an operator may be interested in triggering an alarm for malicious attacks but not on faults caused by equipment failure. In order to solve OSAD we combine techniques from machine learning and control theory in a unique fashion. Machine Learning ideas are used to learn the parameters of an underlying data generating system. Control theory techniques are used to model the feedback and modify the residual generated by the data generating state model. Experiments on synthetic and real data sets confirm that the OSAD problem captures a general scenario and tightly integrates machine learning and control theory to solve a practical problem.
9
Mantere, M. (Matti). "Network security monitoring and anomaly detection in industrial control system networks." Doctoral thesis, Oulun yliopisto, 2015. http://urn.fi/urn:isbn:9789526208152.
Full textAbstract:
Abstract Industrial control system (ICS) networks used to be isolated environments, typically separated by physical air gaps from the wider area networks. This situation has been changing and the change has brought with it new cybersecurity issues. The process has also exacerbated existing problems that were previously less exposed due to the systems’ relative isolation. This process of increasing connectivity between devices, systems and persons can be seen as part of a paradigm shift called the Internet of Things (IoT). This change is progressing and the industry actors need to take it into account when working to improve the cybersecurity of ICS environments and thus their reliability. Ensuring that proper security processes and mechanisms are being implemented and enforced on the ICS network level is an important part of the general security posture of any given industrial actor. Network security and the detection of intrusions and anomalies in the context of ICS networks are the main high-level research foci of this thesis. These issues are investigated through work on machine learning (ML) based anomaly detection (AD). Potentially suitable features, approaches and algorithms for implementing a network anomaly detection system for use in ICS environments are investigated. After investigating the challenges, different approaches and methods, a proof-ofconcept (PoC) was implemented. The PoC implementation is built on top of the Bro network security monitoring framework (Bro) for testing the selected approach and tools. In the PoC, a Self-Organizing Map (SOM) algorithm is implemented using Bro scripting language to demonstrate the feasibility of using Bro as a base system. The implemented approach also represents a minimal case of event-driven machine learning anomaly detection (EMLAD) concept conceived during the research. The contributions of this thesis are as follows: a set of potential features for use in machine learning anomaly detection, proof of the feasibility of the machine learning approach in ICS network setting, a concept for event-driven machine learning anomaly detection, a design and initial implementation of user configurable and extendable machine learning anomaly detection framework for ICS networksTiivistelmä Kehittyneet yhteiskunnat käyttävät teollisuuslaitoksissaan ja infrastruktuuriensa operoinnissa monimuotoisia automaatiojärjestelmiä. Näiden automaatiojärjestelmien tieto- ja kyberturvallisuuden tila on hyvin vaihtelevaa. Laitokset ja niiden hyödyntämät järjestelmät voivat edustaa usean eri aikakauden tekniikkaa ja sisältää useiden eri aikakauden heikkouksia ja haavoittuvaisuuksia. Järjestelmät olivat aiemmin suhteellisen eristyksissä muista tietoverkoista kuin omista kommunikaatioväylistään. Tämä automaatiojärjestelmien eristyneisyyden heikkeneminen on luonut uuden joukon uhkia paljastamalla niiden kommunikaatiorajapintoja ympäröivälle maailmalle. Nämä verkkoympäristöt ovat kuitenkin edelleen verrattaen eristyneitä ja tätä ominaisuutta voidaan hyödyntää niiden valvonnassa. Tässä työssä esitetään tutkimustuloksia näiden verkkojen turvallisuuden valvomisesta erityisesti poikkeamien havainnoinnilla käyttäen hyväksi koneoppimismenetelmiä. Alkuvaiheen haasteiden ja erityispiirteiden tutkimuksen jälkeen työssä käytetään itsejärjestyvien karttojen (Self-Organizing Map, SOM) algoritmia esimerkkiratkaisun toteutuksessa uuden konseptin havainnollistamiseksi. Tämä uusi konsepti on tapahtumapohjainen koneoppiva poikkeamien havainnointi (Event-Driven Machine Learning Anomaly Detection, EMLAD). Työn kontribuutiot ovat seuraavat, kaikki teollisuusautomaatioverkkojen kontekstissa: ehdotus yhdeksi anomalioiden havainnoinnissa käytettävien ominaisuuksien ryhmäksi, koneoppivan poikkeamien havainnoinnin käyttökelpoisuuden toteaminen, laajennettava ja joustava esimerkkitoteutus uudesta EMLAD-konseptista toteutettuna Bro NSM työkalun ohjelmointikielellä
10
Brauckhoff, Daniela [Verfasser]. "Network Traffic Anomaly Detection and Evaluation / Daniela Brauckhoff." Aachen : Shaker, 2010. http://d-nb.info/1122546610/34.
Full text
11
Ding, Qi. "Statistical topics relating to computer network anomaly detection." Thesis, Boston University, 2012. https://hdl.handle.net/2144/31538.
Full textAbstract:
Thesis (Ph.D.)--Boston UniversityPLEASE NOTE: Boston University Libraries did not receive an Authorization To Manage form for this thesis or dissertation. It is therefore not openly accessible, though it may be available by request. If you are the author or principal advisor of this work and would like to request open access for it, please contact us at open-help@bu.edu. Thank you.
This dissertation makes fundamental contributions to statistical methods relating to the detection of anomalies in the context of computer network traffic monitoring. In particular, it contributes basic statistical tools for socially-based network anomaly characterization and detection, it extends a popular detection methodology to high-dimensional contexts, and it demonstrates that standard flow sampling can interact with inherent network topology in ways unexpected. In the first contribution of my research, I define anomalous intrusion in terms of locations in social space, rather than in physical space. I develop statistical detectors based on simple graph-based summaries of the network, with a focus on detecting anti-social behaviors. This research suggests that certain values of local graphical measurements, like clustering coefficients and betweenness centrality, are associated with the malicious antisocial behaviors in the types of network representations of IP flow measurements used in this work. This motivates me to propose a simple, efficient and robust anomaly detection technique. I evaluate this methodology on different network representations and using different social summaries. In the second contribution of my research, I extend the use of the PCA subspace method to high-dimensional spaces. Specifically, I show that, under appropriate conditions,with high probability the magnitude of the residuals of a standard PCA subspace analysis of randomly projected data behaves comparably to that of the residuals of a similar PCA analysis of the original data. My results indicate the feasibility of applying subspacebased anomaly detection algorithms to Gaussian random projection data. This concept is illustrated in the context of computer network traffic anomaly detection for the purpose of detecting volume anomalies. The impact of sampling on so-called Peer-to-Peer (P2P) network analysis is the focus of the third contribution of my research. In this research I use a combination of probability calculations and simulation techniques to characterize the extent to which standard packet sampling in the Internet can adversely affect the topology of stylized versions of Bittorrent download networks reconstructed from measurements of network flows. The results indicate that a certain stratification observed in these networks impacts the reconstructed topology in ways decidedly different from typical networks which have no stratification.
2031-01-01
12
Labonne, Maxime. "Anomaly-based network intrusion detection using machine learning." Electronic Thesis or Diss., Institut polytechnique de Paris, 2020. http://www.theses.fr/2020IPPAS011.
Full textAbstract:
Ces dernières années, le piratage est devenu une industrie à part entière, augmentant le nombre et la diversité des cyberattaques. Les menaces qui pèsent sur les réseaux informatiques vont des logiciels malveillants aux attaques par déni de service, en passant par le phishing et l'ingénierie sociale. Un plan de cybersécurité efficace ne peut plus reposer uniquement sur des antivirus et des pare-feux pour contrer ces menaces : il doit inclure plusieurs niveaux de défense. Les systèmes de détection d'intrusion (IDS) réseaux sont un moyen complémentaire de renforcer la sécurité, avec la possibilité de surveiller les paquets de la couche 2 (liaison) à la couche 7 (application) du modèle OSI. Les techniques de détection d'intrusion sont traditionnellement divisées en deux catégories : la détection par signatures et la détection par anomalies. La plupart des IDS utilisés aujourd'hui reposent sur la détection par signatures ; ils ne peuvent cependant détecter que des attaques connues. Les IDS utilisant la détection par anomalies sont capables de détecter des attaques inconnues, mais sont malheureusement moins précis, ce qui génère un grand nombre de fausses alertes. Dans ce contexte, la création d'IDS précis par anomalies est d'un intérêt majeur pour pouvoir identifier des attaques encore inconnues.Dans cette thèse, les modèles d'apprentissage automatique sont étudiés pour créer des IDS qui peuvent être déployés dans de véritables réseaux informatiques. Tout d'abord, une méthode d'optimisation en trois étapes est proposée pour améliorer la qualité de la détection : 1/ augmentation des données pour rééquilibrer les jeux de données, 2/ optimisation des paramètres pour améliorer les performances du modèle et 3/ apprentissage ensembliste pour combiner les résultats des meilleurs modèles. Les flux détectés comme des attaques peuvent être analysés pour générer des signatures afin d'alimenter les bases de données d'IDS basées par signatures. Toutefois, cette méthode présente l'inconvénient d'exiger des jeux de données étiquetés, qui sont rarement disponibles dans des situations réelles. L'apprentissage par transfert est donc étudié afin d'entraîner des modèles d'apprentissage automatique sur de grands ensembles de données étiquetés, puis de les affiner sur le trafic normal du réseau à surveiller. Cette méthode présente également des défauts puisque les modèles apprennent à partir d'attaques déjà connues, et n'effectuent donc pas réellement de détection d'anomalies. C'est pourquoi une nouvelle solution basée sur l'apprentissage non supervisé est proposée. Elle utilise l'analyse de l'en-tête des protocoles réseau pour modéliser le comportement normal du trafic. Les anomalies détectées sont ensuite regroupées en attaques ou ignorées lorsqu'elles sont isolées. Enfin, la détection la congestion réseau est étudiée. Le taux d'utilisation de la bande passante entre les différents liens est prédit afin de corriger les problèmes avant qu'ils ne se produisentIn recent years, hacking has become an industry unto itself, increasing the number and diversity of cyber attacks. Threats on computer networks range from malware to denial of service attacks, phishing and social engineering. An effective cyber security plan can no longer rely solely on antiviruses and firewalls to counter these threats: it must include several layers of defence. Network-based Intrusion Detection Systems (IDSs) are a complementary means of enhancing security, with the ability to monitor packets from OSI layer 2 (Data link) to layer 7 (Application). Intrusion detection techniques are traditionally divided into two categories: signatured-based (or misuse) detection and anomaly detection. Most IDSs in use today rely on signature-based detection; however, they can only detect known attacks. IDSs using anomaly detection are able to detect unknown attacks, but are unfortunately less accurate, which generates a large number of false alarms. In this context, the creation of precise anomaly-based IDS is of great value in order to be able to identify attacks that are still unknown.In this thesis, machine learning models are studied to create IDSs that can be deployed in real computer networks. Firstly, a three-step optimization method is proposed to improve the quality of detection: 1/ data augmentation to rebalance the dataset, 2/ parameters optimization to improve the model performance and 3/ ensemble learning to combine the results of the best models. Flows detected as attacks can be analyzed to generate signatures to feed signature-based IDS databases. However, this method has the disadvantage of requiring labelled datasets, which are rarely available in real-life situations. Transfer learning is therefore studied in order to train machine learning models on large labeled datasets, then finetune them on benign traffic of the network to be monitored. This method also has flaws since the models learn from already known attacks, and therefore do not actually perform anomaly detection. Thus, a new solution based on unsupervised learning is proposed. It uses network protocol header analysis to model normal traffic behavior. Anomalies detected are then aggregated into attacks or ignored when isolated. Finally, the detection of network congestion is studied. The bandwidth utilization between different links is predicted in order to correct issues before they occur
13
Zhao, Meng John. "Analysis and Evaluation of Social Network Anomaly Detection." Diss., Virginia Tech, 2017. http://hdl.handle.net/10919/79849.
Full textAbstract:
As social networks become more prevalent, there is significant interest in studying these network data, the focus often being on detecting anomalous events. This area of research is referred to as social network surveillance or social network change detection. While there are a variety of proposed methods suitable for different monitoring situations, two important issues have yet to be completely addressed in network surveillance literature. First, performance assessments using simulated data to evaluate the statistical performance of a particular method. Second, the study of aggregated data in social network surveillance. The research presented tackle these issues in two parts, evaluation of a popular anomaly detection method and investigation of the effects of different aggregation levels on network anomaly detection.Ph. D.
14
Dhanapalan, Manojprasadh. "Topology-aware Correlated Network Anomaly Detection and Diagnosis." The Ohio State University, 2012. http://rave.ohiolink.edu/etdc/view?acc_num=osu1339742606.
Full text
15
Fiore, Ugo. "Improving Network Anomaly Detection with Independent Component Analysis." Doctoral thesis, Universita degli studi di Salerno, 2015. http://hdl.handle.net/10556/1978.
Full textAbstract:
2013 - 2014Complexity, sophistication, and rate of growth of modern networks, coupled with the depth, continuity, and pervasiveness of their role in our everyday lives, stress the importance of identifying potential misuse or threats that could undermine regular operation. To ensure an adequate and prompt reaction, anomalies in network traffic should be detected, classified, and identified as quickly and correctly as possible. Several approaches focus on inspecting the content of packets traveling through the network, while other techniques aim at detecting suspicious activity by measuring the network state and comparing it with an expected baseline. Formalizing a model for normal behavior requires the collection and analysis of traffic, in order to isolate a set of features capable of describing traffic completely and in a compact way. The main focus of this dissertation is the quest for good representations for network traffic, representation that are abstract and can capture and describe much of the intricate structure of observed data in a simple manner. In this way, some of the hidden factors and variables governing the traffic data generation process can be unveiled and disentangled and anomalous events can be spotted more reliably. We adopted several methods to achieve such simpler representations, including Independent Component Analysis and deep learning architectures. Machine learning techniques have been used for verifying the improvement in classification effectiveness that can be achieved with the proposed representations. [edited by Author]
XIII n.s.
16
Olsson, Jonathan. "Detecting Faulty Piles of Wood using Anomaly Detection Techniques." Thesis, Luleå tekniska universitet, Institutionen för system- och rymdteknik, 2021. http://urn.kb.se/resolve?urn=urn:nbn:se:ltu:diva-83061.
Full textAbstract:
The forestry and the sawmill industry have a lot of incoming and outgoing piles of wood. It's important to maintain quality and efficiency. This motivates an examination of whether machine learning- or more specifically, anomaly detection techniques can be implemented and used to detect faulty shipments. This thesis presents and evaluates some computer vision techniques and some deep learning techniques. Deep learning can be divided into groups; supervised, semi-supervised and unsupervised. In this thesis, all three groups were examined and it covers supervised methods such as Convolutional Neural Networks, semi-supervised methods such as a modified Convolutional Autoencoder (CAE) and lastly, an unsupervised technique such as Generative Adversarial Network (GAN) was being tested and evaluated. A version of a GAN model proved to perform best for this thesis in terms of the accuracy of faulty detecting shipments with an accuracy rate of 68.2% and 79.8\% overall, which was satisfactory given the problems that were discovered during the progress of the thesis.
17
Jadidi, Zahra. "Flow-based Anomaly Detection in High-Speed Networks." Thesis, Griffith University, 2016. http://hdl.handle.net/10072/367890.
Full textAbstract:
With the advent of online services, the Internet has become extremely busy and demanding faster access. The increased dependency on the Internet obliges Internet service providers to make it reliable and secure. In this regard, researchers are tirelessly working on a number of technologies in order to ensure the continued viability of the Internet. Intrusion detection is one of the fields that enables secure operation of the Internet. An intrusion detection system (IDS) attempts to discover malicious activities in a network. However, with the increasing network throughput, IDSs should be able to analyse high volumes of traffic in real-time. Flow-based analysis is one of the methods capable of handling high-volume traffic. This method reduces the input traffic of IDSs because it analyses only packet headers. Flow-based anomaly detection can increase the reliability of the Internet, provided this method is functional at an early stage and complemented by packet-based IDSs at later stages.
Employing artificial intelligence (AI) methods in IDSs provides the capability to detect attacks with better accuracy. Compared with typical IDSs, AI-based systems are more inclined towards detecting unknown attacks. This thesis proposes an artificial neural network (ANN) based flow anomaly detector optimised with metaheuristic algorithms. The proposed method is evaluated using a number of flow-based datasets generated. An ANN-based flow anomaly detection enables a high detection rate; hence, this thesis investigates this system more thoroughly. The ANN-based system is a supervised method which needs labelled datasets; however, labelling of a large amount of data found in high-speed networks is difficult. Semi-supervised methods are the combination of supervised and unsupervised methods, which can work with both labelled and unlabelled data. A semi-supervised method can provide a high detection rate even when there is a small proportion of labelled data; therefore, the application of this method in flow-based anomaly detection is considered.Thesis (PhD Doctorate)
Doctor of Philosophy (PhD)
School of Information and Cmmunication Technology
Science, Environment, Engineering and Technology
Full Text
18
Abuaitah, Giovani Rimon. "ANOMALIES IN SENSOR NETWORK DEPLOYMENTS: ANALYSIS, MODELING, AND DETECTION." Wright State University / OhioLINK, 2013. http://rave.ohiolink.edu/etdc/view?acc_num=wright1376594068.
Full text
19
Ohlsson, Jonathan. "Anomaly Detection in Microservice Infrastructures." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-231993.
Full textAbstract:
Anomaly detection in time series is a broad field with many application areas, and has been researched for many years. In recent years the need for monitoring and DevOps has increased, partly due to the increased usage of microservice infrastructures. Applying time series anomaly detection to the metrics emitted by these microservices can yield new insights into the system health and could enable detecting anomalous conditions before they are escalated into a full incident. This thesis investigates how two proposed anomaly detectors, one based on the RPCA algorithm and the other on the HTM neural network, perform on metrics emitted by a microservice infrastructure, with the goal of enhancing the infrastructure monitoring. The detectors are evaluated against a random sample of metrics from a digital rights management company’s microservice infrastructure, as well as the open source NAB dataset. It is illustrated that both algorithms are able to detect every known incident in the company metrics tested. Their ability to detect anomalies is shown to be dependent on the defined threshold value for what qualifies as an outlier. The RPCA Detector proved to be better at detecting anomalies on the company microservice metrics, however the HTM detector performed better on the NAB dataset. Findings also highlight the difficulty of manually annotating anomalies even with domain knowledge. An issue found to be true for both the dataset created for this project, and the NAB dataset. The thesis concludes that the proposed detectors possess different abilities, both having their respective trade-offs. Although they are similar in detection accuracy and false positive rates, each has different inert abilities to perform tasks such as continuous monitoring or ease of deployment in an existing monitoring setup.Anomalitetsdetektering i tidsserier är ett brett område med många användningsområden och har undersökts under många år. De senaste åren har behovet av övervakning och DevOps ökat, delvis på grund av ökad användning av microservice-infrastrukturer. Att tillämpa tidsserieanomalitetsdetektering på de mätvärden som emitteras av dessa microservices kan ge nya insikter i systemhälsan och kan möjliggöra detektering av avvikande förhållanden innan de eskaleras till en fullständig incident. Denna avhandling undersöker hur två föreslagna anomalitetsdetektorer, en baserad på RPCA-algoritmen och den andra på HTM neurala nätverk, presterar på mätvärden som emitteras av en microservice-infrastruktur, med målet att förbättra infrastrukturövervakningen. Detektorerna utvärderas mot ett slumpmässigt urval av mätvärden från en microservice-infrastruktur på en digital underhållningstjänst, och från det öppet tillgängliga NAB-dataset. Det illustreras att båda algoritmerna kunde upptäcka alla kända incidenter i de testade underhållningstjänst-mätvärdena. Deras förmåga att upptäcka avvikelser visar sig vara beroende av det definierade tröskelvärdet för vad som kvalificeras som en anomali. RPCA-detektorn visade sig bättre på att upptäcka anomalier i underhållningstjänstens mätvärden, men HTM-detektorn presterade bättre på NAB-datasetet. Fynden markerar också svårigheten med att manuellt annotera avvikelser, även med domänkunskaper. Ett problem som visat sig vara sant för datasetet skapat för detta projekt och NAB-datasetet. Avhandlingen slutleder att de föreslagna detektorerna har olikaförmågor, vilka båda har sina respektive avvägningar. De har liknande detekteringsnoggrannhet, men har olika inerta förmågor för att utföra uppgifter som kontinuerlig övervakning, eller enkelhet att installera i en befintlig övervakningsinstallation.
20
Moe, Lwin P. "Cyber security risk analysis framework : network traffic anomaly detection." Thesis, Massachusetts Institute of Technology, 2018. http://hdl.handle.net/1721.1/118536.
Full textAbstract:
Thesis: S.M. in Engineering and Management, Massachusetts Institute of Technology, System Design and Management Program, 2018.Cataloged from PDF version of thesis.
Includes bibliographical references (pages 84-86).
Cybersecurity is a growing research area with direct commercial impact to organizations and companies in every industry. With all other technological advancements in the Internet of Things (IoT), mobile devices, cloud computing, 5G network, and artificial intelligence, the need for cybersecurity is more critical than ever before. These technologies drive the need for tighter cybersecurity implementations, while at the same time act as enablers to provide more advanced security solutions. This paper will discuss a framework that can predict cybersecurity risk by identifying normal network behavior and detect network traffic anomalies. Our research focuses on the analysis of the historical network traffic data to identify network usage trends and security vulnerabilities. Specifically, this thesis will focus on multiple components of the data analytics platform. It explores the big data platform architecture, and data ingestion, analysis, and engineering processes. The experiments were conducted utilizing various time series algorithms (Seasonal ETS, Seasonal ARIMA, TBATS, Double-Seasonal Holt-Winters, and Ensemble methods) and Long Short-Term Memory Recurrent Neural Network algorithm. Upon creating the baselines and forecasting network traffic trends, the anomaly detection algorithm was implemented using specific thresholds to detect network traffic trends that show significant variation from the baseline. Lastly, the network traffic data was analyzed and forecasted in various dimensions: total volume, source vs. destination volume, protocol, port, machine, geography, and network structure and pattern. The experiments were conducted with multiple approaches to get more insights into the network patterns and traffic trends to detect anomalies.
by Lwin P. Moe.
S.M. in Engineering and Management
21
Lawal, Yusuf Lanre. "Anomaly Detection in Ethereum Transactions Using Network Science Analytics." University of Cincinnati / OhioLINK, 2020. http://rave.ohiolink.edu/etdc/view?acc_num=ucin159585057190135.
Full text
22
Sarossy, George. "Anomaly detection in Network data with unsupervised learning methods." Thesis, Mälardalens högskola, Akademin för innovation, design och teknik, 2021. http://urn.kb.se/resolve?urn=urn:nbn:se:mdh:diva-55096.
Full textAbstract:
Anomaly detection has become a crucial part of the protection of information and integrity. Due to the increase of cyber threats the demand for anomaly detection has grown for companies. Anomaly detection on time series data aims to detect unexpected behavior on the system. Anomalies often occur online, and companies need to be able to protect themselves from these intrusions. Multiple machine learning algorithms have been used and researched to solve the problem with anomaly detection and it is ongoing research to find the most optimal algorithms. Therefore, this study investigates algorithms such as K-means, Mean Shift and DBSCAN algorithm could be a solution for the problem. The study also investigates if combining the algorithms will improve the result. The results that the study reveals that the combinations of the algorithms perform slightly worse than the individual algorithms regarding speed and accuracy to detect anomalies. The algorithms without combinations did perform well during this study, they have slight differences between each other, and the results show the DBSCAN algorithm has slightly better total detection compared to the other algorithms and has slower execution time. The conclusion for this study reveals that the Mean Shift algorithm had the fastest execution time and the DBSCAN algorithm had the highest accuracy. The study also reveals most of the combinations between the algorithms did not improve during the fusion. However, the DBSCAN + Mean Shift fusion did improve the accuracy, and the K-means + Mean Shift fusion did improve the execution time.
23
Carlsson, Oskar, and Daniel Nabhani. "User and Entity Behavior Anomaly Detection using Network Traffic." Thesis, Blekinge Tekniska Högskola, Institutionen för datalogi och datorsystemteknik, 2017. http://urn.kb.se/resolve?urn=urn:nbn:se:bth-14636.
Full text
24
Vignisson, Egill. "Anomaly Detection in Streaming Data from a Sensor Network." Thesis, KTH, Matematisk statistik, 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-257507.
Full textAbstract:
In this thesis, the use of unsupervised and semi-supervised machine learning techniques was analyzed as potential tools for anomaly detection in the sensor network that the electrical system in a Scania truck is comprised of. The experimentation was designed to analyse the need for both point and contextual anomaly detection in this setting. For the point anomaly detection the method of Isolation Forest was experimented with and for contextual anomaly detection two different recurrent neural network architectures using Long Short Term Memory units was relied on. One model was simply a many to one regression model trained to predict a certain signal, while the other was an encoder-decoder network trained to reconstruct a sequence. Both models were trained in an semi-supervised manner, i.e. on data that only depicts normal behaviour, which theoretically should lead to a performance drop on abnormal sequences resulting in higher error terms. In both setting the parameters of a Gaussian distribution were estimated using these error terms which allowed for a convenient way of defining a threshold which would decide if the observation would be flagged as anomalous or not. Additional experimentation's using an exponential weighted moving average over a number of past observations to filter the signal was also conducted. The models performance on this particular task was very different but the regression model showed a lot of promise especially when combined with a filtering preprocessing step to reduce the noise in the data. However the model selection will always be governed by the nature the particular task at hand so the other methods might perform better in other settings.I den här avhandlingen var användningen av oövervakad och halv-övervakad maskininlärning analyserad som ett möjligt verktyg för att upptäcka avvikelser av anomali i det sensornätverk som elektriska systemet en Scanialastbil består av. Experimentet var konstruerat för att analysera behovet av både punkt och kontextuella avvikelser av anomali i denna miljö. För punktavvikelse av anomali var metoden Isolation Forest experimenterad med och för kontextuella avvikelser av anomali användes två arkitekturer av återkommande neurala nätverk. En av modellerna var helt enkelt många-till-en regressionmodell tränad för att förutspå ett visst märke, medan den andre var ett kodare-avkodare nätverk tränat för att rekonstruera en sekvens.Båda modellerna blev tränade på ett halv-övervakat sätt, d.v.s. på data som endast visar normalt beteende, som teoretiskt skulle leda till minskad prestanda på onormala sekvenser som ger ökat antal feltermer. I båda fallen blev parametrarna av en Gaussisk distribution estimerade på grund av dessa feltermer som tillåter ett bekvämt sätt att definera en tröskel som skulle bestämma om iakttagelsen skulle bli flaggad som en anomali eller inte. Ytterligare experiment var genomförda med exponentiellt viktad glidande medelvärde över ett visst antal av tidigare iakttagelser för att filtera märket. Modellernas prestanda på denna uppgift var välidt olika men regressionmodellen lovade mycket, särskilt kombinerad med ett filterat förbehandlingssteg för att minska bruset it datan. Ändå kommer modelldelen alltid styras av uppgiftens natur så att andra metoder skulle kunna ge bättre prestanda i andra miljöer.
25
Liu, Ying. "Outlier detection by network flow." Birmingham, Ala. : University of Alabama at Birmingham, 2007. https://www.mhsl.uab.edu/dt/2007p/liu-ying.pdf.
Full textAbstract:
Thesis (Ph. D.)--University of Alabama at Birmingham, 2007.Additional advisors: Elliot J. Lefkowitz, Kevin D. Reilly, Robert Thacker, Chengcui Zhang. Description based on contents viewed Feb. 7, 2008; title from title screen. Includes bibliographical references (p. 125-132).
26
Kim, Seong Soo. "Real-time analysis of aggregate network traffic for anomaly detection." Texas A&M University, 2005. http://hdl.handle.net/1969.1/2312.
Full textAbstract:
The frequent and large-scale network attacks have led to an increased need for
developing techniques for analyzing network traffic. If efficient analysis tools were
available, it could become possible to detect the attacks, anomalies and to appropriately
take action to contain the attacks before they have had time to propagate across the
network.
In this dissertation, we suggest a technique for traffic anomaly detection based on
analyzing the correlation of destination IP addresses and distribution of image-based
signal in postmortem and real-time, by passively monitoring packet headers of traffic.
This address correlation data are transformed using discrete wavelet transform for
effective detection of anomalies through statistical analysis. Results from trace-driven
evaluation suggest that the proposed approach could provide an effective means of
detecting anomalies close to the source. We present a multidimensional indicator using
the correlation of port numbers as a means of detecting anomalies.
We also present a network measurement approach that can simultaneously detect,
identify and visualize attacks and anomalous traffic in real-time. We propose to
represent samples of network packet header data as frames or images. With such a
formulation, a series of samples can be seen as a sequence of frames or video. Thisenables techniques from image processing and video compression such as DCT to be
applied to the packet header data to reveal interesting properties of traffic. We show that
??scene change analysis?? can reveal sudden changes in traffic behavior or anomalies. We
show that ??motion prediction?? techniques can be employed to understand the patterns of
some of the attacks. We show that it may be feasible to represent multiple pieces of data
as different colors of an image enabling a uniform treatment of multidimensional packet
header data.
Measurement-based techniques for analyzing network traffic treat traffic volume
and traffic header data as signals or images in order to make the analysis feasible. In this
dissertation, we propose an approach based on the classical Neyman-Pearson Test
employed in signal detection theory to evaluate these different strategies. We use both of
analytical models and trace-driven experiments for comparing the performance of
different strategies. Our evaluations on real traces reveal differences in the effectiveness
of different traffic header data as potential signals for traffic analysis in terms of their
detection rates and false alarm rates. Our results show that address distributions and
number of flows are better signals than traffic volume for anomaly detection. Our results
also show that sometimes statistical techniques can be more effective than the NP-test
when the attack patterns change over time.
27
Alipour, Hamid Reza. "An Anomaly Behavior Analysis Methodology for Network Centric Systems." Diss., The University of Arizona, 2013. http://hdl.handle.net/10150/305804.
Full textAbstract:
Information systems and their services (referred to as cyberspace) are ubiquitous and touch all aspects of our life. With the exponential growth in cyberspace activities, the number and complexity of cyber-attacks have increased significantly due to an increase in the number of applications with vulnerabilities and the number of attackers. Consequently, it becomes extremely critical to develop efficient network Intrusion Detection Systems (IDS) that can mitigate and protect cyberspace resources and services against cyber-attacks. On the other hand, since each network system and application has its own specification as defined in its protocol, it is hard to develop a single IDS which works properly for all network protocols. The keener approach is to design customized detection engines for each protocol and then aggregate the reports from these engines to define the final security state of the system. In this dissertation, we developed a general methodology based on data mining, statistical analysis and protocol semantics to perform anomaly behavior analysis and detection for network-centric systems and their protocols. In our approach, we develop runtime models of protocol's state transitions during a time interval ΔΤ. We consider any n consecutive messages in a session during the time interval ΔΤ as an n-transition pattern called n-gram. By applying statistical analysis over these n-gram patterns we can accurately model the normal behavior of any protocol. Then we use the amount of the deviation from this normal model to quantify the anomaly score of the protocol activities. If this anomaly score is higher than a well-defined threshold the system marks that activity as a malicious activity. To validate our methodology, we have applied it to two different protocols: DNS (Domain Name System) at the application layer and the IEEE 802.11(WiFi) at the data link layer, where we have achieved good detection results (>95%) with low detection errors (<0.1%).
28
Riddell, Liam R. "Heterogeneous anomaly detection from network traffic streams using data summarization." Thesis, Edith Cowan University, Research Online, Perth, Western Australia, 2022. https://ro.ecu.edu.au/theses/2599.
Full textAbstract:
The extreme volumes of modern networks and the increasing demands on security professionals present a critical need for analysis efficiency. Network anomaly summarization combines the broad threat detection characteristics of anomaly detection with the big data reducing qualities of summarization. However, summarising anomalies from network traffic data streams presents numerous obstacles. This thesis proposes a novel attack to anomaly mapping technique for heterogeneous network threat classification and provides a novel auto-encoding latent reflection approach for summarising network anomalies. Key findings include several new heterogeneous anomaly variants, promising performance of the novel summarization method, and the shortcomings of existing evaluation metrics.
29
Di, Felice Marco. "Unsupervised anomaly detection in HPC systems." Master's thesis, Alma Mater Studiorum - Università di Bologna, 2019.
Abstract:
Alla base di questo studio vi è l'analisi di tecniche non supervisionate applicate per il rilevamento di stati anomali in sistemi HPC, complessi calcolatori capaci di raggiungere prestazioni dell'ordine dei PetaFLOPS. Nel mondo HPC, per anomalia si intende un particolare stato che induce un cambiamento delle prestazioni rispetto al normale funzionamento del sistema. Le anomalie possono essere di natura diversa come il guasto che può riguardare un componente, una configurazione errata o un'applicazione che entra in uno stato inatteso provocando una prematura interruzione dei processi. I datasets utilizzati in un questo progetto sono stati raccolti da D.A.V.I.D.E., un reale sistema HPC situato presso il CINECA di Casalecchio di Reno, o sono stati generati simulando lo stato di un singolo nodo di un virtuale sistema HPC analogo a quello del CINECA modellato secondo specifiche funzioni non lineari ma privo di rumore. Questo studio propone un approccio inedito, quello non supervisionato, mai applicato prima per svolgere anomaly detection in sistemi HPC. Si è focalizzato sull'individuazione dei possibili vantaggi indotti dall'uso di queste tecniche applicate in tale campo. Sono stati realizzati e mostrati alcuni casi che hanno prodotto raggruppamenti interessanti attraverso le combinazioni di Variational Autoencoders, un particolare tipo di autoencoder probabilistico con la capacità di preservare la varianza dell'input set nel suo spazio latente, e di algoritmi di clustering, come K-Means, DBSCAN, Gaussian Mixture ed altri già noti in letteratura.
30
Mdini, Maha. "Anomaly detection and root cause diagnosis in cellular networks." Thesis, Ecole nationale supérieure Mines-Télécom Atlantique Bretagne Pays de la Loire, 2019. http://www.theses.fr/2019IMTA0144/document.
Full textAbstract:
Grâce à l'évolution des outils d'automatisation et d'intelligence artificielle, les réseauxmobiles sont devenus de plus en plus dépendants de la machine. De nos jours, une grandepartie des tâches de gestion de réseaux est exécutée d'une façon autonome, sans interventionhumaine. Dans cette thèse, nous avons focalisé sur l'utilisation des techniques d'analyse dedonnées dans le but d'automatiser et de consolider le processus de résolution de défaillancesdans les réseaux. Pour ce faire, nous avons défini deux objectifs principaux : la détectiond'anomalies et le diagnostic des causes racines de ces anomalies. Le premier objectif consiste àdétecter automatiquement les anomalies dans les réseaux sans faire appel aux connaissancesdes experts. Pour atteindre cet objectif, nous avons proposé un algorithme, Watchmen AnomalyDetection (WAD), basé sur le concept de la reconnaissance de formes (pattern recognition). Cetalgorithme apprend le modèle du trafic réseau à partir de séries temporelles périodiques etdétecte des distorsions par rapport à ce modèle dans le flux de nouvelles données. Le secondobjectif a pour objet la détermination des causes racines des problèmes réseau sans aucuneconnaissance préalable sur l'architecture du réseau et des différents services. Pour ceci, nousavons conçu un algorithme, Automatic Root Cause Diagnosis (ARCD), qui permet de localiser lessources d'inefficacité dans le réseau. ARCD est composé de deux processus indépendants :l'identification des contributeurs majeurs à l'inefficacité globale du réseau et la détection desincompatibilités. WAD et ARCD ont fait preuve d'efficacité. Cependant, il est possible d'améliorerces algorithmes sur plusieurs aspectsWith the evolution of automation and artificial intelligence tools, mobile networks havebecome more and more machine reliant. Today, a large part of their management tasks runs inan autonomous way, without human intervention. In this thesis, we have focused on takingadvantage of the data analysis tools to automate the troubleshooting task and carry it to a deeperlevel. To do so, we have defined two main objectives: anomaly detection and root causediagnosis. The first objective is about detecting issues in the network automatically withoutincluding expert knowledge. To meet this objective, we have proposed an algorithm, WatchmenAnomaly Detection (WAD), based on pattern recognition. It learns patterns from periodic timeseries and detect distortions in the flow of new data. The second objective aims at identifying theroot cause of issues without any prior knowledge about the network topology and services. Toaddress this question, we have designed an algorithm, Automatic Root Cause Diagnosis (ARCD)that identifies the roots of network issues. ARCD is composed of two independent threads: MajorContributor identification and Incompatibility detection. WAD and ARCD have been proven to beeffective. However, many improvements of these algorithms are possible
31
Yellapragada, Ramani. "Probabilistic Model for Detecting Network Traffic Anomalies." Ohio University / OhioLINK, 2004. http://rave.ohiolink.edu/etdc/view?acc_num=ohiou1088538020.
Full text
32
Taylor, Adrian. "Anomaly-Based Detection of Malicious Activity in In-Vehicle Networks." Thesis, Université d'Ottawa / University of Ottawa, 2017. http://hdl.handle.net/10393/36120.
Full textAbstract:
Modern automobiles have been proven vulnerable to hacking by security researchers. By exploiting vulnerabilities in the car's external interfaces, attackers can access a car's controller area network (CAN) bus and cause malicious effects. We seek to detect these attacks on the bus as a last line of defence against automotive cyber attacks. The CAN bus standard defines a low-level message structure, upon which manufacturers layer their own proprietary command protocols; attacks must similarly be tailored for their target. This variability makes intrusion detection methods difficult to apply to the automotive CAN bus. Nevertheless, the bus traffic is generated by machines; thus we hypothesize that it can be characterized with machine learning, and that attacks produce anomalous traffic. Our goals are to show that anomaly detection trained without understanding of the message contents can detect attacks, and to create a framework for understanding how the characteristics of a novel attack can be used to predict its detectability.
We developed a model that describes attacks based on their effect on bus traffic, informed by a review of published material on car hacking in combination with analysis of CAN traffic from a 2012 Subaru Impreza. The model specifies three high-level categories of effects: attacks that insert foreign packets, attacks that affect packet timing, and attacks that only modify data within packets. Foreign packet attacks are trivially detectable. For timing-based anomalies, we developed features suitable for one-class classification methods. For packet stream data word anomalies, we adapted recurrent neural networks and multivariate Markov model methods to sequence anomaly detection and compared their performance.
We conducted experiments to evaluate our detection methods with special attention to the trade-off between precision and recall, given that a practical system requires a very low false alarm rate. The methods were evaluated by synthesizing anomalies within each attack category, parameterized to adjust their covertness. We generalize from the results to enable prediction of detection rates for new attacks using these methods.
33
Martignano, Anna. "Real-time Anomaly Detection on Financial Data." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-281832.
Full textAbstract:
This work presents an investigation of tailoring Network Representation Learning (NRL) for an application in the Financial Industry. NRL approaches are data-driven models that learn how to encode graph structures into low-dimensional vector spaces, which can be further exploited by downstream Machine Learning applications. They can potentially bring a lot of benefits in the Financial Industry since they extract in an automatic way features that can provide useful input regarding graph structures, called embeddings. Financial transactions can be represented as a network, and through NRL, it is possible to extract embeddings that reflect the intrinsic inter-connected nature of economic relationships. Such embeddings can be used for several purposes, among which Anomaly Detection to fight financial crime.This work provides a qualitative analysis over state-of-the-art NRL models, which identifies Graph Convolutional Network (ConvGNN) as the most suitable category of approaches for Financial Industry but with a certain need for further improvement. Financial Industry poses additional challenges when modelling a NRL solution. Despite the need of having a scalable solution to handle real-world graph with considerable dimensions, it is necessary to take into consideration several characteristics: transactions graphs are inherently dynamic since every day new transactions are executed and nodes can be heterogeneous. Besides, everything is further complicated by the need to have updated information in (near) real-time due to the sensitivity of the application domain. For these reasons, GraphSAGE has been considered as a base for the experiments, which is an inductive ConvGNN model. Two variants of GraphSAGE are presented: a dynamic variant whose weights evolve accordingly with the input sequence of graph snapshots, and a variant specifically meant to handle bipartite graphs. These variants have been evaluated by applying them to real-world data and leveraging the generated embeddings to perform Anomaly Detection. The experiments demonstrate that leveraging these variants leads toimagecomparable results with other state-of-the-art approaches, but having the advantage of being suitable to handle real-world financial data sets.Detta arbete presenterar en undersökning av tillämpningar av Network Representation Learning (NRL) inom den finansiella industrin. Metoder inom NRL möjliggör datadriven kondensering av grafstrukturer till lågdimensionella och lätthanterliga vektorer.Dessa vektorer kan sedan användas i andra maskininlärningsuppgifter. Närmare bestämt, kan metoder inom NRL underlätta hantering av och informantionsutvinning ur beräkningsintensiva och storskaliga grafer inom den finansiella sektorn, till exempel avvikelsehantering bland finansiella transaktioner. Arbetet med data av denna typ försvåras av det faktum att transaktionsgrafer är dynamiska och i konstant förändring. Utöver detta kan noderna, dvs transaktionspunkterna, vara vitt skilda eller med andra ord härstamma från olika fördelningar.I detta arbete har Graph Convolutional Network (ConvGNN) ansetts till den mest lämpliga lösningen för nämnda tillämpningar riktade mot upptäckt av avvikelser i transaktioner. GraphSAGE har använts som utgångspunkt för experimenten i två olika varianter: en dynamisk version där vikterna uppdateras allteftersom nya transaktionssekvenser matas in, och en variant avsedd särskilt för bipartita (tvådelade) grafer. Dessa varianter har utvärderats genom användning av faktiska datamängder med avvikelsehantering som slutmål.
34
Lin, Chih-Yuan. "A timing approach to network-based anomaly detection for SCADA systems." Licentiate thesis, Linköpings universitet, Programvara och system, 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-165155.
Full textAbstract:
Supervisory Control and Data Acquisition (SCADA) systems control and monitor critical infrastructure in society, such as electricity transmission and distribution systems. Modern SCADA systems are increasingly adopting open architectures, protocols, and standards and being connected to the Internet to enable remote control. A boost in sophisticated attacks against SCADA systems makes SCADA security a pressing issue. An Intrusion Detection System (IDS) is a security countermeasure that monitors a network and tracks unauthenticated activities inside the network. Most commercial IDSs used in general IT systems are signature-based, by which an IDS compares the system behaviors with known attack patterns. Unfortunately, recent attacks against SCADA systems exploit zero-day vulnerabilities in SCADA devices which are undetectable by signature-based IDSs. This thesis aims to enhance SCADA system monitoring by anomaly detection that models normal behaviors and finds deviations from the model. With anomaly detection, zero-day attacks are possible to detect. We focus on modeling the timing attributes of SCADA traffic for two reasons: (1) the timing regularity fits the automation nature of SCADA systems, and (2) the timing information (i.e., arrival time) of a packet is captured and sent by a network driver where an IDS is located. Hence, it’s less prone to intentional manipulation by an attacker, compared to the payload of a packet. This thesis first categorises SCADA traffic into two groups, request-response and spontaneous traffic, and studies data collected in three different protocol formats (Modbus, Siemens S7, and IEC-60870-5-104). The request-response traffic is generated by a polling mechanism. For this type of traffic, we model the inter-arrival times for each command and response pair with a statistical approach. Results presented in this thesis show that request-response traffic exists in several SCADA traffic sets collected from systems with different sizes and settings. The proposed statistical approach for request-response traffic can detect attacks having subtle changes in timing, such as a single packet insertion and TCP prediction for two of the three SCADA protocols studied. The spontaneous traffic is generated by remote terminal units when they see significant changes in measurement values. For this type of traffic, we first use a pattern mining approach to find the timing characteristics of the data. Then, we model the suggested attributes with machine learning approaches and run it on traffic collected in a real power facility. We test our anomaly detection model with two types of attacks. One causes persistent anomalies and another only causes intermittent ones. Our anomaly detector exhibits a 100% detection rate with at most 0.5% false positive rate for the attacks with persistent anomalies. For the attacks with intermittent anomalies, we find our approach effective when (1) the anomalies last for a longer period (over 1 hour), or (2) the original traffic has relatively low volume.
35
Zhou, Mian. "Network Intrusion Detection: Monitoring, Simulation and Visualization." Doctoral diss., University of Central Florida, 2005. http://digital.library.ucf.edu/cdm/ref/collection/ETD/id/4063.
Full textAbstract:
This dissertation presents our work on network intrusion detection and intrusion sim- ulation. The work in intrusion detection consists of two different network anomaly-based approaches. The work in intrusion simulation introduces a model using explicit traffic gen- eration for the packet level traffic simulation. The process of anomaly detection is to first build profiles for the normal network activity and then mark any events or activities that deviate from the normal profiles as suspicious. Based on the different schemes of creating the normal activity profiles, we introduce two approaches for intrusion detection. The first one is a frequency-based approach which creates a normal frequency profile based on the periodical patterns existed in the time-series formed by the traffic. It aims at those attacks that are conducted by running pre-written scripts, which automate the process of attempting connections to various ports or sending packets with fabricated payloads, etc. The second approach builds the normal profile based on variations of connection-based behavior of each single computer. The deviations resulted from each individual computer are carried out by a weight assignment scheme and further used to build a weighted link graph representing the overall traffic abnormalities. The functionality of this system is of a distributed personal IDS system that also provides a centralized traffic analysis by graphical visualization. It provides a finer control over the internal network by focusing on connection-based behavior of each single computer. For network intrusion simulation, we explore an alternative method for network traffic simulation using explicit traffic generation. In particular, we build a model to replay the standard DARPA traffic data or the traffic data captured from a real environment. The replayed traffic data is mixed with the attacks, such as DOS and Probe attack, which can create apparent abnormal traffic flow patterns. With the explicit traffic generation, every packet that has ever been sent by the victim and attacker is formed in the simulation model and travels around strictly following the criteria of time and path that extracted from the real scenario. Thus, the model provides a promising aid in the study of intrusion detection techniques.Ph.D.
School of Computer Science
Engineering and Computer Science
Computer Science
36
Syal, Astha. "Automatic Network Traffic Anomaly Detection and Analysis using SupervisedMachine Learning Techniques." Youngstown State University / OhioLINK, 2019. http://rave.ohiolink.edu/etdc/view?acc_num=ysu1578259840945109.
Full text
37
Patsanis, Alexandros. "Network Anomaly Detection and Root Cause Analysis with Deep Generative Models." Thesis, Uppsala universitet, Institutionen för informationsteknologi, 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-397367.
Full textAbstract:
The project's objective is to detect network anomalies happening in a telecommunication network due to hardware malfunction or software defects after a vast upgrade on the network's system over a specific area, such as a city. The network's system generates statistical data at a 15-minute interval for different locations in the area of interest. For every interval, all statistical data generated over an area are aggregated and converted to images. In this way, an image represents a snapshot of the network for a specific interval, where statistical data are represented as points having different density values. To that problem, this project makes use of Generative Adversarial Networks (GANs), which learn a manifold of the normal network pattern. Additionally, mapping from new unseen images to the learned manifold results in an anomaly score used to detect anomalies. The anomaly score is a combination of the reconstruction error and the learned feature representation. Two models for detecting anomalies are used in this project, AnoGAN and f-AnoGAN. Furthermore, f-AnoGAN uses a state-of-the-art approach called Wasstestein GAN with gradient penalty, which improves the initial implementation of GANs. Both quantitative and qualitative evaluation measurements are used to assess GANs models, where F1 Score and Wasserstein loss are used for the quantitative evaluation and linear interpolation in the hidden space for qualitative evaluation. Moreover, to set a threshold, a prediction model used to predict the expected behaviour of the network for a specific interval. Then, the predicted behaviour is used over the anomaly detection model to define a threshold automatically. Our experiments were implemented successfully for both prediction and anomaly detection models. We additionally tested known abnormal behaviours which were detected and visualised. However, more research has to be done over the evaluation of GANs, as there is no universal approach to evaluate them.
38
Taub, Lawrence. "Application of a Layered Hidden Markov Model in the Detection of Network Attacks." NSUWorks, 2013. http://nsuworks.nova.edu/gscis_etd/320.
Full textAbstract:
Network-based attacks against computer systems are a common and increasing problem. Attackers continue to increase the sophistication and complexity of their attacks with the goal of removing sensitive data or disrupting operations. Attack detection technology works very well for the detection of known attacks using a signature-based intrusion detection system. However, attackers can utilize attacks that are undetectable to those signature-based systems whether they are truly new attacks or modified versions of known attacks. Anomaly-based intrusion detection systems approach the problem of attack detection by detecting when traffic differs from a learned baseline. In the case of this research, the focus was on a relatively new area known as payload anomaly detection. In payload anomaly detection, the system focuses exclusively on the payload of packets and learns the normal contents of those payloads. When a payload's contents differ from the norm, an anomaly is detected and may be a potential attack. A risk with anomaly-based detection mechanisms is they suffer from high false positive rates which reduce their effectiveness. This research built upon previous research in payload anomaly detection by combining multiple techniques of detection in a layered approach. The layers of the system included a high-level navigation layer, a request payload analysis layer, and a request-response analysis layer. The system was tested using the test data provided by some earlier payload anomaly detection systems as well as new data sets. The results of the experiments showed that by combining these layers of detection into a single system, there were higher detection rates and lower false positive rates.
39
Garcia, Raymond Christopher. "A soft computing approach to anomaly detection with real-time applicability." Diss., Georgia Institute of Technology, 2001. http://hdl.handle.net/1853/21808.
Full text
40
Caulkins, Bruce. "SESSION-BASED INTRUSION DETECTION SYSTEM TO MAP ANOMALOUS NETWORK TRAFFIC." Doctoral diss., University of Central Florida, 2005. http://digital.library.ucf.edu/cdm/ref/collection/ETD/id/3466.
Full textAbstract:
Computer crime is a large problem (CSI, 2004; Kabay, 2001a; Kabay, 2001b). Security managers have a variety of tools at their disposal – firewalls, Intrusion Detection Systems (IDSs), encryption, authentication, and other hardware and software solutions to combat computer crime. Many IDS variants exist which allow security managers and engineers to identify attack network packets primarily through the use of signature detection; i.e., the IDS recognizes attack packets due to their well-known "fingerprints" or signatures as those packets cross the network's gateway threshold. On the other hand, anomaly-based ID systems determine what is normal traffic within a network and reports abnormal traffic behavior. This paper will describe a methodology towards developing a more-robust Intrusion Detection System through the use of data-mining techniques and anomaly detection. These data-mining techniques will dynamically model what a normal network should look like and reduce the false positive and false negative alarm rates in the process. We will use classification-tree techniques to accurately predict probable attack sessions. Overall, our goal is to model network traffic into network sessions and identify those network sessions that have a high-probability of being an attack and can be labeled as a "suspect session." Subsequently, we will use these techniques inclusive of signature detection methods, as they will be used in concert with known signatures and patterns in order to present a better model for detection and protection of networks and systems.Ph.D.
Other
Arts and Sciences
Modeling and Simulation
41
McGlohon, Mary. "Structural Analysis of Large Networks: Observations and Applications." Research Showcase @ CMU, 2010. http://repository.cmu.edu/dissertations/18.
Full textAbstract:
Network data (also referred to as relational data, social network data, real graph data) has become ubiquitous, and understanding patterns in this data has become an important research problem. We investigate how interactions in social networks are formed and how these interactions facilitate diffusion, model these behaviors, and apply these findings to real-world problems.
We examined graphs of size up to 16 million nodes, across many domains from academic citation networks, to campaign contributions and actor-movie networks. We also performed several case studies in online social networks such as blogs and message board communities.
Our major contributions are the following: (a) We discover several surprising patterns in network topology and interactions, such as Popularity Decay power law (in-links to a blog post decay with a power law with -1:5 exponent) and the oscillating size of connected components; (b) We propose generators such as the Butterfly generator that reproduce both established and new properties found in real networks; (c) several case studies, including a proposed method of detecting misstatements in accounting data, where using network effects gave a significant boost in detection accuracy.
42
Peacock, Matthew. "Anomaly Detection in BACnet/IP managed Building Automation Systems." Thesis, Edith Cowan University, Research Online, Perth, Western Australia, 2019. https://ro.ecu.edu.au/theses/2178.
Full textAbstract:
Building Automation Systems (BAS) are a collection of devices and software which manage the operation of building services. The BAS market is expected to be a $19.25 billion USD industry by 2023, as a core feature of both the Internet of Things and Smart City technologies. However, securing these systems from cyber security threats is an emerging research area. Since initial deployment, BAS have evolved from isolated standalone networks to heterogeneous, interconnected networks allowing external connectivity through the Internet. The most prominent BAS protocol is BACnet/IP, which is estimated to hold 54.6% of world market share. BACnet/IP security features are often not implemented in BAS deployments, leaving systems unprotected against known network threats. This research investigated methods of detecting anomalous network traffic in BACnet/IP managed BAS in an effort to combat threats posed to these systems.
This research explored the threats facing BACnet/IP devices, through analysis of Internet accessible BACnet devices, vendor-defined device specifications, investigation of the BACnet specification, and known network attacks identified in the surrounding literature. The collected data were used to construct a threat matrix, which was applied to models of BACnet devices to evaluate potential exposure. Further, two potential unknown vulnerabilities were identified and explored using state modelling and device simulation.
A simulation environment and attack framework were constructed to generate both normal and malicious network traffic to explore the application of machine learning algorithms to identify both known and unknown network anomalies. To identify network patterns between the generated normal and malicious network traffic, unsupervised clustering, graph analysis with an unsupervised community detection algorithm, and time series analysis were used. The explored methods identified distinguishable network patterns for frequency-based known network attacks when compared to normal network traffic. However, as stand-alone methods for anomaly detection, these methods were found insufficient. Subsequently, Artificial Neural Networks and Hidden Markov Models were explored and found capable of detecting known network attacks. Further, Hidden Markov Models were also capable of detecting unknown network attacks in the generated datasets.
The classification accuracy of the Hidden Markov Models was evaluated using the Matthews Correlation Coefficient which accounts for imbalanced class sizes and assess both positive and negative classification ability for deriving its metric. The Hidden Markov Models were found capable of repeatedly detecting both known and unknown BACnet/IP attacks with True Positive Rates greater than 0.99 and Matthews Correlation Coefficients greater than 0.8 for five of six evaluated hosts.
This research identified and evaluated a range of methods capable of identifying anomalies in simulated BACnet/IP network traffic. Further, this research found that Hidden Markov Models were accurate at classifying both known and unknown attacks in the evaluated BACnet/IP managed BAS network.
43
Satam, Pratik. "An Anomaly Behavior Analysis Intrusion Detection System for Wireless Networks." Thesis, The University of Arizona, 2015. http://hdl.handle.net/10150/595654.
Full textAbstract:
Wireless networks have become ubiquitous, where a wide range of mobile devices are connected to a larger network like the Internet via wireless communications. One widely used wireless communication standard is the IEEE 802.11 protocol, popularly called Wi-Fi. Over the years, the 802.11 has been upgraded to different versions. But most of these upgrades have been focused on the improvement of the throughput of the protocol and not enhancing the security of the protocol, thus leaving the protocol vulnerable to attacks. The goal of this research is to develop and implement an intrusion detection system based on anomaly behavior analysis that can detect accurately attacks on the Wi-Fi networks and track the location of the attacker. As a part of this thesis we present two architectures to develop an anomaly based intrusion detection system for single access point and distributed Wi-Fi networks. These architectures can detect attacks on Wi-Fi networks, classify the attacks and track the location of the attacker once the attack has been detected. The system uses statistical and probability techniques associated with temporal wireless protocol transitions, that we refer to as Wireless Flows (Wflows). The Wflows are modeled and stored as a sequence of n-grams within a given period of analysis. We studied two approaches to track the location of the attacker. In the first approach, we use a clustering approach to generate power maps that can be used to track the location of the user accessing the Wi-Fi network. In the second approach, we use classification algorithms to track the location of the user from a Central Controller Unit. Experimental results show that the attack detection and classification algorithms generate no false positives and no false negatives even when the Wi-Fi network has high frame drop rates. The Clustering approach for location tracking was found to perform highly accurate in static environments (81% accuracy) but the performance rapidly deteriorates with the changes in the environment. While the classification algorithm to track the location of the user at the Central Controller/RADIUS server was seen to perform with lesser accuracy then the clustering approach (76% accuracy) but the system's ability to track the location of the user deteriorated less rapidly with changes in the operating environment.
44
Wu, Xinheng. "A Deep Unsupervised Anomaly Detection Model for Automated Tumor Segmentation." Thesis, The University of Sydney, 2020. https://hdl.handle.net/2123/22502.
Full textAbstract:
Many researches have been investigated to provide the computer aided diagnosis (CAD) automated tumor segmentation in various medical images, e.g., magnetic resonance (MR), computed tomography (CT) and positron-emission tomography (PET). The recent advances in automated tumor segmentation have been achieved by supervised deep learning (DL) methods trained on large labelled data to cover tumor variations. However, there is a scarcity in such training data due to the cost of labeling process. Thus, with insufficient training data, supervised DL methods have difficulty in generating effective feature representations for tumor segmentation. This thesis aims to develop an unsupervised DL method to exploit large unlabeled data generated during clinical process. Our assumption is unsupervised anomaly detection (UAD) that, normal data have constrained anatomy and variations, while anomalies, i.e., tumors, usually differ from the normality with high diversity. We demonstrate our method for automated tumor segmentation on two different image modalities. Firstly, given that bilateral symmetry in normal human brains and unsymmetry in brain tumors, we propose a symmetric-driven deep UAD model using GAN model to model the normal symmetric variations thus segmenting tumors by their being unsymmetrical. We evaluated our method on two benchmarked datasets. Our results show that our method outperformed the state-of-the-art unsupervised brain tumor segmentation methods and achieved competitive performance to the supervised segmentation methods. Secondly, we propose a multi-modal deep UAD model for PET-CT tumor segmentation. We model a manifold of normal variations shared across normal CT and PET pairs; this manifold representing the normal pairing that can be used to segment the anomalies. We evaluated our method on two PET-CT datasets and the results show that we outperformed the state-of-the-art unsupervised methods, supervised methods and baseline fusion techniques.
45
Zhang, Hao. "Discovery of Triggering Relations and Its Applications in Network Security and Android Malware Detection." Diss., Virginia Tech, 2015. http://hdl.handle.net/10919/64246.
Full textAbstract:
An increasing variety of malware, including spyware, worms, and bots, threatens data confidentiality and system integrity on computing devices ranging from backend servers to mobile devices. To address these threats, exacerbated by dynamic network traffic patterns and growing volumes, network security has been undergoing major changes to improve accuracy and scalability in the security analysis techniques.
This dissertation addresses the problem of detecting the network anomalies on a single device by inferring the traffic dependence to ensure the root-triggers. In particular, we propose a dependence model for illustrating the network traffic causality. This model depicts the triggering relation of network requests, and thus can be used to reason about the occurrences of network events and pinpoint stealthy malware activities. The triggering relationships can be inferred by means of both rule-based and learning-based approaches. The rule-based approach originates from several heuristic algorithms based on the domain knowledge. The learning-based approach discovers the triggering relationship using a pairwise comparison operation that converts the requests into event pairs with comparable attributes. Machine learning classifiers predict the triggering relationship and further reason about the legitimacy of requests by enforcing their root-triggers. We apply our dependence model on the network traffic from a single host and a mobile device. Evaluated with real-world malware samples and synthetic attacks, our findings confirm that the traffic dependence model provides a significant source of semantic and contextual information that detects zero-day malicious applications.
This dissertation also studies the usability of visualizing the traffic causality for domain experts. We design and develop a tool with a visual locality property. It supports different levels of visual based querying and reasoning required for the sensemaking process on complex network data.
The significance of this dissertation research is in that it provides deep insights on the dependency of network requests, and leverages structural and semantic information, allowing us to reason about network behaviors and detect stealthy anomalies.Ph. D.
46
Casas, Hernandez Pedro. "Statistical analysis of network traffic for anomaly detection and quality of service provisioning." Télécom Bretagne, 2010. http://www.theses.fr/2010TELB0111.
Full textAbstract:
Traditionnellement, la gestion du trafic en cœur de réseau repose sur le surdimensionnement pour simplifier les opérations de gestion. Cependant, étant donnés la grande variabilité et l'hétérogénéité du trafic actuel, la montée en puissance d'applications qui nécessitent de la Qualité de Service, et le déploiement des technologies à très haut débit dans l'accès au réseau, il est nécessaire de développer des techniques d'ingénierie qui optimisent l'utilisation des ressources déployées. En particulier, il est nécessaire de concevoir une ingénierie de réseau qui s'appuie sur la mesure du trafic. La Matrice de Trafic (TM) donne une vision globale des volumes de trafic échangés sur un réseau. La tendance actuelle est d'estimer les TMs à partir des données remontées par les sondes NetFlow ou par ses avatars. Cependant, les mesures de trafic au niveau flot induisent une charge importante au niveau des routeurs. Par conséquent, les mesures sont sous-échantillonnées, ce qui induit une imprécision dans l'estimation de la TM. Dans nos travaux de thèse, nous avons proposé d'analyser la TM à partir de mesures des volumes de trafic agrégés échangés sur les différents liens du réseau. Cette approche réduit considérablement le coût engendré par la mesure et simplifie les questions d'implémentation. D'un point de vue statistique, le problème de l'estimation de la TM à partir de ces mesures est un problème linéaire inverse fortement mal pose. La première contribution concerne la modélisation et l'estimation de la TM. Nous avons proposé de nouveaux modèles statistiques et des nouvelles méthodes d'estimation instantanée et de poursuite pour analyser une TM à partir des mesures SNMP. La deuxième contribution considère la détection et la localisation d'anomalies volumétriques dans la TM. En utilisant un modèle linéaire parcimonieux de la TM, nous avons traité le problème de détection comme un problème invariant avec paramètres de nuisance. Nous nous sommes basés sur des algorithmes récents de théorie de la décision ayant des propriétés d'optimalité bien établies, contrairement à la plupart des techniques de la littérature qui se basent sur des heuristiques. La dernière contribution concerne l'optimisation de l'équilibrage de charge, dans le cas où la TM est variable et difficile à prévoir. En utilisant des techniques d'optimisation robuste, nous avons étudié différents scénarios en présence d'une demande de trafic fortement variable et incertaine. De plus, nous avons mené de manière critique une étude comparée des approches basées sur le routage robuste et des approches d'équilibrage dynamique basées sur les jeux de routage. Afin de démontrer la pertinence de nos contributions, toutes les méthodes proposées dans cette thèse ont été validées en utilisant des données réelles de trafic mesurées sur différents réseaux opérationnels. De plus, les performances des méthodes développées ont été comparées aux travaux bien connus de la littérature. Les résultats de ces comparaisons démontrent de bien meilleures performances dans la plupart des cas, et mettent également en évidence des défauts de conception de certains des algorithmes de la littératureNetwork-wide traffic analysis and monitoring in large-scale networks is a challenging and expensive task. In this thesis work we have proposed to analyze the traffic of a large-scale IP network from aggregated traffic measurements, reducing measurement overheads and simplifying implementation issues. We have provided contributions in three different networking fields related to network-wide traffic analysis and monitoring in large-scale IP networks. The first contribution regards Traffic Matrix (TM) modeling and estimation, where we have proposed new statistical models and new estimation methods to analyze the Origin-Destination (OD) flows of a large-scale TM from easily available link traffic measurements. The second contribution regards the detection and localization of volume anomalies in the TM, where we have introduced novel methods with solid optimality properties that outperform current well-known techniques for network-wide anomaly detection proposed so far in the literature. The last contribution regards the optimization of the routing configuration in large-scale IP networks, particularly when the traffic is highly variable and difficult to predict. Using the notions of Robust Routing Optimization we have proposed new approaches for Quality of Service provisioning under highly variable and uncertain traffic scenarios. In order to provide strong evidence on the relevance of our contributions, all the methods proposed in this thesis work were validated using real traffic data from different operational networks. Additionally, their performance was compared against well-known works in each field, showing outperforming results in most cases. Taking together the ensemble of developed TM models, the optimal network-wide anomaly detection and localization methods, and the routing optimization algorithms, this thesis work offers a complete solution for network operators to efficiently monitor large-scale IP networks from aggregated traffic measurements and to provide accurate QoS-based performance, even in the event of volume traffic anomalies
47
Edholm, Gustav. "Anomaly Detection and Revenue Loss Estimation in Accounting Data." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-291773.
Full textAbstract:
Loss of revenue due to erroneous invoicing is a serious problem for many companies in the repair and maintenance industry. Revenue loss can occur in many ways, for example by consistently charging the wrong hourly price for services. If a company is experiencing revenue loss, it is incredibly important to detect it, find where it is happening, and estimate the size of it in order to treat it. The goal of this work is to find statistical methods for detecting incorrectly charged services in a dataset of invoices, and estimate the loss of revenue in the same dataset. The dataset used comes from a real company experiencing revenue loss through incorrectly charged prices for services, and thus represents a real world instance of this problem. Multiple machine learning methods with different levels of supervision are tested for detecting anomalous invoice items and estimating revenue loss using raw invoice data. Neural network regression, and different decision tree regression methods, as well as an ensemble of these are tested and compared. The dataset has ground truth labels for each price, thus results are compared to real world targets. It is found that an ensemble using a weighted average of predictions from neural network regression and gradient boosted decision tree regression to predict the charged prices in an invoice dataset performs anomaly detection most reliably. On the top 1000 anomaly candidates, this method flags anomalies correctly 87% of the time, catching 45% of all anomalies. Moreover, in terms of estimating revenue loss, using a neural network to perform regression, a revenue loss error of just 13% is achieved.Förlorad omsättning till följd av felaktig fakturering ar ett alvarligt problem for vissa företag i service- och reparationsbranchen. Detta kan uppstå på manga satt, till exempel genom konsekvent felaktig prissättning av tjänster. Om ett företag har stor förlust av omsättning ar det otroligt viktigt att upptäcka det, hitta var det sker, och uppskatta storleken av förlusten for att kunna behandla den. Malet med detta arbete ar att hitta statistiska metoder for att identifiera felaktigt prissatta tjänster i ett dataset av fakturor, och uppskatta förlorad omsättning i datasetet. Datasetet som används kommer från ett företag som förlorar omsättning på grund av just felfakturerat pris på tjänster, och representerar därför en verklig instans av detta problem. Ett flertal maskininlärningsmetoder, med olika grader av vägledning, används for att upptäcka felaktiga fakturarader och uppskatta förlorad omsättning i omärkt fakturadata. Regression med neuronnät, och olika beslutstradmetoder såväl som en ensembel av dessa testas och jämförs. Datasetet har sanningsenliga ettiketter till varje rad, därmed kan resultaten jämföras och utvärderas mot korrekta priser. Vi finner att en ensembel av ett neuralnät och ett gradientförstärkt beslutstrad for regression identifierar felaktiga prissättningar mest pålitligt. Pa de 1000 mest sannolika felen har denna metod ratt på 87%, vilket fångar 45% av alla fel. Vidare, med hänsyn till förlorad omsättning finner vi att ett neuralnät som utför regresssion uppnår ett fel på endast 13% i sitt estimat av förlorad omsättning.
48
Al, Tobi Amjad Mohamed. "Anomaly-based network intrusion detection enhancement by prediction threshold adaptation of binary classification models." Thesis, University of St Andrews, 2018. http://hdl.handle.net/10023/17050.
Full textAbstract:
Network traffic exhibits a high level of variability over short periods of time. This variability impacts negatively on the performance (accuracy) of anomaly-based network Intrusion Detection Systems (IDS) that are built using predictive models in a batch-learning setup. This thesis investigates how adapting the discriminating threshold of model predictions, specifically to the evaluated traffic, improves the detection rates of these Intrusion Detection models. Specifically, this thesis studied the adaptability features of three well known Machine Learning algorithms: C5.0, Random Forest, and Support Vector Machine. The ability of these algorithms to adapt their prediction thresholds was assessed and analysed under different scenarios that simulated real world settings using the prospective sampling approach. A new dataset (STA2018) was generated for this thesis and used for the analysis. This thesis has demonstrated empirically the importance of threshold adaptation in improving the accuracy of detection models when training and evaluation (test) traffic have different statistical properties. Further investigation was undertaken to analyse the effects of feature selection and data balancing processes on a model's accuracy when evaluation traffic with different significant features were used. The effects of threshold adaptation on reducing the accuracy degradation of these models was statistically analysed. The results showed that, of the three compared algorithms, Random Forest was the most adaptable and had the highest detection rates. This thesis then extended the analysis to apply threshold adaptation on sampled traffic subsets, by using different sample sizes, sampling strategies and label error rates. This investigation showed the robustness of the Random Forest algorithm in identifying the best threshold. The Random Forest algorithm only needed a sample that was 0.05% of the original evaluation traffic to identify a discriminating threshold with an overall accuracy rate of nearly 90% of the optimal threshold.
49
Wang, Qinghua. "Traffic analysis, modeling and their applications in energy-constrained wireless sensor networks on network optimization and anomaly detection /." Doctoral thesis, Sundsvall : Tryckeriet Mittuniversitetet, 2010. http://urn.kb.se/resolve?urn=urn:nbn:se:miun:diva-10690.
Full text
50
Kenar, Serkan. "An Extensible Framework For Automated Network Attack Signature Generation." Master's thesis, METU, 2010. http://etd.lib.metu.edu.tr/upload/2/12611418/index.pdf.
Full textAbstract:
The effectiveness of misuse-based intrusion detection systems (IDS) are seriously broken, with the advance of threats in terms of speed and scale. Today worms, trojans, viruses and other threats can spread all around the globe in less than thirty minutes. In order to detect these emerging threats, signatures must be generated automatically and distributed to intrusion detection systems rapidly. There are studies on automatically generating signatures for worms
and attacks. However, either these systems rely on Honeypots which are supposed to receive only suspicious traffic, or use port-scanning outlier detectors. In this study, an open, extensible system based on an network IDS is proposed to identify suspicious traffic using anomaly
detection methods, and to automatically generate signatures of attacks out of this suspicious traffic. The generated signatures are classified and fedback into the IDS either locally or distributed. Design and proof-of-concept implementation are described and developed system
is tested on both synthetic and real network data. The system is designed as a framework to test different methods and evaluate the outcomes of varying configurations easily.
The test results show that, with a properly defined attack detection algorithm, attack signatures could be generated with high accuracy and efficiency. The resulting system could be used to prevent early damages of fast-spreading worms and other threats.