To see the other types of publications on this topic, follow the link: Neural networks with LSTM.
Dissertations / Theses on the topic 'Neural networks with LSTM'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 dissertations / theses for your research on the topic 'Neural networks with LSTM.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse dissertations / theses on a wide variety of disciplines and organise your bibliography correctly.
1
Paschou, Michail. "ASIC implementation of LSTM neural network algorithm." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-254290.
Full textAbstract:
LSTM neural networks have been used for speech recognition, image recognition and other artificial intelligence applications for many years. Most applications perform the LSTM algorithm and the required calculations on cloud computers. Off-line solutions include the use of FPGAs and GPUs but the most promising solutions include ASIC accelerators designed for this purpose only. This report presents an ASIC design capable of performing the multiple iterations of the LSTM algorithm on a unidirectional and without peepholes neural network architecture. The proposed design provides arithmetic level parallelism options as blocks are instantiated based on parameters. The internal structure of the design implements pipelined, parallel or serial solutions depending on which is optimal in every case. The implications concerning these decisions are discussed in detail in the report. The design process is described in detail and the evaluation of the design is also presented to measure accuracy and error of the design output.This thesis work resulted in a complete synthesizable ASIC design implementing an LSTM layer, a Fully Connected layer and a Softmax layer which can perform classification of data based on trained weight matrices and bias vectors. The design primarily uses 16-bit fixed point format with 5 integer and 11 fractional bits but increased precision representations are used in some blocks to reduce error output. Additionally, a verification environment has also been designed and is capable of performing simulations, evaluating the design output by comparing it with results produced from performing the same operations with 64-bit floating point precision on a SystemVerilog testbench and measuring the encountered error. The results concerning the accuracy and the design output error margin are presented in this thesis report. The design went through Logic and Physical synthesis and successfully resulted in a functional netlist for every tested configuration. Timing, area and power measurements on the generated netlists of various configurations of the design show consistency and are reported in this report.<br>LSTM neurala nätverk har använts för taligenkänning, bildigenkänning och andra artificiella intelligensapplikationer i många år. De flesta applikationer utför LSTM-algoritmen och de nödvändiga beräkningarna i digitala moln. Offline lösningar inkluderar användningen av FPGA och GPU men de mest lovande lösningarna inkluderar ASIC-acceleratorer utformade för endast dettaändamål. Denna rapport presenterar en ASIC-design som kan utföra multipla iterationer av LSTM-algoritmen på en enkelriktad neural nätverksarkitetur utan peepholes. Den föreslagna designed ger aritmetrisk nivå-parallellismalternativ som block som är instansierat baserat på parametrar. Designens inre konstruktion implementerar pipelinerade, parallella, eller seriella lösningar beroende på vilket anternativ som är optimalt till alla fall. Konsekvenserna för dessa beslut diskuteras i detalj i rapporten. Designprocessen beskrivs i detalj och utvärderingen av designen presenteras också för att mäta noggrannheten och felmarginal i designutgången. Resultatet av arbetet från denna rapport är en fullständig syntetiserbar ASIC design som har implementerat ett LSTM-lager, ett fullständigt anslutet lager och ett Softmax-lager som kan utföra klassificering av data baserat på tränade viktmatriser och biasvektorer. Designen använder huvudsakligen 16bitars fast flytpunktsformat med 5 heltal och 11 fraktions bitar men ökade precisionsrepresentationer används i vissa block för att minska felmarginal. Till detta har även en verifieringsmiljö utformats som kan utföra simuleringar, utvärdera designresultatet genom att jämföra det med resultatet som produceras från att utföra samma operationer med 64-bitars flytpunktsprecision på en SystemVerilog testbänk och mäta uppstådda felmarginal. Resultaten avseende noggrannheten och designutgångens felmarginal presenteras i denna rapport.Designen gick genom Logisk och Fysisk syntes och framgångsrikt resulterade i en funktionell nätlista för varje testad konfiguration. Timing, area och effektmätningar på den genererade nätlistorna av olika konfigurationer av designen visar konsistens och rapporteras i denna rapport.
2
Cavallie, Mester Jon William. "Using LSTM Neural Networks To Predict Daily Stock Returns." Thesis, Linnéuniversitetet, Institutionen för datavetenskap och medieteknik (DM), 2021. http://urn.kb.se/resolve?urn=urn:nbn:se:lnu:diva-106124.
Full textAbstract:
Long short-term memory (LSTM) neural networks have been proven to be effective for time series prediction, even in some instances where the data is non-stationary. This lead us to examine their predictive ability of stock market returns, as the development of stock prices and returns tend to be a non-stationary time series. We used daily stock trading data to let an LSTM train models at predicting daily returns for 60 stocks from the OMX30 and Nasdaq-100 indices. Subsequently, we measured their accuracy, precision, and recall. The mean accuracy was 49.75 percent, meaning that the observed accuracy was close to the accuracy one would observe by randomly selecting a prediction for each day and lower than the accuracy achieved by blindly predicting all days to be positive. Finally, we concluded that further improvements need to be made for models trained by LSTMs to have any notable predictive ability in the area of stock returns.
3
Ärlemalm, Filip. "Harbour Porpoise Click Train Classification with LSTM Recurrent Neural Networks." Thesis, KTH, Teknisk informationsvetenskap, 2017. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-215088.
Full textAbstract:
The harbour porpoise is a toothed whale whose presence is threatened in Scandinavia. Onestep towards preserving the species in critical areas is to study and observe the harbourporpoise population growth or decline in these areas. Today this is done by using underwateraudio recorders, so called hydrophones, and manual analyzing tools. This report describes amethod that modernizes the process of harbour porpoise detection with machine learning. Thedetection method is based on data collected by the hydrophone AQUAclick 100. The data isprocessed and classified automatically with a stacked long short-term memory recurrent neuralnetwork designed specifically for this purpose.<br>Vanlig tumlare är en tandval vars närvaro i Skandinavien är hotad. Ett steg mot att kunnabevara arten i utsatta områden är att studera och observera tumlarbeståndets tillväxt ellertillbakagång i dessa områden. Detta görs idag med hjälp av ljudinspelare för undervattensbruk,så kallade hydrofoner, samt manuella analysverktyg. Den här rapporten beskriver enmetod som moderniserar processen för detektering av vanlig tumlare genom maskininlärning.Detekteringen är baserad på insamlad data från hydrofonen AQUAclick 100. Bearbetning ochklassificering av data har automatiserats genom att använda ett staplat återkopplande neuraltnätverk med långt korttidsminne utarbetat specifikt för detta ändamål.
4
Li, Edwin. "LSTM Neural Network Models for Market Movement Prediction." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-231627.
Full textAbstract:
Interpreting time varying phenomena is a key challenge in the capital markets. Time series analysis using autoregressive methods has been carried out over the last couple of decades, often with reassuring results. However, such methods sometimes fail to explain trends and cyclical fluctuations, which may be characterized by long-range dependencies or even dependencies between the input features. The purpose of this thesis is to investigate whether recurrent neural networks with LSTM-cells can be used to capture these dependencies, and ultimately be used as a complement for index trading decisions. Experiments are made on different setups of the S&P-500 stock index, and two distinct models are built, each one being an improvement of the previous model. The first model is a multivariate regression model, and the second model is a multivariate binary classifier. The output of each model is used to reason about the future behavior of the index. The experiment shows for the configuration provided that LSTM RNNs are unsuitable for predicting exact values of daily returns, but gives satisfactory results when used to predict the direction of the movement.<br>Att förstå och kunna förutsäga hur index varierar med tiden och andra parametrar är ett viktigt problem inom kapitalmarknader. Tidsserieanalys med autoregressiva metoder har funnits sedan årtionden tillbaka, och har oftast gett goda resultat. Dessa metoder saknar dock möjligheten att förklara trender och cykliska variationer i tidsserien, något som kan karaktäriseras av tidsvarierande samband, men även samband mellan parametrar som indexet beror utav. Syftet med denna studie är att undersöka om recurrent neural networks (RNN) med long short-term memory-celler (LSTM) kan användas för att fånga dessa samband, för att slutligen användas som en modell för att komplettera indexhandel. Experimenten är gjorda mot en modifierad S&P-500 datamängd, och två distinkta modeller har tagits fram. Den ena är en multivariat regressionsmodell för att förutspå exakta värden, och den andra modellen är en multivariat klassifierare som förutspår riktningen på nästa dags indexrörelse. Experimenten visar för den konfiguration som presenteras i rapporten att LSTM RNN inte passar för att förutspå exakta värden för indexet, men ger tillfredsställande resultat när modellen ska förutsäga indexets framtida riktning.
5
Holm, Noah, and Emil Plynning. "Spatio-temporal prediction of residential burglaries using convolutional LSTM neural networks." Thesis, KTH, Geoinformatik, 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-229952.
Full textAbstract:
The low amount solved residential burglary crimes calls for new and innovative methods in the prevention and investigation of the cases. There were 22 600 reported residential burglaries in Sweden 2017 but only four to five percent of these will ever be solved. There are many initiatives in both Sweden and abroad for decreasing the amount of occurring residential burglaries and one of the areas that are being tested is the use of prediction methods for more efficient preventive actions. This thesis is an investigation of a potential method of prediction by using neural networks to identify areas that have a higher risk of burglaries on a daily basis. The model use reported burglaries to learn patterns in both space and time. The rationale for the existence of patterns is based on near repeat theories in criminology which states that after a burglary both the burgled victim and an area around that victim has an increased risk of additional burglaries. The work has been conducted in cooperation with the Swedish Police authority. The machine learning is implemented with convolutional long short-term memory (LSTM) neural networks with max pooling in three dimensions that learn from ten years of residential burglary data (2007-2016) in a study area in Stockholm, Sweden. The model's accuracy is measured by performing predictions of burglaries during 2017 on a daily basis. It classifies cells in a 36x36 grid with 600 meter square grid cells as areas with elevated risk or not. By classifying 4% of all grid cells during the year as risk areas, 43% of all burglaries are correctly predicted. The performance of the model could potentially be improved by further configuration of the parameters of the neural network, along with a use of more data with factors that are correlated to burglaries, for instance weather. Consequently, further work in these areas could increase the accuracy. The conclusion is that neural networks or machine learning in general could be a powerful and innovative tool for the Swedish Police authority to predict and moreover prevent certain crime. This thesis serves as a first prototype of how such a system could be implemented and used.
6
Zambezi, Samantha. "Predicting social unrest events in South Africa using LSTM neural networks." Master's thesis, Faculty of Science, 2021. http://hdl.handle.net/11427/33986.
Full textAbstract:
This thesis demonstrates an approach to predict the count of social unrest events in South Africa. A comparison is made between traditional forecasting approaches and neural networks; the traditional forecast method selected being the Autoregressive Integrated Moving Average (ARIMA model). The type of neural network implemented was the Long Short-Term Memory (LSTM) neural network. The basic theoretical concepts of ARIMA and LSTM neural networks are explained and subsequently, the patterns of the social unrest time series were analysed using time series exploratory techniques. The social unrest time series contained a significant number of irregular fluctuations with a non-linear trend. The structure of the social unrest time series suggested that traditional linear approaches would fail to model the non-linear behaviour of the time series. This thesis confirms this finding. Twelve experiments were conducted, and in these experiments, features, scaling procedures and model configurations are varied (i.e. univariate and multivariate models). Multivariate LSTM achieved the lowest forecast errors and performance improved as more explanatory features were introduced. The ARIMA model's performance deteriorated with added complexity and the univariate ARIMA produced lower forecast errors compared to the multivariate ARIMA. In conclusion, it can be claimed that multivariate LSTM neural networks are useful for predicting social unrest events.
7
Graffi, Giacomo. "A novel approach for Credit Scoring using Deep Neural Networks with bank transaction data." Master's thesis, Alma Mater Studiorum - Università di Bologna, 2021.
Find full textAbstract:
With the PSD2 open banking revolution FinTechs obtained a key role in the financial industry. This role implies the inquiry and development of new techniques, products and solutions to compete with other players in this area.
The aim of this thesis is to investigate the applicability of the state-of-the-art Deep Learning techniques for Credit Risk Modeling. In order to accomplish it, a PSD2-related synthetic and anonymized dataset has been used to simulate an application process with only one account per user. Firstly, a machine-readable representation of the bank accounts has been created, starting from the raw transactions’ data and scaling the variables using the quantile function.
Afterwards, a Deep Neural Network has been created in order to capture the complex relations between the input variables and to extract information from the accounts’ representations. The proposed architecture accomplished the assigned tasks with a Gini index of 0.55, exploiting a Convolutional encoder to extract features from the inputs and a Recurrent decoder to analyze them.
8
Lin, Alvin. "Video Based Automatic Speech Recognition Using Neural Networks." DigitalCommons@CalPoly, 2020. https://digitalcommons.calpoly.edu/theses/2343.
Full textAbstract:
Neural network approaches have become popular in the field of automatic speech recognition (ASR). Most ASR methods use audio data to classify words. Lip reading ASR techniques utilize only video data, which compensates for noisy environments where audio may be compromised. A comprehensive approach, including the vetting of datasets and development of a preprocessing chain, to video-based ASR is developed. This approach will be based on neural networks, namely 3D convolutional neural networks (3D-CNN) and Long short-term memory (LSTM). These types of neural networks are designed to take in temporal data such as videos. Various combinations of different neural network architecture and preprocessing techniques are explored. The best performing neural network architecture, a CNN with bidirectional LSTM, compares favorably against recent works on video-based ASR.
9
Alam, Samiul. "Recurrent neural networks in electricity load forecasting." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-233254.
Full textAbstract:
In this thesis two main studies are conducted to compare the predictive capabilities of feed-forward neural networks (FFNN) and long short-term memory networks (LSTM) in electricity load forecasting. The first study compares univariate networks using past electricity load, as well as multivariate networks using past electricity load and air temperature, in day-ahead load forecasting using varying lookback periods and sparsity of past observations. The second study compares FFNNs and LSTMs of different complexities (i.e. network sizes) when restrictions imposed by limitations of the real world are taken into consideration. No significant differences are found between the predictive performances of the two neural network approaches. However, adding air temperature as extra input to the LSTM is found to significantly decrease its performance. Furthermore, the predictive performance of the FFNN is found to significantly decrease as the network complexity grows, while the predictive performance of the LSTM is found to increase as the network complexity grows. All the findings considered, we do not find that there is enough evidence in favour of the LSTM in electricity load forecasting.<br>I denna uppsats beskrivs två studier som jämför feed-forward neurala nätverk (FFNN) och long short-term memory neurala nätverk (LSTM) i prognostisering av elkonsumtion. I den första studien undersöks univariata modeller som använder tidigare elkonsumtion, och flervariata modeller som använder tidigare elkonsumtion och temperaturmätningar, för att göra prognoser av elkonsumtion för nästa dag. Hur långt bak i tiden tidigare information hämtas ifrån samt upplösningen av tidigare information varieras. I den andra studien undersöks FFNN- och LSTM-modeller med praktiska begränsningar såsom tillgänglighet av data i åtanke. Även storleken av nätverken varieras. I studierna finnes ingen skillnad mellan FFNN- och LSTM-modellernas förmåga att prognostisera elkonsumtion. Däremot minskar FFNN-modellens förmåga att prognostisera elkonsumtion då storleken av modellen ökar. Å andra sidan ökar LSTM-modellens förmåga då storkelen ökar. Utifrån dessa resultat anser vi inte att det finns tillräckligt med bevis till förmån för LSTM-modeller i prognostisering av elkonsumtion.
10
Roxbo, Daniel. "A Detailed Analysis of Semantic Dependency Parsing with Deep Neural Networks." Thesis, Linköpings universitet, Interaktiva och kognitiva system, 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-156831.
Full textAbstract:
The use of Long Short Term Memory (LSTM) networks continues to yield better results in natural language processing tasks. One area which recently has seen significant improvements is semantic dependency parsing, where the current state-of-the-art model uses a multilayer LSTM combined with an attention-based scoring function to predict the dependencies. In this thesis the state of the art model is first replicated and then extended to include features based on syntactical trees, which was found to be useful in a similar model. In addition, the effect of part-of-speech tags is studied. The replicated model achieves a labeled F1 score of 93.6 on the in-domain data and 89.2 on the out-of-domain data on the DM dataset, which shows that the model is indeed replicable. Using multiple features extracted from syntactic gold standard trees of the DELPH-IN Derivation Tree (DT) type increased the labeled scores to 97.1 and 94.1 respectively, while the use of predicted trees of the Stanford Basic (SB) type did not improve the results at all. The usefulness of part-of-speech tags was found to be diminished in the presence of other features.
11
Martins, Helder. "Predicting user churn on streaming services using recurrent neural networks." Thesis, KTH, Skolan för datavetenskap och kommunikation (CSC), 2017. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-217109.
Full textAbstract:
Providers of online services have witnessed a rapid growth of their user base in the last few years. The phenomenon has attracted an increasing number of competitors determined on obtaining their own share of the market. In this context, the cost of attracting new customers has increased significantly, raising the importance of retaining existing clients. Therefore, it has become progressively more important for the companies to improve user experience and ensure they keep a larger share of their users active in consuming their product. Companies are thus compelled to build tools that can identify what prompts customers to stay and also identify the users intent on abandoning the service. The focus of this thesis is to address the problem of predicting user abandonment, also known as "churn", and also detecting motives for user retention on data provided by an online streaming service. Classical models like logistic regression and random forests have been used to predict the churn probability of a customer with a fair amount of precision in the past, commonly by aggregating all known information about a user over a time period into a unique data point. On the other hand, recurrent neural networks, especially the long short-term memory (LSTM) variant, have shown impressive results for other domains like speech recognition and video classification, where the data is treated as a sequence instead. This thesis investigates how LSTM models perform for the task of predicting churn compared to standard nonsequential baseline methods when applied to user behavior data of a music streaming service. It was also explored how different aspects of the data, like the distribution between the churning and retaining classes, the size of user event history and feature representation influences the performance of predictive models. The obtained results show that LSTMs has a comparable performance to random forest for churn detection, while being significantly better than logistic regression. Additionally, a framework for creating a dataset suitable for training predictive models is provided, which can be further explored as to analyze user behavior and to create retention actions that minimize customer abandonment.<br>Leverantörer av onlinetjänster har bevittnat en snabb användartillväxt under de senaste åren. Denna trend har lockat ett ökande antal konkurrenter som vill ta del av denna växande marknad. Detta har resulterat i att kostnaden för att locka nya kunder ökat avsevärt, vilket även ökat vikten av att behålla befintliga kunder. Det har därför gradvis blivit viktigare för företag att förbättra användarupplevelsen och se till att de behåller en större andel avanvändarna aktiva. Företag har därför ett starkt intresse avatt bygga verktyg som kan identifiera vad som driver kunder att stanna eller vad som får dem lämna. Detta arbete fokuserar därför på hur man kan prediktera att en användare är på väg att överge en tjänst, så kallad “churn”, samt identifiera vad som driver detta baserat på data från en onlinetjänst. Klassiska modeller som logistisk regression och random forests har tidigare använts på aggregerad användarinformation över en given tidsperiod för att med relativt god precision prediktera sannolikheten för att en användare kommer överge produkten. Under de senaste åren har dock sekventiella neurala nätverk (särskilt LSTM-varianten Long Short Term Memory), där data istället behandlas som sekvenser, visat imponerande resultat för andra domäner såsom taligenkänning och videoklassificering. Detta arbete undersöker hur väl LSTM-modeller kan användas för att prediktera churn jämfört med traditionella icke-sekventiella metoder när de tillämpas på data över användarbeteende från en musikstreamingtjänst. Arbetet undersöker även hur olika aspekter av data påverkar prestandan av modellerna inklusive distributionen mellan gruppen av användare som överger produkten mot de som stannar, längden av användarhändelseshistorik och olika val av användarfunktioner för modeller och användardatan. De erhållna resultaten visar att LSTM har en jämförbar prestanda med random forest för prediktering av användarchurn samt är signifikant bättre än logistisk regression. LSTMs visar sig således vara ett lämpligt val för att förutsäga churn på användarnivå. Utöver dessa resultat utvecklades även ett ramverk för att skapa dataset som är lämpliga för träning av prediktiva modeller, vilket kan utforskas ytterligare för att analysera användarbeteende och för att skapa förbättrade åtgärder för att behålla användare och minimera antalet kunder som överger tjänsten.
12
Näslund, Per. "Artificial Neural Networks in Swedish Speech Synthesis." Thesis, KTH, Tal-kommunikation, 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-239350.
Full textAbstract:
Text-to-speech (TTS) systems have entered our daily lives in the form of smart assistants and many other applications. Contemporary re- search applies machine learning and artificial neural networks (ANNs) to synthesize speech. It has been shown that these systems outperform the older concatenative and parametric methods. In this paper, ANN-based methods for speech synthesis are ex- plored and one of the methods is implemented for the Swedish lan- guage. The implemented method is dubbed “Tacotron” and is a first step towards end-to-end ANN-based TTS which puts many differ- ent ANN-techniques to work. The resulting system is compared to a parametric TTS through a strength-of-preference test that is carried out with 20 Swedish speaking subjects. A statistically significant pref- erence for the ANN-based TTS is found. Test subjects indicate that the ANN-based TTS performs better than the parametric TTS when it comes to audio quality and naturalness but sometimes lacks in intelli- gibility.<br>Talsynteser, också kallat TTS (text-to-speech) används i stor utsträckning inom smarta assistenter och många andra applikationer. Samtida forskning applicerar maskininlärning och artificiella neurala nätverk (ANN) för att utföra talsyntes. Det har visats i studier att dessa system presterar bättre än de äldre konkatenativa och parametriska metoderna. I den här rapporten utforskas ANN-baserade TTS-metoder och en av metoderna implementeras för det svenska språket. Den använda metoden kallas “Tacotron” och är ett första steg mot end-to-end TTS baserat på neurala nätverk. Metoden binder samman flertalet olika ANN-tekniker. Det resulterande systemet jämförs med en parametriskt TTS genom ett graderat preferens-test som innefattar 20 svensktalande försökspersoner. En statistiskt säkerställd preferens för det ANN- baserade TTS-systemet fastställs. Försökspersonerna indikerar att det ANN-baserade TTS-systemet presterar bättre än det parametriska när det kommer till ljudkvalitet och naturlighet men visar brister inom tydlighet.
13
Lagerhjelm, Linus. "Extracting Information from Encrypted Data using Deep Neural Networks." Thesis, Umeå universitet, Institutionen för tillämpad fysik och elektronik, 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:umu:diva-155904.
Full textAbstract:
In this paper we explore various approaches to using deep neural networks to per- form cryptanalysis, with the ultimate goal of having a deep neural network deci- pher encrypted data. We use long short-term memory networks to try to decipher encrypted text and we use a convolutional neural network to perform classification tasks on encrypted MNIST images. We find that although the network is unable to decipher encrypted data, it is able to perform classification on encrypted data. We also find that the networks performance is depending on what key were used to en- crypt the data. These findings could be valuable for further research into the topic of cryptanalysis using deep neural networks.
14
Valluru, Aravind-Deshikh. "Realization of LSTM Based Cognitive Radio Network." Thesis, University of North Texas, 2019. https://digital.library.unt.edu/ark:/67531/metadc1538697/.
Full textAbstract:
This thesis presents the realization of an intelligent cognitive radio network that uses long short term memory (LSTM) neural network for sensing and predicting the spectrum activity at each instant of time. The simulation is done using Python and GNU Radio. The implementation is done using GNU Radio and Universal Software Radio Peripherals (USRP). Simulation results show that the confidence factor of opportunistic users not causing interference to licensed users of the spectrum is 98.75%. The implementation results demonstrate high reliability of the LSTM based cognitive radio network.
15
Larsson, Joel. "Optimizing text-independent speaker recognition using an LSTM neural network." Thesis, Mälardalens högskola, Akademin för innovation, design och teknik, 2014. http://urn.kb.se/resolve?urn=urn:nbn:se:mdh:diva-26312.
Full textAbstract:
In this paper a novel speaker recognition system is introduced. Automated speaker recognition has become increasingly popular to aid in crime investigations and authorization processes with the advances in computer science. Here, a recurrent neural network approach is used to learn to identify ten speakers within a set of 21 audio books. Audio signals are processed via spectral analysis into Mel Frequency Cepstral Coefficients that serve as speaker specific features, which are input to the neural network. The Long Short-Term Memory algorithm is examined for the first time within this area, with interesting results. Experiments are made as to find the optimum network model for the problem. These show that the network learns to identify the speakers well, text-independently, when the recording situation is the same. However the system has problems to recognize speakers from different recordings, which is probably due to noise sensitivity of the speech processing algorithm in use.
16
Lousseief, Elias. "MahlerNet : Unbounded Orchestral Music with Neural Networks." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-264993.
Full textAbstract:
Modelling music with mathematical and statistical methods in general, and with neural networks in particular, has a long history and has been well explored in the last decades. Exactly when the first attempt at strictly systematic music took place is hard to say; some would say in the days of Mozart, others would say even earlier, but it is safe to say that the field of algorithmic composition has a long history. Even though composers have always had structure and rules as part of the writing process, implicitly or explicitly, following rules at a stricter level was well investigated in the middle of the 20th century at which point also the first music writing computer program based on mathematics was implemented. This work in computer science focuses on the history of musical composition with computers, also known as algorithmic composition, using machine learning and neural networks and consists of two parts: a literature survey covering in-depth the last decades in the field from which is drawn inspiration and experience to construct MahlerNet, a neural network based on the previous architectures MusicVAE, BALSTM, PerformanceRNN and BachProp, capable of modelling polyphonic symbolic music with up to 23 instruments. MahlerNet is a new architecture that uses a custom preprocessor with musical heuristics to normalize and filter the input and output files in MIDI format into a data representation that it uses for processing. MahlerNet, and its preprocessor, was written altogether for this project and produces music that clearly shows musical characteristics reminiscent of the data it was trained on, with some long-term structure, albeit not in the form of motives and themes.<br>Matematik och statistik i allmänhet, och maskininlärning och neurala nätverk i synnerhet, har sedan långt tillbaka använts för att modellera musik med en utveckling som kulminerat under de senaste decennierna. Exakt vid vilken historisk tidpunkt som musikalisk komposition för första gången tillämpades med strikt systematiska regler är svårt att säga; vissa skulle hävda att det skedde under Mozarts dagar, andra att det skedde redan långt tidigare. Oavsett vilket, innebär det att systematisk komposition är en företeelse med lång historia. Även om kompositörer i alla tider följt strukturer och regler, medvetet eller ej, som en del av kompositionsprocessen började man under 1900-talets mitt att göra detta i högre utsträckning och det var också då som de första programmen för musikalisk komposition, baserade på matematik, kom till. Den här uppsatsen i datateknik behandlar hur musik historiskt har komponerats med hjälp av datorer, ett område som också är känt som algoritmisk komposition. Uppsatsens fokus ligger på användning av maskininlärning och neurala nätverk och består av två delar: en litteraturstudie som i hög detalj behandlar utvecklingen under de senaste decennierna från vilken tas inspiration och erfarenheter för att konstruera MahlerNet, ett neuralt nätverk baserat på de tidigare modellerna MusicVAE, BALSTM, PerformanceRNN och BachProp. MahlerNet kan modellera polyfon musik med upp till 23 instrument och är en ny arkitektur som kommer tillsammans med en egen preprocessor som använder heuristiker från musikteori för att normalisera och filtrera data i MIDI-format till en intern representation. MahlerNet, och dess preprocessor, är helt och hållet implementerade för detta arbete och kan komponera musik som tydligt uppvisar egenskaper från den musik som nätverket tränats på. En viss kontinuitet finns i den skapade musiken även om det inte är i form av konkreta teman och motiv.
17
Poormehdi, Ghaemmaghami Masoumeh. "Tracking of Humans in Video Stream Using LSTM Recurrent Neural Network." Thesis, KTH, Teoretisk datalogi, TCS, 2017. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-217495.
Full textAbstract:
In this master thesis, the problem of tracking humans in video streams by using Deep Learning is examined. We use spatially supervised recurrent convolutional neural networks for visual human tracking. In this method, the recurrent convolutional network uses both the history of locations and the visual features from the deep neural networks. This method is used for tracking, based on the detection results. We concatenate the location of detected bounding boxes with high-level visual features produced by convolutional networks and then predict the tracking bounding box for next frames. Because a video contain continuous frames, we decide to have a method which uses the information from history of frames to have a robust tracking in different visually challenging cases such as occlusion, motion blur, fast movement, etc. Long Short-Term Memory (LSTM) is a kind of recurrent convolutional neural network and useful for our purpose. Instead of using binary classification which is commonly used in deep learning based tracking methods, we use a regression for direct prediction of the tracking locations. Our purpose is to test our method on real videos which is recorded by head-mounted camera. So our test videos are very challenging and contain different cases of fast movements, motion blur, occlusions, etc. Considering the limitation of the training data-set which is spatially imbalanced, we have a problem for tracking the humans who are in the corners of the image but in other challenging cases, the proposed tracking method worked well.<br>I detta examensarbete undersöks problemet att spåra människor i videoströmmar genom att använda deep learning. Spårningen utförs genom att använda ett recurrent convolutional neural network. Input till nätverket består av visuella features extraherade med hjälp av ett convolutional neural network, samt av detektionsresultat från tidigare frames. Vi väljer att använda oss av historiska detektioner för att skapa en metod som är robust mot olika utmanande situationer, som t.ex. snabba rörelser, rörelseoskärpa och ocklusion. Long Short- Term Memory (LSTM) är ett recurrent convolutional neural network som är användbart för detta ändamål. Istället för att använda binära klassificering, vilket är vanligt i många deep learning-baserade tracking-metoder, så använder vi oss av regression för att direkt förutse positionen av de spårade subjekten. Vårt syfte är att testa vår metod på videor som spelats in med hjälp av en huvudmonterad kamera. På grund av begränsningar i våra träningsdataset som är spatiellt oblanserade har vi problem att spåra människor som befinner sig i utkanten av bildområdet, men i andra utmanande fall lyckades spårningen bra.
18
Fors, Johansson Christoffer. "Arrival Time Predictions for Buses using Recurrent Neural Networks." Thesis, Linköpings universitet, Artificiell intelligens och integrerade datorsystem, 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-165133.
Full textAbstract:
In this thesis, two different types of bus passengers are identified. These two types, namely current passengers and passengers-to-be have different needs in terms of arrival time predictions. A set of machine learning models based on recurrent neural networks and long short-term memory units were developed to meet these needs. Furthermore, bus data from the public transport in Östergötland county, Sweden, were collected and used for training new machine learning models. These new models are compared with the current prediction system that is used today to provide passengers with arrival time information. The models proposed in this thesis uses a sequence of time steps as input and the observed arrival time as output. Each input time step contains information about the current state such as the time of arrival, the departure time from thevery first stop and the current position in Cartesian coordinates. The targeted value for each input is the arrival time at the next time step. To predict the rest of the trip, the prediction for the next step is simply used as input in the next time step. The result shows that the proposed models can improve the mean absolute error per stop between 7.2% to 40.9% compared to the system used today on all eight routes tested. Furthermore, the choice of loss function introduces models thatcan meet the identified passengers need by trading average prediction accuracy for a certainty that predictions do not overestimate or underestimate the target time in approximately 95% of the cases.
19
Bonato, Tommaso. "Time Series Predictions With Recurrent Neural Networks." Bachelor's thesis, Alma Mater Studiorum - Università di Bologna, 2018.
Find full textAbstract:
L'obiettivo principale di questa tesi è studiare come gli algoritmi di apprendimento automatico (machine learning in inglese) e in particolare le reti neurali LSTM (Long Short Term Memory) possano essere utilizzati per prevedere i valori futuri di una serie storica regolare come, per esempio, le funzioni seno e coseno. Una serie storica è definita come una sequenza di osservazioni s_t ordinate nel tempo.
Inoltre cercheremo di applicare gli stessi principi per prevedere i valori di una serie storica prodotta utilizzando i dati di vendita di un prodotto cosmetico durante un periodo di tre anni.
Prima di arrivare alla parte pratica di questa tesi è necessario introdurre alcuni concetti fondamentali che saranno necessari per sviluppare l'architettura e il codice del nostro modello.
Sia nell'introduzione teorica che nella parte pratica l'attenzione sarà focalizzata sull'uso di RNN (Recurrent Neural Network o Rete Neurale Ricorrente) poiché sono le reti neurali più adatte a questo tipo di problema. Un particolare tipo di RNN, chiamato Long Short Term Memory (LSTM), sarà soggetto dello studio principale di questa tesi e verrà presentata e utilizzata anche una delle sue varianti chiamata Gated Recurrent Unit (GRU).
Questa tesi, in conclusione, conferma che LSTM e GRU sono il miglior tipo di rete neurale per le previsioni di serie temporali. Nell'ultima parte analizzeremo le differenze tra l'utilizzo di una CPU e una GPU durante la fase di training della rete neurale.
20
Nääs, Starberg Filip, and Axel Rooth. "Predicting a business application's cloud server CPU utilization using the machine learning model LSTM." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2021. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-301247.
Full textAbstract:
Cloud Computing sees increased adoption as companies seek to increase flexibility and reduce cost. Although the large cloud service providers employ a pay-as-you-go pricing model and enable customers to scale up and down quickly, there is still room for improvement. Workload in the form of CPU utilization often fluctuates which leads to unnecessary cost and environmental impact for companies. To help mitigate this issue, the aim of this paper is to predict future CPU utilization using a long short-term memory (LSTM) machine learning model. By predicting utilization up to 30 minutes into the future, companies are able to scale their capacity just in time and avoid unnecessary cost and damage to the environment. The study is divided into two parts. The first part analyses how well the LSTM model performs when predicting one step at a time compared with a state-of-the-art model. The second part analyses the accuracy of the LSTM when making predictions up to 30 minutes into the future. To allow for an objective analysis of results, the LSTM is compared with a standard RNN, which is similar to the LSTM in its inherit algorithmic structure. To conclude, the results suggest that LSTM may be a useful tool for reducing cost and unnecessary environmental impact for business applications hosted on a public cloud.<br>Användandet av molntjänster ökar bland företag som önskar förbättrad flexibilitet och sänkta kostnader. De stora molntjänstleverantörerna använder en prismodell där kostnaden är direkt kopplad till användningen, och låter kunderna snabbt ställa om sin kapacitet, men det finns ändå förbättringsmöjligheter. CPU-behoven fluktuerar ofta vilket leder till meningslösa kostnader och onödig påverkan på klimatet när kapacitet är outnyttjad. För att lindra detta problem används i denna rapport en LSTM maskininlärningsmodell för att förutspå framtida CPU-utnyttjande. Genom att förutspå utnyttjandet upp till 30 minuter in i framtiden hinner företag ställa om sin kapacitet och undvika onödig kostnad och klimatpåverkan. Arbetet ¨ar uppdelat i två delar. Först en del där LSTM-modellen förutspår ett tidssteg åt gången. Därefter en del som analyserar träffsäkerheten för LSTM flera tidssteg in i framtiden, upp till 30 tidssteg. För att möjliggöra en objektiv utvärdering så jämfördes LSTM-modellen med ett standard recurrent neural network (RNN) vilken liknar LSTM i sin struktur. Resultaten i denna studie visar att LSTM verkar vara ¨överlägsen RNN, både när det gäller att förutspå ett tidssteg in i framtiden och när det gäller flera tidssteg in i framtiden. LSTM-modellen var kapabel att förutspå CPU-utnyttjandet 30 minuter in i framtiden med i hög grad bibehållen träffsäkerhet, vilket också var målet med studien. Sammanfattningsvis tyder resultaten på att denna LSTM-modell, och möjligen liknande LSTM-modeller, har potential att användas i samband med företagsapplikationer då man önskar att reducera onödig kostnad och klimatpåverkan.
21
Díaz, González Fernando. "Federated Learning for Time Series Forecasting Using LSTM Networks: Exploiting Similarities Through Clustering." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-254665.
Full textAbstract:
Federated learning poses a statistical challenge when training on highly heterogeneous sequence data. For example, time-series telecom data collected over long intervals regularly shows mixed fluctuations and patterns. These distinct distributions are an inconvenience when a node not only plans to contribute to the creation of the global model but also plans to apply it on its local dataset. In this scenario, adopting a one-fits-all approach might be inadequate, even when using state-of-the-art machine learning techniques for time series forecasting, such as Long Short-Term Memory (LSTM) networks, which have proven to be able to capture many idiosyncrasies and generalise to new patterns. In this work, we show that by clustering the clients using these patterns and selectively aggregating their updates in different global models can improve local performance with minimal overhead, as we demonstrate through experiments using realworld time series datasets and a basic LSTM model.<br>Federated Learning utgör en statistisk utmaning vid träning med starkt heterogen sekvensdata. Till exempel så uppvisar tidsseriedata inom telekomdomänen blandade variationer och mönster över längre tidsintervall. Dessa distinkta fördelningar utgör en utmaning när en nod inte bara ska bidra till skapandet av en global modell utan även ämnar applicera denna modell på sin lokala datamängd. Att i detta scenario införa en global modell som ska passa alla kan visa sig vara otillräckligt, även om vi använder oss av de mest framgångsrika modellerna inom maskininlärning för tidsserieprognoser, Long Short-Term Memory (LSTM) nätverk, vilka visat sig kunna fånga komplexa mönster och generalisera väl till nya mönster. I detta arbete visar vi att genom att klustra klienterna med hjälp av dessa mönster och selektivt aggregera deras uppdateringar i olika globala modeller kan vi uppnå förbättringar av den lokal prestandan med minimala kostnader, vilket vi demonstrerar genom experiment med riktigt tidsseriedata och en grundläggande LSTM-modell.
22
Andréasson, David, and Blomquist Jesper Mortensen. "Forecasting the OMXS30 - a comparison between ARIMA and LSTM." Thesis, Uppsala universitet, Statistiska institutionen, 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-413793.
Full textAbstract:
Machine learning is a rapidly growing field with more and more applications being proposed every year, including but not limited to the financial sector. In this thesis, historical adjusted closing prices from the OMXS30 index are used to forecast the corresponding future values using two different approaches; one using an ARIMA model and the other using an LSTM neural network. The forecasts are made on three different time intervals: 90, 30 and 7 days ahead. The results showed that the LSTM model performs slightly better when forecasting 90 and 30 days ahead, whereas the ARIMA model has comparable accuracy on the seven day forecast.
23
Bergström, Carl, and Oscar Hjelm. "Impact of Time Steps on Stock Market Prediction with LSTM." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-262221.
Full textAbstract:
Machine learning models as tools for predicting time series have in recent years proven to perform exceptionally well. With financial time series in the form of stock indices being inherently complex and subject to noise and volatility, the prediction of stock market movements has proven to be especially difficult throughout extensive research. The objective of this study is to thoroughly analyze the LSTM architecture for neural networks and its performance when applied to the S&P 500 stock index. The main research question revolves around quantifying the impact of varying the number of time steps in the LSTM model on predictive performance when applied to the S&P 500 index. The data used in the model is of high reliability downloaded from the Bloomberg Terminal, where the closing price has been used as feature in the model. Other constituents of the model have been based in previous research, where satisfactory results have been reached. The results indicate that among the evaluated time steps, ten steps provided the superior performance. However, the impact of varying time steps is not all too significant for the overall performance of the model. Finally, the implications of the results for the field of research present themselves as good basis for future research, where parameters are varied and fine-tuned in pursuit of optimal performance.<br>Maskininlärningsmodeller som redskap för att förutspå tidsserier har de senaste åren visat sig prestera exceptionellt bra. Vad gäller finansiella tidsserier i formen av aktieindex, som har en inneboende komplexitet, och är föremål för störningar och volatilitet, har förutsägelse av aktiemarknadsrörelser visat sig vara särskilt svårt igenom omfattande forskning. Målet med denna studie är att grundligt undersöka LSTM-arkitekturen för neurala nätverk och dess prestanda när den appliceras på aktieindexet S&P 500. Huvudfrågan kretsar kring att kvantifiera inverkan som varierande av antal tidssteg i LTSM-modellen har på prediktivprestanda när den appliceras på aktieindexet S&P 500. Data som använts i modellen är av hög pålitlighet, nedladdad frånBloomberg-terminalen, där stängningskurs har använts som feature i modellen. Andra beståndsdelar av modellen har baserats i tidigare forskning, där tillfredsställande resultat har uppnåtts. Resultaten indikerar att bland de testade tidsstegen så producerartio tidssteg bäst resultat. Dock verkar inte påverkan av antalet tidssteg vara särskilt signifikant för modellens övergripandeprestanda. Slutligen så presenterar sig implikationerna av resultaten för forskningsområdet som god grund för framtida forskning, där parametrar kan varieras och finjusteras i strävan efter optimal prestanda.
24
Jansson, Anton. "Predicting trajectories of golf balls using recurrent neural networks." Thesis, KTH, Skolan för datavetenskap och kommunikation (CSC), 2017. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-210552.
Full textAbstract:
This thesis is concerned with the problem of predicting the remaining part of the trajectory of a golf ball as it travels through the air where only the three-dimensional position of the ball is captured. The approach taken to solve this problem relied on recurrent neural networks in the form of the long short-term memory networks (LSTM). The motivation behind this choice was that this type of networks had led to state-of-the-art performance for similar problems such as predicting the trajectory of pedestrians. The results show that using LSTMs led to an average reduction of 36.6 % of the error in the predicted impact position of the ball, compared to previous methods based on numerical simulations of a physical model, when the model was evaluated on the same driving range that it was trained on. Evaluating the model on a different driving range than it was trained on leads to improvements in general, but not for all driving ranges, in particular when the ball was captured at a different frequency compared to the data that the model was trained on. This problem was solved to some extent by retraining the model with small amounts of data on the new driving range.<br>Detta examensarbete har studerat problemet att förutspå den fullständiga bollbanan för en golfboll när den flyger i luften där endast den tredimensionella positionen av bollen observerades. Den typ av metod som användes för att lösa problemet använde sig av recurrent neural networks, i form av long short-term memory nätverk (LSTM). Motivationen bakom detta var att denna typ av nätverk hade lett till goda resultatet för liknande problem. Resultatet visar att använda sig av LSTM nätverk leder i genomsnitt till en 36.6 % förminskning av felet i den förutspådda nedslagsplatsen för bollen jämfört mot tidigare metoder som använder sig av numeriska simuleringar av en fysikalisk modell, om modellen användes på samma golfbana som den tränades på. Att använda en modell som var tränad på en annan golfbana leder till förbättringar i allmänhet, men inte om modellen användes på en golfbana där bollen fångades in med en annan frekvens. Detta problem löstes till en viss mån genom att träna om modellen med lite data från den nya golfbanan.
25
Nordberg, Andreas. "Evaluation of Neural Networks for Predictive Maintenance : A Volvo Penta Study." Thesis, Linköpings universitet, Institutionen för datavetenskap, 2021. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-176390.
Full textAbstract:
As part of Volvo Penta's initiative to further the development of predictive maintenance in their field test environments, this thesis compares neural networks in an effort to predict the occurrence of three common diagnostics trouble codes using field test data. To quantify the neural networks' performances for comparison a number of evaluation metrics were used. By training a multitude of differently configured feedforward neural networks with the processed field test data and evaluating the resulting models, it was found that the resulting models perform better than that of a baseline classifier. As such it is possible to use Volvo Penta's field test data along with neural networks to achieve predictive maintenance. It was also found that Long Short-Term Memory (LSTM) networks with methodically selected hyperparameters were able to predict the diagnostic trouble codes with the greatest performance among all the tested neural networks.
26
Kvedaraite, Indre. "Sentiment Analysis of YouTube Public Videos based on their Comments." Thesis, Linnéuniversitetet, Institutionen för datavetenskap och medieteknik (DM), 2021. http://urn.kb.se/resolve?urn=urn:nbn:se:lnu:diva-105754.
Full textAbstract:
With the rise of social media and publicly available data, opinion mining is more accessible than ever. It is valuable for content creators, companies and advertisers to gain insights into what users think and feel. This work examines comments on YouTube videos, and builds a deep learning classifier to automatically determine their sentiment. Four Long Short-Term Memory-based models are trained and evaluated. Experiments are performed to determine which deep learning model performs with the best accuracy, recall, precision, F1 score and ROC curve on a labelled YouTube Comment dataset. The results indicate that a BiLSTM-based model has the overall best performance, with the accuracy of 89%. Furthermore, the four LSTM-based models are evaluated on an IMDB movie review dataset, achieving an average accuracy of 87%, showing that the models can predict the sentiment of different textual data. Finally, a statistical analysis is performed on the YouTube videos, revealing that videos with positive sentiment have a statistically higher number of upvotes and views. However, the number of downvotes is not significantly higher in videos with negative sentiment.
27
Shu, Xingliang. "Electrocardiograph Signal Classification By Using Neural Network." University of Cincinnati / OhioLINK, 2020. http://rave.ohiolink.edu/etdc/view?acc_num=ucin1592395089900722.
Full text
28
Korte, Christopher M. "A Preliminary Investigation into using Artificial Neural Networks to Generate Surgical Trajectories to Enable Semi-Autonomous Surgery in Space." University of Cincinnati / OhioLINK, 2020. http://rave.ohiolink.edu/etdc/view?acc_num=ucin1595499765813353.
Full text
29
Shojaee, Ali B. S. "Bacteria Growth Modeling using Long-Short-Term-Memory Networks." University of Cincinnati / OhioLINK, 2021. http://rave.ohiolink.edu/etdc/view?acc_num=ucin1617105038908441.
Full text
30
Bermell, Måns. "Identification of Problem Gambling via Recurrent Neural Networks : Predicting self-exclusion due to problem gambling within the remote gambling sector by means of recurrent neural networks." Thesis, Umeå universitet, Institutionen för fysik, 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:umu:diva-159125.
Full textAbstract:
Under recent years the gambling industry has been moving towards providing their customer the possibility to gamble online instead of visiting a physical location. Aggressive marketing, fast growth and a multitude of actors within the market have resulted in a spike of customers who have developed a gambling problem. Decision makers are trying to fight back by regulating markets in order to make the companies take responsibility and work towards preventing these problems. One method of working proactively in this regards is to identify vulnerable customers before they develop a destructive habit. In this work a novel method of predicting customers that have a higher risk in regards to gambling-related problems is explored. More concretely, a recurrent neural network with long short-term memory cells is created to process raw behaviour data that are aggregated on a daily basis to classify them as high-risk or not. Supervised training is used in order to learn from historical data, where the usage of permanent self-exclusions due to gambling related problems defines problem gamblers. The work consists of: obtain a local optimal configuration of the network which enhances the performance for identifying problem gam- blers who favour the casino section over sports section, and analyze the model to provide insights in the field. This project was carried out together with LeoVegas Mobile Gaming Group. The group offers both online casino games and sports booking in a number of countries in Europe. This collaboration made both data and expertise within the industry accessible to perform this work. The company currently have a model in production to perform these predictions, but want to explore other approaches. The model that has been developed showed a significant increase in performance compared to the one that is currently used at the company. Specifically, the precision and recall which are two metrics important for a two class classification model, increased by 37% and 21% respectively. Using raw time series data, instead of aggregated data increased the responsiveness regarding customers change in behaviour over time. The model also scaled better with more history compared to the current model, which could be a result of the nature of a recurrent network compared to the current model used.
31
Lindskog, Jakob, and Robin Gunnarsson. "Databearbetning på Ringhals." Thesis, Högskolan i Halmstad, Akademin för informationsteknologi, 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:hh:diva-39773.
Full textAbstract:
Den nya generationens digitalisering har slagit rot i samhället. Algoritmer och datamodeller styr nyhetsflödet i social media, röststyr mobilen genom att tolka rösten och självstyr bilen, helt och hållet i autonoma fordon. Inom industrierna finns det också en pågående process där machine learning kan appliceras för att öka drifttillgänglighet och minska kostnader. Det nuvarande paradigmet för att underhålla icke-säkerhetsklassade maskiner i kärnkraftindustrin är en kombination av Avhjälpande Underhåll och Förebyggande Underhåll. Avhjälpande underhåll innebär att underhålla maskinen när fel inträffar, förebyggande underhåll innebär att underhålla med periodiska intervall. Båda sätten är kostsamma för att de riskerar att under- respektive över-underhålla maskinen och blir därmed resurskrävande. Ett paradigmskifte är på väg, det stavas Prediktivt Underhåll - att kunna förutspå fel innan de inträffar och planera underhåll därefter. Den här rapporten utforskar möjligheten att använda sig av de neurala nätverken LSTM och GRU för att kunna prognostisera eventuella skador på maskiner. Det här baseras på mätdata och historiska fel på maskinen.<br>The new generation of digitalization has been ingrained into society. Algorithms and data models are controlling the news feed of social media, controlling the phone by interpreting voices and controlling the car, altogether with automonous vehicles. In the industries there is also an ongoing process where machine learning is applied to increase availability and reduce costs. The current paradigm for maintaining non-critical machines in the nuclear power industry is a combination of corrective maintenance and preventive maintenance. Corrective maintenance means doing repairs on the machine upon faults, preventive maintenance means doing repairs periodically. Both ways are costly because they run the risk of under- and over-maintaining the machine and therefore becoming resource-intensive. A paradigm shift is on it's way, and it's spelled Predictive Maintenance - being able to predict faults before they happen and plan maintenance thence. This report explores the possibilities of using LSTM and GRU to forecast potential damage on machines. This is based on data from measurements and historical issues on the machine.
32
Song, Philip, and André Brogärd. "Performance Analysis of Various Activation Functions Using LSTM Neural Network For Movie Recommendation Systems." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-280451.
Full textAbstract:
The growth of importance and popularity of recommendations system has increased in many various areas. This thesis focuses on recommendation systems for movies. Recurrent neural networks using LSTM blocks have shown some success for movie recommendation systems. Research has indicated that by changing activation functions in LSTM blocks, the performance, measured as accuracy in predictions, can be improved. In this study we compare four different activation functions (hyperbolic tangent, sigmoid, ELU and SELU activation functions) used in LSTM blocks, and how they impact the prediction accuracy of the neural networks. Specifically, they are applied to the block input and the block output of the LSTM blocks. Our results indicate that the hyperbolic tangent, which is the default, and sigmoid function perform about the same, whereas the ELU and SELU functions perform worse. Further research is needed to identify other activation functions that could improve the prediction accuracy and improve certain aspects of our methodology.<br>Rekommendationssystem har ökat i betydelse och popularitet i många olika områden. Denna avhandling fokuserar på rekommendationssystem för filmer. Recurrent neurala nätverk med LSTM blocks har visat viss framgång för rekommendationssystem för filmer. Tidigare forskning har indikerat att en ändring av aktiverings funktioner har resulterat i förbättrad prediktering. I denna studie jämför vi fyra olika aktiveringsfunktioner (hyperbolic tangent, sigmoid, ELU and SELU) som appliceras i LSTM blocks och hur de påverkar predikteringen i det neurala nätverket. De appliceras specifikt på block input och block output av LSTM blocken. Våra resultat indikerar att den hyperboliska tangentfunktionen, som är standardvalet, och sigmoid funktionen presterar lika, men ELU och SELU presterar båda sämre. Ytterligare forskning krävs för att indentifiera andra aktiveringsfunktioner och för att förbättra flera delar av metodologin.
33
Nawaz, Sabeen. "Analysis of Transactional Data with Long Short-Term Memory Recurrent Neural Networks." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-281282.
Full textAbstract:
An issue authorities and banks face is fraud related to payments and transactions where huge monetary losses occur to a party or where money laundering schemes are carried out. Previous work in the field of machine learning for fraud detection has addressed the issue as a supervised learning problem. In this thesis, we propose a model which can be used in a fraud detection system with transactions and payments that are unlabeled. The proposed modelis a Long Short-term Memory in an auto-encoder decoder network (LSTMAED)which is trained and tested on transformed data. The data is transformed by reducing it to Principal Components and clustering it with K-means. The model is trained to reconstruct the sequence with high accuracy. Our results indicate that the LSTM-AED performs better than a random sequence generating process in learning and reconstructing a sequence of payments. We also found that huge a loss of information occurs in the pre-processing stages.<br>Obehöriga transaktioner och bedrägerier i betalningar kan leda till stora ekonomiska förluster för banker och myndigheter. Inom maskininlärning har detta problem tidigare hanterats med hjälp av klassifierare via supervised learning. I detta examensarbete föreslår vi en modell som kan användas i ett system för att upptäcka bedrägerier. Modellen appliceras på omärkt data med många olika variabler. Modellen som används är en Long Short-term memory i en auto-encoder decoder nätverk. Datan transformeras med PCA och klustras med K-means. Modellen tränas till att rekonstruera en sekvens av betalningar med hög noggrannhet. Vår resultat visar att LSTM-AED presterar bättre än en modell som endast gissar nästa punkt i sekvensen. Resultatet visar också att mycket information i datan går förlorad när den förbehandlas och transformeras.
34
Backer-Meurke, Henrik, and Marcus Polland. "Predicting Road Rut with a Multi-time-series LSTM Model." Thesis, Högskolan Dalarna, Institutionen för information och teknik, 2021. http://urn.kb.se/resolve?urn=urn:nbn:se:du-37599.
Full textAbstract:
Road ruts are depressions or grooves worn into a road. Increases in rut depth are highly undesirable due to the heightened risk of hydroplaning. Accurately predicting increases in road rut depth is important for maintenance planning within the Swedish Transport Administration. At the time of writing this paper, the agency utilizes a linear regression model and is developing a feed-forward neural network for road rut predictions. The aim of the study was to evaluate the possibility of using a Recurrent Neural Network to predict road rut. Through design science research, an artefact in the form of a LSTM model was designed, developed, and evaluated.The dataset consisted of multiple-multivariate short time series where research was limited. Case studies were conducted which inspired the conceptual design of the model. The baseline LSTM model proposed in this paper utilizes the full dataset in combination with time-series individualization through an added index feature. Additional features thought to correlate with rut depth was also studied through multiple training set variations. The model was evaluated by calculating the Root Mean Squared Error (RMSE) and the Mean Absolute Error (MAE) for each training set variation. The baseline model predicted rut depth with a MAE of 0.8110 (mm) and a RMSE of 1.124 (mm) outperforming a control set without the added index. The feature with the highest correlation to rut depth was curvature with a MAEof 0.8031 and a RMSE of 1.1093. Initial finding shows that there is a possibility of utilizing an LSTM model trained on multiple-multivariate time series to predict rut depth. Time series individualization through an added index feature yielded better results than control, indicating that it had the desired effect on model performance.
35
Racette, Olsén Michael. "Electrocardiographic deviation detection : Using long short-term memory recurrent neural networks to detect deviations within electrocardiographic records." Thesis, Linnéuniversitetet, Institutionen för datavetenskap (DV), 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:lnu:diva-76411.
Full textAbstract:
Artificial neural networks have been gaining attention in recent years due to theirimpressive ability to map out complex nonlinear relations within data. In this report,an attempt is made to use a Long short-term memory neural network for detectinganomalies within electrocardiographic records. The hypothesis is that if a neuralnetwork is trained on records of normal ECGs to predict future ECG sequences, it isexpected to have trouble predicting abnormalities not previously seen in the trainingdata. Three different LSTM model configurations were trained using records fromthe MIT-BIH Arrhythmia database. Afterwards the models were evaluated for theirability to predict previously unseen normal and anomalous sections. This was doneby measuring the mean squared error of each prediction and the uncertainty of over-lapping predictions. The preliminary results of this study demonstrate that recurrentneural networks with the use of LSTM units are capable of detecting anomalies.
36
Mussumeci, Elisa. "A machine learning approach to dengue forecasting: comparing LSTM, Random Forest and Lasso." reponame:Repositório Institucional do FGV, 2018. http://hdl.handle.net/10438/24093.
Full textAbstract:
Submitted by Elisa Mussumeci (elisamussumeci@gmail.com) on 2018-05-29T18:53:58Z
No. of bitstreams: 1
machine-learning-aproach (4).pdf: 11272802 bytes, checksum: 52b25abf2711fdd6d1a338316c15c154 (MD5)<br>Approved for entry into archive by ÁUREA CORRÊA DA FONSECA CORRÊA DA FONSECA (aurea.fonseca@fgv.br) on 2018-05-29T19:15:35Z (GMT) No. of bitstreams: 1
machine-learning-aproach (4).pdf: 11272802 bytes, checksum: 52b25abf2711fdd6d1a338316c15c154 (MD5)<br>Made available in DSpace on 2018-06-14T19:45:29Z (GMT). No. of bitstreams: 1
machine-learning-aproach (4).pdf: 11272802 bytes, checksum: 52b25abf2711fdd6d1a338316c15c154 (MD5)
Previous issue date: 2018-04-12<br>We used the Infodengue database of incidence and weather time-series, to train predictive models for the weekly number of cases of dengue in 790 cities of Brazil. To overcome a limitation in the length of time-series available to train the model, we proposed using the time series of epidemiologically similar cities as predictors for the incidence of each city. As Machine Learning-based forecasting models have been used in recent years with reasonable success, in this work we compare three machine learning models: Random Forest, lasso and Long-short term memory neural network in their forecasting performance for all cities monitored by the Infodengue Project.
37
Ghandeharioon, Cosar. "An evaluation of deep neural network approaches for traffic speed prediction." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-254911.
Full textAbstract:
The transportation industry has a significant effect on the sustainability and development of a society. Learning traffic patterns, and predicting the traffic parameters such as flow or speed for a specific spatiotemporal point is beneficial for transportation systems. For instance, intelligent transportation systems (ITS) can use forecasted results to improve services such as driver assistance systems. Furthermore, the prediction can facilitate urban planning by making management decisions data driven. There are several prediction models for time series regression on traffic data to predict the average speed for different forecasting horizons. In this thesis work, we evaluated Long Short-Term Memory (LSTM), one of the recurrent neural network models and Neural decomposition (ND), a neural network that performs Fourier-like decomposition. The results were compared with the ARIMA model. The persistent model was chosen as a baseline for the evaluation task. We proposed two new criteria in addition to RMSE and r2, to evaluate models for forecasting highly variable velocity changes. The dataset was gathered from highway traffic sensors around the E4 in Stockholm, taken from the “Motorway Control System” (MCS) operated by Trafikverket. Our experiments show that none of the models could predict the highly variable velocity changes at the exact times they happen. The reason was that the adjacent local area had no indications of sudden changes in the average speed of vehicles passing the selected sensor. We also conclude that traditional ML metrics of RMSE and r2 could be augmented with domain specific measures.<br>Transportbranschen har en betydande inverkan på samhällets hållbarhet och utveckling. Att lära sig trafikmönster och förutsäga trafikparametrar som flöde eller hastighet för en specifik spatio-temporal punkt är fördelaktigt för transportsystem. Intelligenta transportsystem (ITS) kan till exempel använda prognostiserade resultat för att förbättra tjänster som förarassistanssystem. Vidare kan förutsägelsen underlätta stadsplanering genom att göra ledningsbeslut datadrivna. Det finns flera förutsägelsemodeller för tidsserieregression på trafikdata för att förutsäga medelhastigheten för olika prognoshorisonter. I det här avhandlingsarbetet utvärderade vi Långtidsminne (LSTM), en av de återkommande neurala nätverksmodellerna och Neural dekomposition (ND), ett neuralt nätverk som utför Fourierliknande sönderdelning. Resultaten jämfördes med ARIMA-modellen. Den ihållande modellen valdes som utgångspunkt för utvärderingsuppgiften. Vi föreslog två nya kriterier utöver RMSE och r2, för att utvärdera modeller för prognoser av högt variabla hastighetsändringar. Datasetet insamlades från trafiksensor på motorvägar runt E4 i Stockholm, för det så kallade motorvägskontrollsystemet (MCS). Våra experiment visar att ingen av modellerna kan förutsäga de höga variabla hastighetsförändringarna vid exakta tider som de händer. Anledningen var att det intilliggande lokala området inte hade några indikationer på plötsliga förändringar i medelhastigheten hos fordon som passerade den valda sensorn. Vi drar också slutsatsen att traditionella ML-metrics av RMSE och R2 kan kompletteras med domänspecifika åtgärder.
38
Peng, Yingqi. "Japanese Black Cattle Behavior Pattern Classification Based on Neural Networks Using Inertial Sensors and Magnetic Direction Sensor." Kyoto University, 2019. http://hdl.handle.net/2433/244558.
Full text
39
Persson, Peter. "Starved neural learning : Morpheme segmentation using low amounts of data." Thesis, Stockholms universitet, Avdelningen för datorlingvistik, 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:su:diva-160953.
Full textAbstract:
Automatic morpheme segmentation as a field has been dominated by unsupervised methods since its inception. Partly due to theoretical motivations, but also due to resource constraints. Given the success neural network methods have shown on a wide variety of field in later years, it would seem compelling to apply these methods to the morpheme segmentation field. This study explores the efficacy of modern neural networks, specifically convolutional neural networks and Bi-directional LSTM networks, on the morpheme segmentation task in a resource low setting to determine their viability as contenders with previous unsupervised, minimally supervised, and semi-supervised systems in the field. One architecture of each type is implemented and trained on a new gold standard data set and the results are compared to previously established methods. A qualitative error analysis of the architectures’ segmentations is also performed. The study demonstrates that a BLSTM system can be trained with minimal effort to produce a proof of concept solution at low levels of training data and suggests that BLSTM methods may be a fruitful direction for further research in this field.
40
Gelly, Grégory. "Réseaux de neurones récurrents pour le traitement automatique de la parole." Thesis, Université Paris-Saclay (ComUE), 2017. http://www.theses.fr/2017SACLS295/document.
Full textAbstract:
Le domaine du traitement automatique de la parole regroupe un très grand nombre de tâches parmi lesquelles on trouve la reconnaissance de la parole, l'identification de la langue ou l'identification du locuteur. Ce domaine de recherche fait l'objet d'études depuis le milieu du vingtième siècle mais la dernière rupture technologique marquante est relativement récente et date du début des années 2010. C'est en effet à ce moment qu'apparaissent des systèmes hybrides utilisant des réseaux de neurones profonds (DNN) qui améliorent très notablement l'état de l'art. Inspirés par le gain de performance apporté par les DNN et par les travaux d'Alex Graves sur les réseaux de neurones récurrents (RNN), nous souhaitions explorer les capacités de ces derniers. En effet, les RNN nous semblaient plus adaptés que les DNN pour traiter au mieux les séquences temporelles du signal de parole. Dans cette thèse, nous nous intéressons tout particulièrement aux RNN à mémoire court-terme persistante (Long Short Term Memory (LSTM) qui permettent de s'affranchir d'un certain nombre de difficultés rencontrées avec des RNN standards. Nous augmentons ce modèle et nous proposons des processus d'optimisation permettant d'améliorer les performances obtenues en segmentation parole/non-parole et en identification de la langue. En particulier, nous introduisons des fonctions de coût dédiées à chacune des deux tâches: un simili-WER pour la segmentation parole/non-parole dans le but de diminuer le taux d'erreur d'un système de reconnaissance de la parole et une fonction de coût dite de proximité angulaire pour les problèmes de classification multi-classes tels que l'identification de la langue parlée<br>Automatic speech processing is an active field of research since the 1950s. Within this field the main area of research is automatic speech recognition but simpler tasks such as speech activity detection, language identification or speaker identification are also of great interest to the community. The most recent breakthrough in speech processing appeared around 2010 when speech recognition systems using deep neural networks drastically improved the state-of-the-art. Inspired by this gains and the work of Alex Graves on recurrent neural networks (RNN), we decided to explore the possibilities brought by these models on realistic data for two different tasks: speech activity detection and spoken language identification. In this work, we closely look at a specific model for the RNNs: the Long Short Term Memory (LSTM) which mitigates a lot of the difficulties that can arise when training an RNN. We augment this model and introduce optimization methods that lead to significant performance gains for speech activity detection and language identification. More specifically, we introduce a WER-like loss function to train a speech activity detection system so as to minimize the word error rate of a downstream speech recognition system. We also introduce two different methods to successfully train a multiclass classifier based on neural networks for tasks such as LID. The first one is based on a divide-and-conquer approach and the second one is based on an angular proximity loss function. Both yield performance gains but also speed up the training process
41
Parthiban, Dwarak Govind. "On the Softmax Bottleneck of Word-Level Recurrent Language Models." Thesis, Université d'Ottawa / University of Ottawa, 2020. http://hdl.handle.net/10393/41412.
Full textAbstract:
For different input contexts (sequence of previous words), to predict the next word, a neural word-level language model outputs a probability distribution over all the words in the vocabulary using a softmax function. When the log of probability outputs for all such contexts are stacked together, the resulting matrix is a log probability matrix which can be denoted as Q_theta, where theta denotes the model parameters. When language modeling is formulated as a matrix factorization problem, the matrix to be factorized Q_theta is expected to be high-rank as natural language is highly context-dependent. But existing softmax based word-level language models have a limitation of not being able to produce such matrices; this is known as the softmax bottleneck.
There are several works that attempted to overcome the limitations introduced by softmax bottleneck, such as the models that can produce high-rank Q_theta. During the process of reproducing the results of these works, we observed that the rank of Q_theta does not always positively correlate with better performance (i.e., lower test perplexity). This puzzling observation triggered us to conduct a systematic investigation to check the influence of rank of Q_theta on better performance of a language model. We first introduce a new family of activation functions called the Generalized SigSoftmax (GSS). By controlling the parameters of GSS, we were able to construct language models that can produce Q_theta with diverse ranks (i.e., low, medium, and high ranks). For models that use GSS with different parameters, we observe that rank does not have a strong positive correlation with perplexity on the test data, reinforcing the support of our initial observation. By inspecting the top-5 predictions made by different models for a selected set of input contexts, we observe that a high-rank Q_theta does not guarantee a strong qualitative performance. Then, we conduct experiments to check if there are any other additional benefits in having models that can produce high-rank Q_theta. We expose that Q_theta rather suffers from the phenomenon of fast singular value decay. Additionally, we also propose an alternative metric to denote the rank of any matrix known as epsilon-effective rank, which can be useful to approximately quantify the singular value distribution when different values for epsilon are used.
We conclude by showing that it is the regularization which has played a positive role in the performance of these high-rank models in comparison to the chosen baselines, and there is no single model yet which truly gains improved expressiveness just because of breaking the softmax bottleneck.
42
Broomé, Sofia. "Objectively recognizing human activity in body-worn sensor data with (more or less) deep neural networks." Thesis, KTH, Skolan för datavetenskap och kommunikation (CSC), 2017. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-210243.
Full textAbstract:
This thesis concerns the application of different artificial neural network architectures on the classification of multivariate accelerometer time series data into activity classes such as sitting, lying down, running, or walking. There is a strong correlation between increased health risks in children and their amount of daily screen time (as reported in questionnaires). The dependency is not clearly understood, as there are no such dependencies reported when the sedentary (idle) time is measured objectively. Consequently, there is an interest from the medical side to be able to perform such objective measurements. To enable large studies the measurement equipment should ideally be low-cost and non-intrusive. The report investigates how well these movement patterns can be distinguished given a certain measurement setup and a certain network structure, and how well the networks generalise to noisier data. Recurrent neural networks are given extra attention among the different networks, since they are considered well suited for data of sequential nature. Close to state-of-the-art results (95% weighted F1-score) are obtained for the tasks with 4 and 5 classes, which is notable since a considerably smaller number of sensors is used than in the previously published results. Another contribution of this thesis is that a new labeled dataset with 12 activity categories is provided, consisting of around 6 hours of recordings, comparable in number of samples to benchmarking datasets. The data collection was made in collaboration with the Department of Public Health at Karolinska Institutet.<br>Inom ramen för uppsatsen testas hur väl rörelsemönster kan urskiljas ur accelerometerdatamed hjälp av den gren av maskininlärning som kallas djupinlärning; där djupa artificiellaneurala nätverk av noder funktionsapproximerar mappandes från domänen av sensordatatill olika fördefinerade kategorier av aktiviteter så som gång, stående, sittande eller liggande.Det finns ett intresse från den medicinska sidan att kunna mäta fysisk aktivitet objektivt,bland annat eftersom det visats att det finns en korrelation mellan ökade hälsorisker hosbarn och deras mängd daglig skärmtid. Denna typ av mätningar ska helst kunna göras medicke-invasiv utrustning till låg kostnad för att kunna göra större studier.Enklare nätverksarkitekturer samt återimplementeringar av bästa möjliga teknik inomområdet Mänsklig aktivitetsigenkänning (HAR) testas både på ett benchmarkingdataset ochpå egeninhämtad data i samarbete med Institutet för Folkhälsovetenskap på Karolinska Institutetoch resultat redovisas för olika val av möjliga klassificeringar och olika antal dimensionerper mätpunkt. De uppnådda resultaten (95% F1-score) på ett 4- och 5-klass-problem ärjämförbara med de bästa tidigare publicerade resultaten för aktivitetsigenkänning, vilket äranmärkningsvärt då då betydligt färre accelerometrar har använts här än i de åsyftade studierna.Förutom klassificeringsresultaten som redovisas bidrar det här arbetet med ett nyttinhämtat och kategorimärkt dataset; KTH-KI-AA. Det är jämförbart i antal datapunkter medspridda benchmarkingdataset inom HAR-området.
43
Bopaiah, Jeevith. "A recurrent neural network architecture for biomedical event trigger classification." UKnowledge, 2018. https://uknowledge.uky.edu/cs_etds/73.
Full textAbstract:
A “biomedical event” is a broad term used to describe the roles and interactions between entities (such as proteins, genes and cells) in a biological system. The task of biomedical event extraction aims at identifying and extracting these events from unstructured texts. An important component in the early stage of the task is biomedical trigger classification which involves identifying and classifying words/phrases that indicate an event. In this thesis, we present our work on biomedical trigger classification developed using the multi-level event extraction dataset. We restrict the scope of our classification to 19 biomedical event types grouped under four broad categories - Anatomical, Molecular, General and Planned. While most of the existing approaches are based on traditional machine learning algorithms which require extensive feature engineering, our model relies on neural networks to implicitly learn important features directly from the text. We use natural language processing techniques to transform the text into vectorized inputs that can be used in a neural network architecture. As per our knowledge, this is the first time neural attention strategies are being explored in the area of biomedical trigger classification. Our best results were obtained from an ensemble of 50 models which produced a micro F-score of 79.82%, an improvement of 1.3% over the previous best score.
44
Stark, Love. "Outlier detection with ensembled LSTM auto-encoders on PCA transformed financial data." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2021. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-296161.
Full textAbstract:
Financial institutions today generate a large amount of data, data that can contain interesting information to investigate to further the economic growth of said institution. There exists an interest in analyzing these points of information, especially if they are anomalous from the normal day-to-day work. However, to find these outliers is not an easy task and not possible to do manually due to the massive amounts of data being generated daily. Previous work to solve this has explored the usage of machine learning to find outliers in these financial datasets. Previous studies have shown that the pre-processing of data usually stands for a big part in information loss. This work aims to study if there is a proper balance in how the pre-processing is carried out to retain the highest amount of information while simultaneously not letting the data remain too complex for the machine learning models. The dataset used consisted of Foreign exchange transactions supplied by the host company and was pre-processed through the use of Principal Component Analysis (PCA). The main purpose of this work is to test if an ensemble of Long Short-Term Memory Recurrent Neural Networks (LSTM), configured as autoencoders, can be used to detect outliers in the data and if the ensemble is more accurate than a single LSTM autoencoder. Previous studies have shown that Ensemble autoencoders can prove more accurate than a single autoencoder, especially when SkipCells have been implemented (a configuration that skips over LSTM cells to make the model perform with more variation). A datapoint will be considered an outlier if the LSTM model has trouble properly recreating it, i.e. a pattern that is hard to classify, making it available for further investigations done manually. The results show that the ensembled LSTM model proved to be more accurate than that of a single LSTM model in regards to reconstructing the dataset, and by our definition of an outlier, more accurate in outlier detection. The results from the pre-processing experiments reveal different methods of obtaining an optimal number of components for your data. One of those is by studying retained variance and accuracy of PCA transformation compared to model performance for a certain number of components. One of the conclusions from the work is that ensembled LSTM networks can prove very powerful, but that alternatives to pre-processing should be explored such as categorical embedding instead of PCA.<br>Finansinstitut genererar idag en stor mängd data, data som kan innehålla intressant information värd att undersöka för att främja den ekonomiska tillväxten för nämnda institution. Det finns ett intresse för att analysera dessa informationspunkter, särskilt om de är avvikande från det normala dagliga arbetet. Att upptäcka dessa avvikelser är dock inte en lätt uppgift och ej möjligt att göra manuellt på grund av de stora mängderna data som genereras dagligen. Tidigare arbete för att lösa detta har undersökt användningen av maskininlärning för att upptäcka avvikelser i finansiell data. Tidigare studier har visat på att förbehandlingen av datan vanligtvis står för en stor del i förlust av emphinformation från datan. Detta arbete syftar till att studera om det finns en korrekt balans i hur förbehandlingen utförs för att behålla den högsta mängden information samtidigt som datan inte förblir för komplex för maskininlärnings-modellerna. Det emphdataset som användes bestod av valutatransaktioner som tillhandahölls av värdföretaget och förbehandlades genom användning av Principal Component Analysis (PCA). Huvudsyftet med detta arbete är att undersöka om en ensemble av Long Short-Term Memory Recurrent Neural Networks (LSTM), konfigurerad som autoenkodare, kan användas för att upptäcka avvikelser i data och om ensemblen är mer precis i sina predikteringar än en ensam LSTM-autoenkodare. Tidigare studier har visat att en ensembel avautoenkodare kan visa sig vara mer precisa än en singel autokodare, särskilt när SkipCells har implementerats (en konfiguration som hoppar över vissa av LSTM-cellerna för att göra modellerna mer varierade). En datapunkt kommer att betraktas som en avvikelse om LSTM-modellen har problem med att återskapa den väl, dvs ett mönster som nätverket har svårt att återskapa, vilket gör datapunkten tillgänglig för vidare undersökningar. Resultaten visar att en ensemble av LSTM-modeller predikterade mer precist än en singel LSTM-modell när det gäller att återskapa datasetet, och då enligt vår definition av avvikelser, mer precis avvikelse detektering. Resultaten från förbehandlingen visar olika metoder för att uppnå ett optimalt antal komponenter för dina data genom att studera bibehållen varians och precision för PCA-transformation jämfört med modellprestanda. En av slutsatserna från arbetet är att en ensembel av LSTM-nätverk kan visa sig vara mycket kraftfulla, men att alternativ till förbehandling bör undersökas, såsom categorical embedding istället för PCA.
45
Zhu, Yuehan. "Automated Supply-Chain Quality Inspection Using Image Analysis and Machine Learning." Thesis, Högskolan Kristianstad, Fakulteten för naturvetenskap, 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:hkr:diva-20069.
Full textAbstract:
An image processing method for automatic quality assurance of Ericsson products is developed. The method consists of taking an image of the product, extract the product labels from the image, OCR the product numbers and make a database lookup to match the mounted product with the customer specification. The engineering innovation of the method developed in this report is that the OCR is performed using machine learning techniques. It is shown that machine learning can produce results that are on par or better than baseline OCR methods. The advantage with a machine learning based approach is that the associated neural network can be trained for the specific input images from the Ericsson factory. Imperfections in the image quality and varying type fonts etc. can be handled by properly training the net, a task that would have been very difficult with legacy OCR algorithms where poor OCR results typically need to be mitigated by improving the input image quality rather than changing the algorithm.
46
Cadarso, Salamanca Manuel. "Influence of different frequencies order in a multi-step LSTM forecast for crowd movement in the domains of transportation and retail." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-254884.
Full textAbstract:
Denna avhandling presenterar ett tillvägagångssätt för att förutspå förflyttning inom folkmassor med hjälp av LSTM-neurala nätverk. Specifikt analyseras inflytandet som olika frekvenser av tidsserier har på både prognosen för folkmassorna och designen i arkitekturen inom transport och handel. Arkitekturen påverkas även då frekvensändringar provocerar fram en ökning eller minskning i datamängd och arkitekturen därför bör anpassas. Tidigare forskning inom prognoser relaterade till folkmassor har huvudsakligen fokuserat på att förutspå folkmassans nästa förflyttning snarare än att definiera mängden människor på en specifik plats under ett specifikt tidsspann. Dessa studier har använt olika tekniker som till exempel Random Forest eller Feed Forward neurala nätverk för att ta reda på inflytandet som de olika frekvenserna har över prognosens resultat. Denna avhandling tillämpar istället LSTM-neurala nätverk för analysering av detta inflytande och använder specifika fältrelaterade tekniker för att hitta de bästa parametrarna för att förutspå framtida välstånd i folkmassor. Resultatet visar att frekvensordningen i en tidsserie tydligt påverkar resultatet av prognoserna inom transport och handel, och att detta inflytande är positivt när frekvensordningen av tidsserierna kan fånga upp frekvensens form i prognosen. Därför, med frekvensordningen i åtanke, visar resultaten i prognoserna för de analyserade platserna en förbättring på 40% för SMAPE och 50% för RMSE jämfört med inhemska tillvägagångssätt och andra tekniker. Utöver detta visar de även att det finns ett samband mellan frekvensordningen och komponenterna i arkitekturerna.<br>This thesis presents an approach to predict crowd movement in defined placesusing LSTM neural networks. Specifically, it analyses the influence that different frequencies of time series have in both the crowd forecast and the design of the architecture in the domains of transportation and retail. The architecture is also affected because changes in the frequency provokes an increment or decrement in the quantity of data and, therefore, the architecture should be adapted. Previous research in the field of crowd prediction has been mainly focused on anticipating the next movement of the crowd rather than defining the amount of people during a specific range of time in a particular place. These studies have used different techniques such as Random Forest or Feed-Forward neural networks in order to find out the influence that the different frequencies have in the results of the forecast. However, this thesis applies LSTM neural networks for analysing this influence and uses specific field-related techniques in order to find the best parameters for forecasting future crowd movement. The results show that the order of the frequency of a time series clearly affects the outcomes of the predictions in the field of transportation and retail, being this influence positive when the order of the frequency of time series is able to catch the shape of the frequency of the forecast. Therefore, taking into account the order of the frequency, the results of the forecast for the analyzed places show an improvement of 40% for SMAPE and 50% for RMSE compared to the Naive approach and other techniques. Furthermore, they point out that there is a relation between the order of the frequency and the components of the architectures.
47
Sarika, Pawan Kumar. "Comparing LSTM and GRU for Multiclass Sentiment Analysis of Movie Reviews." Thesis, Blekinge Tekniska Högskola, Fakulteten för datavetenskaper, 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:bth-20213.
Full textAbstract:
Today, we are living in a data-driven world. Due to a surge in data generation, there is a need for efficient and accurate techniques to analyze data. One such kind of data which is needed to be analyzed are text reviews given for movies. Rather than classifying the reviews as positive or negative, we will classify the sentiment of the reviews on the scale of one to ten. In doing so, we will compare two recurrent neural network algorithms Long short term memory(LSTM) and Gated recurrent unit(GRU). The main objective of this study is to compare the accuracies of LSTM and GRU models. For training models, we collected data from two different sources. For filtering data, we used porter stemming and stop words. We coupled LSTM and GRU with the convolutional neural networks to increase the performance. After conducting experiments, we have observed that LSTM performed better in predicting border values. Whereas, GRU predicted every class equally. Overall GRU was able to predict multiclass text data of movie reviews slightly better than LSTM. GRU was computationally expansive when compared to LSTM.
48
Larsson, Klara, and Freja Ling. "Time Series forecasting of the SP Global Clean Energy Index using a Multivariate LSTM." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2021. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-301904.
Full textAbstract:
Clean energy and machine learning are subjects that play significant roles in shaping our future. The current climate crisis has forced the world to take action towards more sustainable solutions. Arrangements such as the UN’s Sustainable Development Goals and the Paris Agreement are causing an increased interest in renewable energy solutions. Further, the EU Taxonomy Regulation, applied in 2020, aims to scale up sustainable investments and to direct cash flows toward sustainable projects and activities. These measures create interest in investing in renewable energy alternatives and predicting future movements of stocks related to these businesses. Machine learning models have previously been used to predict time series with promising results. However, predicting time series in the form of stock price indices has, throughout previous attempts, proved to be a difficult task due to the complexity of the variables that play a role in the indices’ movements. This paper uses the machine learning algorithm long short-term memory (LSTM) to predict the S&P Global Clean Energy Index. The research question revolves around how well the LSTM model performs on this specific index and how the result is affected when past returns from correlating variables are added to the model. The researched variables are crude oil price, gold price, and interest. A model for each correlating variable was created, as well as one with all three, and one standard model which used only historical data from the index. The study found that while the model with the variable which had the strongest correlation performed best among the multivariate models, the standard model using only the target variable gave the most accurate result of any of the LSTM models.<br>Den pågående klimatkrisen har tvingat allt fler länder till att vidta åtgärder, och FN:s globala hållbarhetsmål och Parisavtalet ökar intresset för förnyelsebar energi. Vidare lanserade EU-kommissionen den 21 april 2021 ett omfattande åtgärdspaket, med syftet att öka investeringar i hållbara verksamheter. Detta skapar i sin tur ett ökat intresse för investeringar i förnyelsebar energi och metoder för att förutspå aktiepriser för dessa bolag. Maskininlärningsmodeller har tidigare använts för tidsserieanalyser med goda resultat, men att förutspå aktieindex har visat sig svårt till stor del på grund av uppgiftens komplexitet och antalet variabler som påverkar börsen. Den här uppsatsen använder sig av maskininlärningsmodellen long short-term memory (LSTM) för att förutspå S&P:s Global Clean Energy Index. Syftet är att ta reda på hur träffsäkert en LSTM-modell kan förutspå detta index, och hur resultatet påverkas då modellen används med ytterligare variabler som korrelerar med indexet. De variabler som undersöks är priset på råolja, priset på guld, och ränta. Modeller för var variabel skapades, samt en modell med samtliga variabler och en med endast historisk data från indexet. Resultatet visar att den modell med den variabel som korrelerar starkast med indexet presterade bäst bland flervariabelmodellerna, men den modell som endast användes med historisk data från indexet gav det mest träffsäkra resultatet.
49
Hernandez, Villapol Jorge Luis. "Spectrum Analysis and Prediction Using Long Short Term Memory Neural Networks and Cognitive Radios." Thesis, University of North Texas, 2017. https://digital.library.unt.edu/ark:/67531/metadc1062877/.
Full textAbstract:
One statement that we can make with absolute certainty in our current time is that wireless communication is now the standard and the de-facto type of communication. Cognitive radios are able to interpret the frequency spectrum and adapt. The aim of this work is to be able to predict whether a frequency channel is going to be busy or free in a specific time located in the future. To do this, the problem is modeled as a time series problem where each usage of a channel is treated as a sequence of busy and free slots in a fixed time frame. For this time series problem, the method being implemented is one of the latest, state-of-the-art, technique in machine learning for time series and sequence prediction: long short-term memory neural networks, or LSTMs.
50
Kindbom, Hannes. "LSTM vs Random Forest for Binary Classification of Insurance Related Text." Thesis, KTH, Matematisk statistik, 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-252748.
Full textAbstract:
The field of natural language processing has received increased attention lately, but less focus is put on comparing models, which differ in complexity. This thesis compares Random Forest to LSTM, for the task of classifying a message as question or non-question. The comparison was done by training and optimizing the models on historic chat data from the Swedish insurance company Hedvig. Different types of word embedding were also tested, such as Word2vec and Bag of Words. The results demonstrated that LSTM achieved slightly higher scores than Random Forest, in terms of F1 and accuracy. The models’ performance were not significantly improved after optimization and it was also dependent on which corpus the models were trained on. An investigation of how a chatbot would affect Hedvig’s adoption rate was also conducted, mainly by reviewing previous studies about chatbots’ effects on user experience. The potential effects on the innovation’s five attributes, relative advantage, compatibility, complexity, trialability and observability were analyzed to answer the problem statement. The results showed that the adoption rate of Hedvig could be positively affected, by improving the first two attributes. The effects a chatbot would have on complexity, trialability and observability were however suggested to be negligible, if not negative.<br>Det vetenskapliga området språkteknologi har fått ökad uppmärksamhet den senaste tiden, men mindre fokus riktas på att jämföra modeller som skiljer sig i komplexitet. Den här kandidatuppsatsen jämför Random Forest med LSTM, genom att undersöka hur väl modellerna kan användas för att klassificera ett meddelande som fråga eller icke-fråga. Jämförelsen gjordes genom att träna och optimera modellerna på historisk chattdata från det svenska försäkringsbolaget Hedvig. Olika typer av word embedding, så som Word2vec och Bag of Words, testades också. Resultaten visade att LSTM uppnådde något högre F1 och accuracy än Random Forest. Modellernas prestanda förbättrades inte signifikant efter optimering och resultatet var också beroende av vilket korpus modellerna tränades på. En undersökning av hur en chattbot skulle påverka Hedvigs adoption rate genomfördes också, huvudsakligen genom att granska tidigare studier om chattbotars effekt på användarupplevelsen. De potentiella effekterna på en innovations fem attribut, relativ fördel, kompatibilitet, komplexitet, prövbarhet and observerbarhet analyserades för att kunna svara på frågeställningen. Resultaten visade att Hedvigs adoption rate kan påverkas positivt, genom att förbättra de två första attributen. Effekterna en chattbot skulle ha på komplexitet, prövbarhet och observerbarhet ansågs dock vara försumbar, om inte negativ.