Relevant bibliographies by topics / Neural networks with LSTM / Journal articles

To see the other types of publications on this topic, follow the link: Neural networks with LSTM.

Journal articles on the topic 'Neural networks with LSTM'

Author: Grafiati

Published: 4 June 2021

Last updated: 1 February 2022

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles

Select a source type:

Consult the top 50 journal articles for your research on the topic 'Neural networks with LSTM.'

Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.

You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.

Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.

Bakir, Houda, Ghassen Chniti, and Hédi Zaher. "E-Commerce Price Forecasting Using LSTM Neural Networks." International Journal of Machine Learning and Computing 8, no. 2 (2018): 169–74. http://dx.doi.org/10.18178/ijmlc.2018.8.2.682.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Yu, Yong, Xiaosheng Si, Changhua Hu, and Jianxun Zhang. "A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures." Neural Computation 31, no. 7 (2019): 1235–70. http://dx.doi.org/10.1162/neco_a_01199.

Full text

Abstract:

Recurrent neural networks (RNNs) have been widely adopted in research areas concerned with sequential data, such as text, audio, and video. However, RNNs consisting of sigma cells or tanh cells are unable to learn the relevant information of input data when the input gap is large. By introducing gate functions into the cell structure, the long short-term memory (LSTM) could handle the problem of long-term dependencies well. Since its introduction, almost all the exciting results based on RNNs have been achieved by the LSTM. The LSTM has become the focus of deep learning. We review the LSTM cell and its variants to explore the learning capacity of the LSTM cell. Furthermore, the LSTM networks are divided into two broad categories: LSTM-dominated networks and integrated LSTM networks. In addition, their various applications are discussed. Finally, future research directions are presented for LSTM networks.

APA, Harvard, Vancouver, ISO, and other styles

Jia, YuKang, Zhicheng Wu, Yanyan Xu, Dengfeng Ke, and Kaile Su. "Long Short-Term Memory Projection Recurrent Neural Network Architectures for Piano’s Continuous Note Recognition." Journal of Robotics 2017 (2017): 1–7. http://dx.doi.org/10.1155/2017/2061827.

Full text

Abstract:

Long Short-Term Memory (LSTM) is a kind of Recurrent Neural Networks (RNN) relating to time series, which has achieved good performance in speech recogniton and image recognition. Long Short-Term Memory Projection (LSTMP) is a variant of LSTM to further optimize speed and performance of LSTM by adding a projection layer. As LSTM and LSTMP have performed well in pattern recognition, in this paper, we combine them with Connectionist Temporal Classification (CTC) to study piano’s continuous note recognition for robotics. Based on the Beijing Forestry University music library, we conduct experiments to show recognition rates and numbers of iterations of LSTM with a single layer, LSTMP with a single layer, and Deep LSTM (DLSTM, LSTM with multilayers). As a result, the single layer LSTMP proves performing much better than the single layer LSTM in both time and the recognition rate; that is, LSTMP has fewer parameters and therefore reduces the training time, and, moreover, benefiting from the projection layer, LSTMP has better performance, too. The best recognition rate of LSTMP is 99.8%. As for DLSTM, the recognition rate can reach 100% because of the effectiveness of the deep structure, but compared with the single layer LSTMP, DLSTM needs more training time.

APA, Harvard, Vancouver, ISO, and other styles

Kalinin, Maxim, Vasiliy Krundyshev, and Evgeny Zubkov. "Estimation of applicability of modern neural network methods for preventing cyberthreats to self-organizing network infrastructures of digital economy platforms,." SHS Web of Conferences 44 (2018): 00044. http://dx.doi.org/10.1051/shsconf/20184400044.

Full text

Abstract:

The problems of applying neural network methods for solving problems of preventing cyberthreats to flexible self-organizing network infrastructures of digital economy platforms: vehicle adhoc networks, wireless sensor networks, industrial IoT, “smart buildings” and “smart cities” are considered. The applicability of the classic perceptron neural network, recurrent, deep, LSTM neural networks and neural networks ensembles in the restricting conditions of fast training and big data processing are estimated. The use of neural networks with a complex architecture– recurrent and LSTM neural networks – is experimentally justified for building a system of intrusion detection for self-organizing network infrastructures.

APA, Harvard, Vancouver, ISO, and other styles

Wang, Hao, Xiaofang Zhang, Bin Liang, Qian Zhou, and Baowen Xu. "Gated Hierarchical LSTMs for Target-Based Sentiment Analysis." International Journal of Software Engineering and Knowledge Engineering 28, no. 11n12 (2018): 1719–37. http://dx.doi.org/10.1142/s0218194018400259.

Full text

Abstract:

In the field of target-based sentiment analysis, the deep neural model combining attention mechanism is a remarkable success. In current research, it is commonly seen that attention mechanism is combined with Long Short-Term Memory (LSTM) networks. However, such neural network-based architectures generally rely on complex computation and only focus on single target. In this paper, we propose a gated hierarchical LSTM (GH-LSTMs) model which combines regional LSTM and sentence-level LSTM via a gated operation for the task of target-based sentiment analysis. This approach can distinguish different polarities of sentiment of different targets in the same sentence through a regional LSTM. Furthermore, it is able to concentrate on the long-distance dependency of target in the whole sentence via a sentence-level LSTM. The final results of our experiments on multi-domain datasets of two languages from SemEval 2016 indicate that our approach yields better performance than Support Vector Machine (SVM) and several typical neural network models. A case study of some typical examples also makes a supplement to this conclusion.

APA, Harvard, Vancouver, ISO, and other styles

Pal, Subarno, Soumadip Ghosh, and Amitava Nag. "Sentiment Analysis in the Light of LSTM Recurrent Neural Networks." International Journal of Synthetic Emotions 9, no. 1 (2018): 33–39. http://dx.doi.org/10.4018/ijse.2018010103.

Full text

Abstract:

Long short-term memory (LSTM) is a special type of recurrent neural network (RNN) architecture that was designed over simple RNNs for modeling temporal sequences and their long-range dependencies more accurately. In this article, the authors work with different types of LSTM architectures for sentiment analysis of movie reviews. It has been showed that LSTM RNNs are more effective than deep neural networks and conventional RNNs for sentiment analysis. Here, the authors explore different architectures associated with LSTM models to study their relative performance on sentiment analysis. A simple LSTM is first constructed and its performance is studied. On subsequent stages, the LSTM layer is stacked one upon another which shows an increase in accuracy. Later the LSTM layers were made bidirectional to convey data both forward and backward in the network. The authors hereby show that a layered deep LSTM with bidirectional connections has better performance in terms of accuracy compared to the simpler versions of LSTM used here.

APA, Harvard, Vancouver, ISO, and other styles

Yu, Dian, and Shouqian Sun. "A Systematic Exploration of Deep Neural Networks for EDA-Based Emotion Recognition." Information 11, no. 4 (2020): 212. http://dx.doi.org/10.3390/info11040212.

Full text

Abstract:

Subject-independent emotion recognition based on physiological signals has become a research hotspot. Previous research has proved that electrodermal activity (EDA) signals are an effective data resource for emotion recognition. Benefiting from their great representation ability, an increasing number of deep neural networks have been applied for emotion recognition, and they can be classified as a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), or a combination of these (CNN+RNN). However, there has been no systematic research on the predictive power and configurations of different deep neural networks in this task. In this work, we systematically explore the configurations and performances of three adapted deep neural networks: ResNet, LSTM, and hybrid ResNet-LSTM. Our experiments use the subject-independent method to evaluate the three-class classification on the MAHNOB dataset. The results prove that the CNN model (ResNet) reaches a better accuracy and F1 score than the RNN model (LSTM) and the CNN+RNN model (hybrid ResNet-LSTM). Extensive comparisons also reveal that our three deep neural networks with EDA data outperform previous models with handcraft features on emotion recognition, which proves the great potential of the end-to-end DNN method.

APA, Harvard, Vancouver, ISO, and other styles

Dropka, Natasha, Stefan Ecklebe, and Martin Holena. "Real Time Predictions of VGF-GaAs Growth Dynamics by LSTM Neural Networks." Crystals 11, no. 2 (2021): 138. http://dx.doi.org/10.3390/cryst11020138.

Full text

Abstract:

The aim of this study was to assess the aptitude of the recurrent Long Short-Term Memory (LSTM) neural networks for fast and accurate predictions of process dynamics in vertical-gradient-freeze growth of gallium arsenide crystals (VGF-GaAs) using datasets generated by numerical transient simulations. Real time predictions of the temperatures and solid–liquid interface position in GaAs are crucial for control applications and for process visualization, i.e., for generation of digital twins. In the reported study, an LSTM network was trained on 1950 datasets with 2 external inputs and 6 outputs. Based on network performance criteria and training results, LSTMs showed the very accurate predictions of the VGF-GaAs growth process with median root-mean-square-error (RMSE) values of 2 × 10−3. This deep learning method achieved a superior predictive accuracy and timeliness compared with more traditional Nonlinear AutoRegressive eXogenous (NARX) recurrent networks.

APA, Harvard, Vancouver, ISO, and other styles

Xu, Lingfeng, Xiang Chen, Shuai Cao, Xu Zhang, and Xun Chen. "Feasibility Study of Advanced Neural Networks Applied to sEMG-Based Force Estimation." Sensors 18, no. 10 (2018): 3226. http://dx.doi.org/10.3390/s18103226.

Full text

Abstract:

To find out the feasibility of different neural networks in sEMG-based force estimation, in this paper, three types of networks, namely convolutional neural network (CNN), long short-term memory (LSTM) network and their combination (C-LSTM) were applied to predict muscle force generated in static isometric elbow flexion across three different circumstances (multi-subject, subject-dependent and subject-independent). Eight healthy men were recruited for the experiments, and the results demonstrated that all the three models were applicable for force estimation, and LSTM and C-LSTM achieved better performances. Even under subject-independent situation, they maintained mean RMSE% of as low as 9.07 ± 1.29 and 8.67 ± 1.14. CNN turned out to be a worse choice, yielding a mean RMSE% of 12.13 ± 1.98. To our knowledge, this work was the first to employ CNN, LSTM and C-LSTM in sEMG-based force estimation, and the results not only prove the strength of the proposed networks, but also pointed out a potential way of achieving high accuracy in real-time, subject-independent force estimation.

APA, Harvard, Vancouver, ISO, and other styles

Wan, Huaiyu, Shengnan Guo, Kang Yin, Xiaohui Liang, and Youfang Lin. "CTS-LSTM: LSTM-based neural networks for correlatedtime series prediction." Knowledge-Based Systems 191 (March 2020): 105239. http://dx.doi.org/10.1016/j.knosys.2019.105239.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Tra, Nguyen Ngoc, Ho Phuoc Tien, Nguyen Thanh Dat, and Nguyen Ngoc Vu. "VN-INDEX TREND PREDICTION USING LONG-SHORT TERM MEMORY NEURAL NETWORKS." Journal of Science and Technology: Issue on Information and Communications Technology 17, no. 12.2 (2019): 61. http://dx.doi.org/10.31130/ict-ud.2019.94.

Full text

Abstract:

The paper attemps to forecast the future trend of Vietnam index (VN-index) by using long-short term memory (LSTM) networks. In particular, an LSTM-based neural network is employed to study the temporal dependence in time-series data of past and present VN index values. Empirical forecasting results show that LSTM-based stock trend prediction offers an accuracy of about 60% which outperforms moving-average-based prediction.

APA, Harvard, Vancouver, ISO, and other styles

Gers, Felix A., Jürgen Schmidhuber, and Fred Cummins. "Learning to Forget: Continual Prediction with LSTM." Neural Computation 12, no. 10 (2000): 2451–71. http://dx.doi.org/10.1162/089976600300015015.

Full text

Abstract:

Long short-term memory (LSTM; Hochreiter & Schmidhuber, 1997) can solve numerous tasks not solvable by previous learning algorithms for recurrent neural networks (RNNs). We identify a weakness of LSTM networks processing continual input streams that are not a priori segmented into subsequences with explicitly marked ends at which the network's internal state could be reset. Without resets, the state may grow indefinitely and eventually cause the network to break down. Our remedy is a novel, adaptive “forget gate” that enables an LSTM cell to learn to reset itself at appropriate times, thus releasing internal resources. We review illustrative benchmark problems on which standard LSTM outperforms other RNN algorithms. All algorithms (including LSTM) fail to solve continual versions of these problems. LSTM with forget gates, however, easily solves them, and in an elegant way.

APA, Harvard, Vancouver, ISO, and other styles

Bucci, Andrea. "Realized Volatility Forecasting with Neural Networks." Journal of Financial Econometrics 18, no. 3 (2020): 502–31. http://dx.doi.org/10.1093/jjfinec/nbaa008.

Full text

Abstract:

Abstract In the last few decades, a broad strand of literature in finance has implemented artificial neural networks as a forecasting method. The major advantage of this approach is the possibility to approximate any linear and nonlinear behaviors without knowing the structure of the data generating process. This makes it suitable for forecasting time series which exhibit long-memory and nonlinear dependencies, like conditional volatility. In this article, the predictive performance of feed-forward and recurrent neural networks (RNNs) was compared, particularly focusing on the recently developed long short-term memory (LSTM) network and nonlinear autoregressive model process with eXogenous input (NARX) network, with traditional econometric approaches. The results show that RNNs are able to outperform all the traditional econometric methods. Additionally, capturing long-range dependence through LSTM and NARX models seems to improve the forecasting accuracy also in a highly volatile period.

APA, Harvard, Vancouver, ISO, and other styles

Wei, Jun, Fan Yang, Xiao-Chen Ren, and Silin Zou. "A Short-Term Prediction Model of PM2.5 Concentration Based on Deep Learning and Mode Decomposition Methods." Applied Sciences 11, no. 15 (2021): 6915. http://dx.doi.org/10.3390/app11156915.

Full text

Abstract:

Based on a set of deep learning and mode decomposition methods, a short-term prediction model for PM2.5 concentration for Beijing city is established in this paper. An ensemble empirical mode decomposition (EEMD) algorithm is first used to decompose the original PM2.5 timeseries to several high- to low-frequency intrinsic mode functions (IMFs). Each IMF component is then trained and predicted by a combination of three neural networks: back propagation network (BP), long short-term memory network (LSTM), and a hybrid network of a convolutional neural network (CNN) + LSTM. The results showed that both BP and LSTM are able to fit the low-frequency IMFs very well, and the total prediction errors of the summation of all IMFs are remarkably reduced from 21 g/m3 in the single BP model to 4.8 g/m3 in the EEMD + BP model. Spatial information from 143 stations surrounding Beijing city is extracted by CNN, which is then used to train the CNN+LSTM. It is found that, under extreme weather conditions of PM2.5 <35 g/m3 and PM2.5 >150 g/m3, the prediction errors of the CNN + LSTM model are improved by ~30% compared to the single LSTM model. However, the prediction of the very high-frequency IMF mode (IMF-1) remains a challenge for all neural networks, which might be due to microphysical turbulences and chaotic processes that cannot be resolved by the above-mentioned neural networks based on variable–variable relationship.

APA, Harvard, Vancouver, ISO, and other styles

Du, Shaohui, Zhenghan Chen, Haoyan Wu, Yihong Tang, and YuanQing Li. "Image Recommendation Algorithm Combined with Deep Neural Network Designed for Social Networks." Complexity 2021 (July 2, 2021): 1–9. http://dx.doi.org/10.1155/2021/5196190.

Full text

Abstract:

In recent years, deep neural networks have achieved great success in many fields, such as computer vision and natural language processing. Traditional image recommendation algorithms use text-based recommendation methods. The process of displaying images requires a lot of time and labor, and the time-consuming labor is inefficient. Therefore, this article mainly studies image recommendation algorithms based on deep neural networks in social networks. First, according to the time stamp information of the dataset, the interaction records of each user are sorted by the closest time. Then, some feature vectors are created via traditional feature algorithms like LBP, BGC3, RTU, or CNN extraction. For image recommendation, two LSTM neural networks are established, which accept these feature vectors as input, respectively. The compressed output of the two sub-ESTM neural networks is used as the input of another LSTM neural network. The multilayer regression algorithm is adopted to randomly sample some network nodes to obtain the cognitive information of the nodes sampled in the entire network, predict the relationship between all nodes in the network based on the cognitive information, and perform low sampling to achieve relationship prediction. The experiments show that proposed LSTM model together with CNN feature vectors can outperform other algorithms.

APA, Harvard, Vancouver, ISO, and other styles

Han, Shipeng, Zhen Meng, Xingcheng Zhang, and Yuepeng Yan. "Hybrid Deep Recurrent Neural Networks for Noise Reduction of MEMS-IMU with Static and Dynamic Conditions." Micromachines 12, no. 2 (2021): 214. http://dx.doi.org/10.3390/mi12020214.

Full text

Abstract:

Micro-electro-mechanical system inertial measurement unit (MEMS-IMU), a core component in many navigation systems, directly determines the accuracy of inertial navigation system; however, MEMS-IMU system is often affected by various factors such as environmental noise, electronic noise, mechanical noise and manufacturing error. These can seriously affect the application of MEMS-IMU used in different fields. Focus has been on MEMS gyro since it is an essential and, yet, complex sensor in MEMS-IMU which is very sensitive to noises and errors from the random sources. In this study, recurrent neural networks are hybridized in four different ways for noise reduction and accuracy improvement in MEMS gyro. These are two-layer homogenous recurrent networks built on long short term memory (LSTM-LSTM) and gated recurrent unit (GRU-GRU), respectively; and another two-layer but heterogeneous deep networks built on long short term memory-gated recurrent unit (LSTM-GRU) and a gated recurrent unit-long short term memory (GRU-LSTM). Practical implementation with static and dynamic experiments was carried out for a custom MEMS-IMU to validate the proposed networks, and the results show that GRU-LSTM seems to be overfitting large amount data testing for three-dimensional axis gyro in the static test. However, for X-axis and Y-axis gyro, LSTM-GRU had the best noise reduction effect with over 90% improvement in the three axes. For Z-axis gyroscope, LSTM-GRU performed better than LSTM-LSTM and GRU-GRU in quantization noise and angular random walk, while LSTM-LSTM shows better improvement than both GRU-GRU and LSTM-GRU networks in terms of zero bias stability. In the dynamic experiments, the Hilbert spectrum carried out revealed that time-frequency energy of the LSTM-LSTM, GRU-GRU, and GRU-LSTM denoising are higher compared to LSTM-GRU in terms of the whole frequency domain. Similarly, Allan variance analysis also shows that LSTM-GRU has a better denoising effect than the other networks in the dynamic experiments. Overall, the experimental results demonstrate the effectiveness of deep learning algorithms in MEMS gyro noise reduction, among which LSTM-GRU network shows the best noise reduction effect and great potential for application in the MEMS gyroscope area.

APA, Harvard, Vancouver, ISO, and other styles

Khalil, Kasem, Omar Eldash, Ashok Kumar, and Magdy Bayoumi. "Economic LSTM Approach for Recurrent Neural Networks." IEEE Transactions on Circuits and Systems II: Express Briefs 66, no. 11 (2019): 1885–89. http://dx.doi.org/10.1109/tcsii.2019.2924663.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Ergen, Tolga, and Suleyman Serdar Kozat. "Unsupervised Anomaly Detection With LSTM Neural Networks." IEEE Transactions on Neural Networks and Learning Systems 31, no. 8 (2020): 3127–41. http://dx.doi.org/10.1109/tnnls.2019.2935975.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Lee, Jaekyung, Hyunwoo Kim, and Hyungkyoo Kim. "Commercial Vacancy Prediction Using LSTM Neural Networks." Sustainability 13, no. 10 (2021): 5400. http://dx.doi.org/10.3390/su13105400.

Full text

Abstract:

Previous studies on commercial vacancy have mostly focused on the survival rate of commercial buildings over a certain time frame and the cause of their closure, due to a lack of appropriate data. Based on a time-series of 2,940,000 individual commercial facility data, the main purpose of this research is two-fold: (1) to examine long short-term memory (LSTM) as a feasible option for predicting trends in commercial districts and (2) to identify the influence of each variable on prediction results for establishing evidence-based decision-making on the primary influences of commercial vacancy. The results indicate that LSTM can be useful in simulating commercial vacancy dynamics. Furthermore, sales, floating population, and franchise rate were found to be the main determinants for commercial vacancy. The results suggest that it is imperative to control the cannibalization of commercial districts and develop their competitiveness to retain a consistent floating population.

APA, Harvard, Vancouver, ISO, and other styles

Victor, Nancy, and Daphne Lopez. "sl-LSTM." International Journal of Grid and High Performance Computing 12, no. 3 (2020): 1–16. http://dx.doi.org/10.4018/ijghpc.2020070101.

Full text

Abstract:

The volume of data in diverse data formats from various data sources has led the way for a new drift in the digital world, Big Data. This article proposes sl-LSTM (sequence labelling LSTM), a neural network architecture that combines the effectiveness of typical LSTM models to perform sequence labeling tasks. This is a bi-directional LSTM which uses stochastic gradient descent optimization and combines two features of the existing LSTM variants: coupled input-forget gates for reducing the computational complexity and peephole connections that allow all gates to inspect the current cell state. The model is tested on different datasets and the results show that the integration of various neural network models can further improve the efficiency of approach for identifying sensitive information in Big data.

APA, Harvard, Vancouver, ISO, and other styles

Wei, Chih-Chiang. "Comparison of River Basin Water Level Forecasting Methods: Sequential Neural Networks and Multiple-Input Functional Neural Networks." Remote Sensing 12, no. 24 (2020): 4172. http://dx.doi.org/10.3390/rs12244172.

Full text

Abstract:

To precisely forecast downstream water levels in catchment areas during typhoons, the deep learning artificial neural networks were employed to establish two water level forecasting models using sequential neural networks (SNNs) and multiple-input functional neural networks (MIFNNs). SNNs, which have a typical neural network structure, are network models constructed using sequential methods. To develop a network model capable of flexibly consolidating data, MIFNNs are employed for processing data from multiple sources or with multiple dimensions. Specifically, when images (e.g., radar reflectivity images) are used as input attributes, feature extraction is required to provide effective feature maps for model training. Therefore, convolutional layers and pooling layers were adopted to extract features. Long short-term memory (LSTM) layers adopted during model training enabled memory cell units to automatically determine the memory length, providing more useful information. The Hsintien River basin in northern Taiwan was selected as the research area and collected relevant data from 2011 to 2019. The input attributes comprised one-dimensional data (e.g., water levels at river stations, rain rates at rain gauges, and reservoir release) and two-dimensional data (i.e., radar reflectivity mosaics). Typhoons Saola, Soudelor, Dujuan, and Megi were selected, and the water levels 1 to 6 h after the typhoons struck were forecasted. The results indicated that compared with linear regressions (REG), SNN using dense layers (SNN-Dense), and SNN using LSTM layers (SNN-LSTM) models, superior forecasting results were achieved for the MIFNN model. Thus, the MIFNN model, as the optimal model for water level forecasting, was identified.

APA, Harvard, Vancouver, ISO, and other styles

Pan, Yu, Jing Xu, Maolin Wang, et al. "Compressing Recurrent Neural Networks with Tensor Ring for Action Recognition." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 4683–90. http://dx.doi.org/10.1609/aaai.v33i01.33014683.

Full text

Abstract:

Recurrent Neural Networks (RNNs) and their variants, such as Long-Short Term Memory (LSTM) networks, and Gated Recurrent Unit (GRU) networks, have achieved promising performance in sequential data modeling. The hidden layers in RNNs can be regarded as the memory units, which are helpful in storing information in sequential contexts. However, when dealing with high dimensional input data, such as video and text, the input-to-hidden linear transformation in RNNs brings high memory usage and huge computational cost. This makes the training of RNNs very difficult. To address this challenge, we propose a novel compact LSTM model, named as TR-LSTM, by utilizing the low-rank tensor ring decomposition (TRD) to reformulate the input-to-hidden transformation. Compared with other tensor decomposition methods, TR-LSTM is more stable. In addition, TR-LSTM can complete an end-to-end training and also provide a fundamental building block for RNNs in handling large input data. Experiments on real-world action recognition datasets have demonstrated the promising performance of the proposed TR-LSTM compared with the tensor-train LSTM and other state-of-the-art competitors.

APA, Harvard, Vancouver, ISO, and other styles

Wang, Qinghua, Yuexiao Yu, Hosameldin O. A. Ahmed, Mohamed Darwish, and Asoke K. Nandi. "Open-Circuit Fault Detection and Classification of Modular Multilevel Converters in High Voltage Direct Current Systems (MMC-HVDC) with Long Short-Term Memory (LSTM) Method." Sensors 21, no. 12 (2021): 4159. http://dx.doi.org/10.3390/s21124159.

Full text

Abstract:

Fault detection and classification are two of the challenging tasks in Modular Multilevel Converters in High Voltage Direct Current (MMC-HVDC) systems. To directly classify the raw sensor data without certain feature extraction and classifier design, a long short-term memory (LSTM) neural network is proposed and used for seven states of the MMC-HVDC transmission power system simulated by Power Systems Computer Aided Design/Electromagnetic Transients including DC (PSCAD/EMTDC). It is observed that the LSTM method can detect faults with 100% accuracy and classify different faults as well as provide promising fault classification performance. Compared with a bidirectional LSTM (BiLSTM), the LSTM can get similar classification accuracy, requiring less training time and testing time. Compared with Convolutional Neural Networks (CNN) and AutoEncoder-based deep neural networks (AE-based DNN), the LSTM method can get better classification accuracy around the middle of the testing data proportion, but it needs more training time.

APA, Harvard, Vancouver, ISO, and other styles

Nguyen, Viet-Hung, Minh-Tuan Nguyen, Jeongsik Choi, and Yong-Hwa Kim. "NLOS Identification in WLANs Using Deep LSTM with CNN Features." Sensors 18, no. 11 (2018): 4057. http://dx.doi.org/10.3390/s18114057.

Full text

Abstract:

Identifying channel states as line-of-sight or non-line-of-sight helps to optimize location-based services in wireless communications. The received signal strength identification and channel state information are used to estimate channel conditions for orthogonal frequency division multiplexing systems in indoor wireless local area networks. This paper proposes a joint convolutional neural network and recurrent neural network architecture to classify channel conditions. Convolutional neural networks extract the feature from frequency-domain characteristics of channel state information data and recurrent neural networks extract the feature from time-varying characteristics of received signal strength identification and channel state information between packet transmissions. The performance of the proposed methods is verified under indoor propagation environments. Experimental results show that the proposed method has a 2% improvement in classification performance over the conventional recurrent neural network model.

APA, Harvard, Vancouver, ISO, and other styles

Wang, Geng, Xuemin Yao, Jianjun Cui, Yonggang Yan, Jun Dai, and Wu Zhao. "A novel piezoelectric hysteresis modeling method combining LSTM and NARX neural networks." Modern Physics Letters B 34, no. 28 (2020): 2050306. http://dx.doi.org/10.1142/s0217984920503066.

Full text

Abstract:

In order to study the hysteresis nonlinear characteristics of piezoelectric actuators, a novel hybrid modeling method based on Long-Short-Term Memory (LSTM) and Nonlinear autoregressive with external input (NARX) neural networks is proposed. First, the input–output curve between the applied voltage and the produced angle of a piezoelectric tip/tilt mirror is measured. Second, two hysteresis models named LSTM and NARX neural networks were, respectively, established mathematically, and then were tested and verified experimentally. Third, a novel adaptive weighted hybrid hysteresis model which combines LSTM and NARX neural networks was proposed through analyzing and comparing the unique characteristics of the above two hysteresis models. The proposed hybrid model combines LSTM’s ability to approximate nonlinear static hysteresis and NARX’s high dynamic-fitting ability. Experimental results show that the RMS errors of the hybrid model are smaller than those of LSTM model and NARX model. That is to say, the proposed hybrid model has a relatively high accuracy. Compared with the traditional differential equation-based and operator-based hysteresis models, the presented hybrid neural network method has higher flexibility and accuracy in modeling performance, and is a more promising method for modeling piezoelectric hysteresis.

APA, Harvard, Vancouver, ISO, and other styles

Wei, Xiaolu, Binbin Lei, Hongbing Ouyang, and Qiufeng Wu. "Stock Index Prices Prediction via Temporal Pattern Attention and Long-Short-Term Memory." Advances in Multimedia 2020 (December 10, 2020): 1–7. http://dx.doi.org/10.1155/2020/8831893.

Full text

Abstract:

This study attempts to predict stock index prices using multivariate time series analysis. The study’s motivation is based on the notion that datasets of stock index prices involve weak periodic patterns, long-term and short-term information, for which traditional approaches and current neural networks such as Autoregressive models and Support Vector Machine (SVM) may fail. This study applied Temporal Pattern Attention and Long-Short-Term Memory (TPA-LSTM) for prediction to overcome the issue. The results show that stock index prices prediction through the TPA-LSTM algorithm could achieve better prediction performance over traditional deep neural networks, such as recurrent neural network (RNN), convolutional neural network (CNN), and long and short-term time series network (LSTNet).

APA, Harvard, Vancouver, ISO, and other styles

Zhou, Lixia, Xia Chen, Runsha Dong, and Shan Yang. "Hotspots Prediction Based on LSTM Neural Network for Cellular Networks." Journal of Physics: Conference Series 1624 (October 2020): 052016. http://dx.doi.org/10.1088/1742-6596/1624/5/052016.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Opałka, Sławomir, Dominik Szajerman, and Adam Wojciechowski. "LSTM multichannel neural networks in mental task classification." COMPEL - The international journal for computation and mathematics in electrical and electronic engineering 38, no. 4 (2019): 1204–13. http://dx.doi.org/10.1108/compel-10-2018-0429.

Full text

Abstract:

Purpose The purpose of this paper is to apply recurrent neural networks (RNNs) and more specifically long-short term memory (LSTM)-based ones for mental task classification in terms of BCI systems. The authors have introduced novel LSTM-based multichannel architecture model which proved to be highly promising in other fields, yet was not used for mental tasks classification. Design/methodology/approach Validity of the multichannel LSTM-based solution was confronted with the results achieved by a non-multichannel state-of-the-art solutions on a well-recognized data set. Findings The results demonstrated evident advantage of the introduced method. The best of the provided variants outperformed most of the RNNs approaches and was comparable with the best state-of-the-art methods. Practical implications The approach presented in the manuscript enables more detailed investigation of the electroencephalography analysis methods, invaluable for BCI mental tasks classification. Originality/value The new approach to mental task classification, exploiting LSTM-based RNNs with multichannel architecture, operating on spatial features retrieving filters, has been adapted to mental tasks with noticeable results. To the best of the authors’ knowledge, such an approach was not present in the literature before.

APA, Harvard, Vancouver, ISO, and other styles

Yolchuyeva, Sevinj, Géza Németh, and Bálint Gyires-Tóth. "Grapheme-to-Phoneme Conversion with Convolutional Neural Networks." Applied Sciences 9, no. 6 (2019): 1143. http://dx.doi.org/10.3390/app9061143.

Full text

Abstract:

Grapheme-to-phoneme (G2P) conversion is the process of generating pronunciation for words based on their written form. It has a highly essential role for natural language processing, text-to-speech synthesis and automatic speech recognition systems. In this paper, we investigate convolutional neural networks (CNN) for G2P conversion. We propose a novel CNN-based sequence-to-sequence (seq2seq) architecture for G2P conversion. Our approach includes an end-to-end CNN G2P conversion with residual connections and, furthermore, a model that utilizes a convolutional neural network (with and without residual connections) as encoder and Bi-LSTM as a decoder. We compare our approach with state-of-the-art methods, including Encoder-Decoder LSTM and Encoder-Decoder Bi-LSTM. Training and inference times, phoneme and word error rates were evaluated on the public CMUDict dataset for US English, and the best performing convolutional neural network-based architecture was also evaluated on the NetTalk dataset. Our method approaches the accuracy of previous state-of-the-art results in terms of phoneme error rate.

APA, Harvard, Vancouver, ISO, and other styles

Vlachas, Pantelis R., Wonmin Byeon, Zhong Y. Wan, Themistoklis P. Sapsis, and Petros Koumoutsakos. "Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks." Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 474, no. 2213 (2018): 20170844. http://dx.doi.org/10.1098/rspa.2017.0844.

Full text

Abstract:

We introduce a data-driven forecasting method for high-dimensional chaotic systems using long short-term memory (LSTM) recurrent neural networks. The proposed LSTM neural networks perform inference of high-dimensional dynamical systems in their reduced order space and are shown to be an effective set of nonlinear approximators of their attractor. We demonstrate the forecasting performance of the LSTM and compare it with Gaussian processes (GPs) in time series obtained from the Lorenz 96 system, the Kuramoto–Sivashinsky equation and a prototype climate model. The LSTM networks outperform the GPs in short-term forecasting accuracy in all applications considered. A hybrid architecture, extending the LSTM with a mean stochastic model (MSM–LSTM), is proposed to ensure convergence to the invariant measure. This novel hybrid method is fully data-driven and extends the forecasting capabilities of LSTM networks.

APA, Harvard, Vancouver, ISO, and other styles

Qin, Huafeng, and Peng Wang. "Finger-Vein Verification Based on LSTM Recurrent Neural Networks." Applied Sciences 9, no. 8 (2019): 1687. http://dx.doi.org/10.3390/app9081687.

Full text

Abstract:

Finger-vein biometrics has been extensively investigated for personal verification. A challenge is that the finger-vein acquisition is affected by many factors, which results in many ambiguous regions in the finger-vein image. Generally, the separability between vein and background is poor in such regions. Despite recent advances in finger-vein pattern segmentation, current solutions still lack the robustness to extract finger-vein features from raw images because they do not take into account the complex spatial dependencies of vein pattern. This paper proposes a deep learning model to extract vein features by combining the Convolutional Neural Networks (CNN) model and Long Short-Term Memory (LSTM) model. Firstly, we automatically assign the label based on a combination of known state of the art handcrafted finger-vein image segmentation techniques, and generate various sequences for each labeled pixel along different directions. Secondly, several Stacked Convolutional Neural Networks and Long Short-Term Memory (SCNN-LSTM) models are independently trained on the resulting sequences. The outputs of various SCNN-LSTMs form a complementary and over-complete representation and are conjointly put into Probabilistic Support Vector Machine (P-SVM) to predict the probability of each pixel of being foreground (i.e., vein pixel) given several sequences centered on it. Thirdly, we propose a supervised encoding scheme to extract the binary vein texture. A threshold is automatically computed by taking into account the maximal separation between the inter-class distance and the intra-class distance. In our approach, the CNN learns robust features for vein texture pattern representation and LSTM stores the complex spatial dependencies of vein patterns. So, the pixels in any region of a test image can then be classified effectively. In addition, the supervised information is employed to encode the vein patterns, so the resulting encoding images contain more discriminating features. The experimental results on one public finger-vein database show that the proposed approach significantly improves the finger-vein verification accuracy.

APA, Harvard, Vancouver, ISO, and other styles

Shewalkar, Apeksha, Deepika Nyavanandi, and Simone A. Ludwig. "Performance Evaluation of Deep Neural Networks Applied to Speech Recognition: RNN, LSTM and GRU." Journal of Artificial Intelligence and Soft Computing Research 9, no. 4 (2019): 235–45. http://dx.doi.org/10.2478/jaiscr-2019-0006.

Full text

Abstract:

Abstract Deep Neural Networks (DNN) are nothing but neural networks with many hidden layers. DNNs are becoming popular in automatic speech recognition tasks which combines a good acoustic with a language model. Standard feedforward neural networks cannot handle speech data well since they do not have a way to feed information from a later layer back to an earlier layer. Thus, Recurrent Neural Networks (RNNs) have been introduced to take temporal dependencies into account. However, the shortcoming of RNNs is that long-term dependencies due to the vanishing/exploding gradient problem cannot be handled. Therefore, Long Short-Term Memory (LSTM) networks were introduced, which are a special case of RNNs, that takes long-term dependencies in a speech in addition to short-term dependencies into account. Similarily, GRU (Gated Recurrent Unit) networks are an improvement of LSTM networks also taking long-term dependencies into consideration. Thus, in this paper, we evaluate RNN, LSTM, and GRU to compare their performances on a reduced TED-LIUM speech data set. The results show that LSTM achieves the best word error rates, however, the GRU optimization is faster while achieving word error rates close to LSTM.

APA, Harvard, Vancouver, ISO, and other styles

Yin, Aijun, Yinghua Yan, Zhiyu Zhang, Chuan Li, and René-Vinicio Sánchez. "Fault Diagnosis of Wind Turbine Gearbox Based on the Optimized LSTM Neural Network with Cosine Loss." Sensors 20, no. 8 (2020): 2339. http://dx.doi.org/10.3390/s20082339.

Full text

Abstract:

The gearbox is one of the most fragile parts of a wind turbine (WT). Fault diagnosis of the WT gearbox is of great importance to reduce operation and maintenance (O&M) costs and improve cost-effectiveness. At present, intelligent fault diagnosis methods based on long short-term memory (LSTM) networks have been widely adopted. As the traditional softmax loss of an LSTM network usually lacks the power of discrimination, this paper proposes a fault diagnosis method for wind turbine gearboxes based on optimized LSTM neural networks with cosine loss (Cos-LSTM). The loss can be converted from Euclid space to angular space by cosine loss, thus eliminating the effect of signal strength and improve the diagnosis accuracy. The energy sequence features and the wavelet energy entropy of the vibration signals are used to evaluate the Cos-LSTM networks. The effectiveness of the proposed method is verified with the fault vibration data collected on a gearbox fault diagnosis experimental platform. In addition, the Cos-LSTM method is also compared with other classic fault diagnosis techniques. The results demonstrate that the Cos-LSTM has better performance for gearbox fault diagnosis.

APA, Harvard, Vancouver, ISO, and other styles

Bilgera, Christian, Akifumi Yamamoto, Maki Sawano, Haruka Matsukura, and Hiroshi Ishida. "Application of Convolutional Long Short-Term Memory Neural Networks to Signals Collected from a Sensor Network for Autonomous Gas Source Localization in Outdoor Environments." Sensors 18, no. 12 (2018): 4484. http://dx.doi.org/10.3390/s18124484.

Full text

Abstract:

Convolutional Long Short-Term Memory Neural Networks (CNN-LSTM) are a variant of recurrent neural networks (RNN) that can extract spatial features in addition to classifying or making predictions from sequential data. In this paper, we analyzed the use of CNN-LSTM for gas source localization (GSL) in outdoor environments using time series data from a gas sensor network and anemometer. CNN-LSTM is used to estimate the location of a gas source despite the challenges created from inconsistent airflow and gas distribution in outdoor environments. To train CNN-LSTM for GSL, we used temporal data taken from a 5 × 6 metal oxide semiconductor (MOX) gas sensor array, spaced 1.5 m apart, and an anemometer placed in the center of the sensor array in an open area outdoors. The output of the CNN-LSTM is one of thirty cells approximating the location of a gas source. We show that by using CNN-LSTM, we were able to determine the location of a gas source from sequential data. In addition, we compared several artificial neural network (ANN) architectures as well as trained them without wind vector data to estimate the complexity of the task. We found that ANN is a promising prospect for GSL tasks.

APA, Harvard, Vancouver, ISO, and other styles

Liu, Tianyuan, Jinsong Bao, Junliang Wang, and Yiming Zhang. "A Hybrid CNN–LSTM Algorithm for Online Defect Recognition of CO2 Welding." Sensors 18, no. 12 (2018): 4369. http://dx.doi.org/10.3390/s18124369.

Full text

Abstract:

At present, realizing high-quality automatic welding through online monitoring is a research focus in engineering applications. In this paper, a CNN–LSTM algorithm is proposed, which combines the advantages of convolutional neural networks (CNNs) and long short-term memory networks (LSTMs). The CNN–LSTM algorithm establishes a shallow CNN to extract the primary features of the molten pool image. Then the feature tensor extracted by the CNN is transformed into the feature matrix. Finally, the rows of the feature matrix are fed into the LSTM network for feature fusion. This process realizes the implicit mapping from molten pool images to welding defects. The test results on the self-made molten pool image dataset show that CNN contributes to the overall feasibility of the CNN–LSTM algorithm and LSTM network is the most superior in the feature hybrid stage. The algorithm converges at 300 epochs and the accuracy of defects detection in CO2 welding molten pool is 94%. The processing time of a single image is 0.067 ms, which fully meets the real-time monitoring requirement based on molten pool image. The experimental results on the MNIST and FashionMNIST datasets show that the algorithm is universal and can be used for similar image recognition and classification tasks.

APA, Harvard, Vancouver, ISO, and other styles

Narayanan, Barath Narayanan, and Venkata Salini Priyamvada Davuluru. "Ensemble Malware Classification System Using Deep Neural Networks." Electronics 9, no. 5 (2020): 721. http://dx.doi.org/10.3390/electronics9050721.

Full text

Abstract:

With the advancement of technology, there is a growing need of classifying malware programs that could potentially harm any computer system and/or smaller devices. In this research, an ensemble classification system comprising convolutional and recurrent neural networks is proposed to distinguish malware programs. Microsoft’s Malware Classification Challenge (BIG 2015) dataset with nine distinct classes is utilized for this study. This dataset contains an assembly file and a compiled file for each malware program. Compiled files are visualized as images and are classified using Convolutional Neural Networks (CNNs). Assembly files consist of machine language opcodes that are distinguished among classes using Long Short-Term Memory (LSTM) networks after converting them into sequences. In addition, features are extracted from these architectures (CNNs and LSTM) and are classified using a support vector machine or logistic regression. An accuracy of 97.2% is achieved using LSTM network for distinguishing assembly files, 99.4% using CNN architecture for classifying compiled files and an overall accuracy of 99.8% using the proposed ensemble approach thereby setting a new benchmark. An independent and automated classification system for assembly and/or compiled files provides the luxury to anti-malware industry experts to choose the type of system depending on their available computational resources.

APA, Harvard, Vancouver, ISO, and other styles

González-Enrique, Javier, Juan Jesús Ruiz-Aguilar, José Antonio Moscoso-López, Daniel Urda, Lipika Deka, and Ignacio J. Turias. "Artificial Neural Networks, Sequence-to-Sequence LSTMs, and Exogenous Variables as Analytical Tools for NO2 (Air Pollution) Forecasting: A Case Study in the Bay of Algeciras (Spain)." Sensors 21, no. 5 (2021): 1770. http://dx.doi.org/10.3390/s21051770.

Full text

Abstract:

This study aims to produce accurate predictions of the NO2 concentrations at a specific station of a monitoring network located in the Bay of Algeciras (Spain). Artificial neural networks (ANNs) and sequence-to-sequence long short-term memory networks (LSTMs) were used to create the forecasting models. Additionally, a new prediction method was proposed combining LSTMs using a rolling window scheme with a cross-validation procedure for time series (LSTM-CVT). Two different strategies were followed regarding the input variables: using NO2 from the station or employing NO2 and other pollutants data from any station of the network plus meteorological variables. The ANN and LSTM-CVT exogenous models used lagged datasets of different window sizes. Several feature ranking methods were used to select the top lagged variables and include them in the final exogenous datasets. Prediction horizons of t + 1, t + 4 and t + 8 were employed. The exogenous variables inclusion enhanced the model’s performance, especially for t + 4 (ρ ≈ 0.68 to ρ ≈ 0.74) and t + 8 (ρ ≈ 0.59 to ρ ≈ 0.66). The proposed LSTM-CVT method delivered promising results as the best performing models per prediction horizon employed this new methodology. Additionally, per each parameter combination, it obtained lower error values than ANNs in 85% of the cases.

APA, Harvard, Vancouver, ISO, and other styles

Xu, Xijie, Xiaoping Rui, Yonglei Fan, Tian Yu, and Yiwen Ju. "Forecasting of Coalbed Methane Daily Production Based on T-LSTM Neural Networks." Symmetry 12, no. 5 (2020): 861. http://dx.doi.org/10.3390/sym12050861.

Full text

Abstract:

Accurately forecasting the daily production of coalbed methane (CBM) is important forformulating associated drainage parameters and evaluating the economic benefit of CBM mining. Daily production of CBM depends on many factors, making it difficult to predict using conventional mathematical models. Because traditional methods do not reflect the long-term time series characteristics of CBM production, this study first used a long short-term memory neural network (LSTM) and transfer learning (TL) method for time series forecasting of CBM daily production. Based on the LSTM model, we introduced the idea of transfer learning and proposed a Transfer-LSTM (T-LSTM) CBM production forecasting model. This approach first uses a large amount of data similar to the target to pretrain the weights of the LSTM network, then uses transfer learning to fine-tune LSTM network parameters a second time, so as to obtain the final T-LSTM model. Experiments were carried out using daily CBM production data for the Panhe Demonstration Zone at southern Qinshui basin in China. Based on the results, the idea of transfer learning can solve the problem of insufficient samples during LSTM training. Prediction results for wells that entered the stable period earlier were more accurate, whereas results for types with unstable production in the early stage require further exploration. Because CBM wells daily production data have symmetrical similarities, which can provide a reference for the prediction of other wells, so our proposed T-LSTM network can achieve good results for the production forecast and can provide guidance for forecasting production of CBM wells.

APA, Harvard, Vancouver, ISO, and other styles

Wan, Renzhuo, Shuping Mei, Jun Wang, Min Liu, and Fan Yang. "Multivariate Temporal Convolutional Network: A Deep Neural Networks Approach for Multivariate Time Series Forecasting." Electronics 8, no. 8 (2019): 876. http://dx.doi.org/10.3390/electronics8080876.

Full text

Abstract:

Multivariable time series prediction has been widely studied in power energy, aerology, meteorology, finance, transportation, etc. Traditional modeling methods have complex patterns and are inefficient to capture long-term multivariate dependencies of data for desired forecasting accuracy. To address such concerns, various deep learning models based on Recurrent Neural Network (RNN) and Convolutional Neural Network (CNN) methods are proposed. To improve the prediction accuracy and minimize the multivariate time series data dependence for aperiodic data, in this article, Beijing PM2.5 and ISO-NE Dataset are analyzed by a novel Multivariate Temporal Convolution Network (M-TCN) model. In this model, multi-variable time series prediction is constructed as a sequence-to-sequence scenario for non-periodic datasets. The multichannel residual blocks in parallel with asymmetric structure based on deep convolution neural network is proposed. The results are compared with rich competitive algorithms of long short term memory (LSTM), convolutional LSTM (ConvLSTM), Temporal Convolution Network (TCN) and Multivariate Attention LSTM-FCN (MALSTM-FCN), which indicate significant improvement of prediction accuracy, robust and generalization of our model.

APA, Harvard, Vancouver, ISO, and other styles

Lobacheva, Ekaterina, Nadezhda Chirkova, Alexander Markovich, and Dmitry Vetrov. "Structured Sparsification of Gated Recurrent Neural Networks." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (2020): 4989–96. http://dx.doi.org/10.1609/aaai.v34i04.5938.

Full text

Abstract:

One of the most popular approaches for neural network compression is sparsification — learning sparse weight matrices. In structured sparsification, weights are set to zero by groups corresponding to structure units, e. g. neurons. We further develop the structured sparsification approach for the gated recurrent neural networks, e. g. Long Short-Term Memory (LSTM). Specifically, in addition to the sparsification of individual weights and neurons, we propose sparsifying the preactivations of gates. This makes some gates constant and simplifies an LSTM structure. We test our approach on the text classification and language modeling tasks. Our method improves the neuron-wise compression of the model in most of the tasks. We also observe that the resulting structure of gate sparsity depends on the task and connect the learned structures to the specifics of the particular tasks.

APA, Harvard, Vancouver, ISO, and other styles

Dangovski, Rumen, Li Jing, Preslav Nakov, Mićo Tatalović, and Marin Soljačić. "Rotational Unit of Memory: A Novel Representation Unit for RNNs with Scalable Applications." Transactions of the Association for Computational Linguistics 7 (November 2019): 121–38. http://dx.doi.org/10.1162/tacl_a_00258.

Full text

Abstract:

Stacking long short-term memory (LSTM) cells or gated recurrent units (GRUs) as part of a recurrent neural network (RNN) has become a standard approach to solving a number of tasks ranging from language modeling to text summarization. Although LSTMs and GRUs were designed to model long-range dependencies more accurately than conventional RNNs, they nevertheless have problems copying or recalling information from the long distant past. Here, we derive a phase-coded representation of the memory state, Rotational Unit of Memory (RUM), that unifies the concepts of unitary learning and associative memory. We show experimentally that RNNs based on RUMs can solve basic sequential tasks such as memory copying and memory recall much better than LSTMs/GRUs. We further demonstrate that by replacing LSTM/GRU with RUM units we can apply neural networks to real-world problems such as language modeling and text summarization, yielding results comparable to the state of the art.

APA, Harvard, Vancouver, ISO, and other styles

Tedla, Yemane, and Kazuhide Yamamoto. "Morphological Segmentation with LSTM Neural Networks for Tigrinya." International Journal on Natural Language Computing 7, no. 2 (2018): 29–44. http://dx.doi.org/10.5121/ijnlc.2018.7203.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Masouros, Dimosthenis, Sotirios Xydis, and Dimitrios Soudris. "Rusty: Runtime System Predictability Leveraging LSTM Neural Networks." IEEE Computer Architecture Letters 18, no. 2 (2019): 103–6. http://dx.doi.org/10.1109/lca.2019.2924622.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Gonzalez, Jesús, and Wen Yu. "Non-linear system modeling using LSTM neural networks." IFAC-PapersOnLine 51, no. 13 (2018): 485–89. http://dx.doi.org/10.1016/j.ifacol.2018.07.326.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Mirza, Ali H., Mine Kerpicci, and Suleyman S. Kozat. "Efficient online learning with improved LSTM neural networks." Digital Signal Processing 102 (July 2020): 102742. http://dx.doi.org/10.1016/j.dsp.2020.102742.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Men, Lu, Noyan Ilk, Xinlin Tang, and Yuan Liu. "Multi-disease prediction using LSTM recurrent neural networks." Expert Systems with Applications 177 (September 2021): 114905. http://dx.doi.org/10.1016/j.eswa.2021.114905.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Wei, Chih-Chiang. "Development of Stacked Long Short-Term Memory Neural Networks with Numerical Solutions for Wind Velocity Predictions." Advances in Meteorology 2020 (July 23, 2020): 1–18. http://dx.doi.org/10.1155/2020/5462040.

Full text

Abstract:

Taiwan, being located on a path in the west Pacific Ocean where typhoons often strike, is often affected by typhoons. The accompanying strong winds and torrential rains make typhoons particularly damaging in Taiwan. Therefore, we aimed to establish an accurate wind speed prediction model for future typhoons, allowing for better preparation to mitigate a typhoon’s toll on life and property. For more accurate wind speed predictions during a typhoon episode, we used cutting-edge machine learning techniques to construct a wind speed prediction model. To ensure model accuracy, we used, as variable input, simulated values from the Weather Research and Forecasting model of the numerical weather prediction system in addition to adopting deeper neural networks that can deepen neural network structures in the construction of estimation models. Our deeper neural networks comprise multilayer perceptron (MLP), deep recurrent neural networks (DRNNs), and stacked long short-term memory (LSTM). These three model-structure types differ by their memory capacity: MLPs are model networks with no memory capacity, whereas DRNNs and stacked LSTM are model networks with memory capacity. A model structure with memory capacity can analyze time-series data and continue memorizing and learning along the time axis. The study area is northeastern Taiwan. Results showed that MLP, DRNN, and stacked LSTM prediction error rates increased with prediction time (1–6 hours). Comparing the three models revealed that model networks with memory capacity (DRNN and stacked LSTM) were more accurate than those without memory capacity. A further comparison of model networks with memory capacity revealed that stacked LSTM yielded slightly more accurate results than did DRNN. Additionally, we determined that in the construction of the wind speed prediction model, the use of numerically simulated values reduced the error rate approximately by 30%. These results indicate that the inclusion of numerically simulated values in wind speed prediction models enhanced their prediction accuracy.

APA, Harvard, Vancouver, ISO, and other styles

Kang, Jinle, Huimin Wang, Feifei Yuan, Zhiqiang Wang, Jing Huang, and Tian Qiu. "Prediction of Precipitation Based on Recurrent Neural Networks in Jingdezhen, Jiangxi Province, China." Atmosphere 11, no. 3 (2020): 246. http://dx.doi.org/10.3390/atmos11030246.

Full text

Abstract:

Precipitation is a critical input for hydrologic simulation and prediction, and is widely used for agriculture, water resources management, and prediction of flood and drought, among other activities. Traditional precipitation prediction researches often established one or more probability models of historical data based on the statistical prediction methods and machine learning techniques. However, few studies have been attempted deep learning methods such as the state-of-the-art for Recurrent Neural Networks (RNNs) networks in meteorological sequence time series predictions. We deployed Long Short-Term Memory (LSTM) network models for predicting the precipitation based on meteorological data from 2008 to 2018 in Jingdezhen City. After identifying the correlation between meteorological variables and the precipitation, nine significant input variables were selected to construct the LSTM model. Then, the selected meteorological variables were refined by the relative importance of input variables to reconstruct the LSTM model. Finally, the LSTM model with final selected input variables is used to predict the precipitation and the performance is compared with other classical statistical algorithms and the machine learning algorithms. The experimental results show that the LSTM is suitable for precipitation prediction. The RNN models, combined with meteorological variables, could predict the precipitation accurately in Jingdezhen City and provide sufficient time to prepare strategies against potential related disasters.

APA, Harvard, Vancouver, ISO, and other styles

He, Zhen, Shaobing Gao, Liang Xiao, Daxue Liu, and Hangen He. "Multimedia Data Modelling Using Multidimensional Recurrent Neural Networks." Symmetry 10, no. 9 (2018): 370. http://dx.doi.org/10.3390/sym10090370.

Full text

Abstract:

Modelling the multimedia data such as text, images, or videos usually involves the analysis, prediction, or reconstruction of them. The recurrent neural network (RNN) is a powerful machine learning approach to modelling these data in a recursive way. As a variant, the long short-term memory (LSTM) extends the RNN with the ability to remember information for longer. Whilst one can increase the capacity of LSTM by widening or adding layers, additional parameters and runtime are usually required, which could make learning harder. We therefore propose a Tensor LSTM where the hidden states are tensorised as multidimensional arrays (tensors) and updated through a cross-layer convolution. As parameters are spatially shared within the tensor, we can efficiently widen the model without extra parameters by increasing the tensorised size; as deep computations of each time step are absorbed by temporal computations of the time series, we can implicitly deepen the model with little extra runtime by delaying the output. We show by experiments that our model is well-suited for various multimedia data modelling tasks, including text generation, text calculation, image classification, and video prediction.

APA, Harvard, Vancouver, ISO, and other styles

Kim, Seungnyun, Junwon Son, and Byonghyo Shim. "Energy-Efficient Ultra-Dense Network Using LSTM-based Deep Neural Networks." IEEE Transactions on Wireless Communications 20, no. 7 (2021): 4702–15. http://dx.doi.org/10.1109/twc.2021.3061577.

Full text

APA, Harvard, Vancouver, ISO, and other styles

We offer discounts on all premium plans for authors whose works are included in thematic literature selections. Contact us to get a unique promo code!