To see the other types of publications on this topic, follow the link: Recursive Neural Networks (RNNs).
Journal articles on the topic 'Recursive Neural Networks (RNNs)'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Recursive Neural Networks (RNNs).'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Zelios, Andreas, Achilleas Grammenos, Maria Papatsimouli, Nikolaos Asimopoulos, and George Fragulis. "Recursive neural networks: recent results and applications." SHS Web of Conferences 139 (2022): 03007. http://dx.doi.org/10.1051/shsconf/202213903007.
Full textAbstract:
Neural Network’s basic principles and functions are based on the nervous system of living organisms, they aim to simulate neurons of the human brain to solve complicated real-world problems by working in a forward-only manner. A recursive Neural Network on the other hand is based on a recursive design principle over a given sequence input, to come up with a scalar assessment of the structured input. This means that is ideal for a given sequence of input data that is when processed dependent on its previous input sequence, which by default are used in various problems of our era. A common example could be devices such as Amazon Alexa, which uses speech recognition i.e., given an audio input source that receives audio signals, tries to predict logical expressions extracted from its different audio segments to form complete sentences. But RNNs do not come with no problems or difficulties. Today’s problems become more and more complex involving parameters in big data form, therefore a need for bigger and deeper RNNs is being created. This paper aims to explore these problems and ways to reduce them while also providing a description of RNN’s beneficial nature and listing different uses of the state-of-the-art RNNs and their use in different problems as those mentioned above.
2
Wang, Qinglong, Kaixuan Zhang, Alexander G. Ororbia II, Xinyu Xing, Xue Liu, and C. Lee Giles. "An Empirical Evaluation of Rule Extraction from Recurrent Neural Networks." Neural Computation 30, no. 9 (2018): 2568–91. http://dx.doi.org/10.1162/neco_a_01111.
Full textAbstract:
Rule extraction from black box models is critical in domains that require model validation before implementation, as can be the case in credit scoring and medical diagnosis. Though already a challenging problem in statistical learning in general, the difficulty is even greater when highly nonlinear, recursive models, such as recurrent neural networks (RNNs), are fit to data. Here, we study the extraction of rules from second-order RNNs trained to recognize the Tomita grammars. We show that production rules can be stably extracted from trained RNNs and that in certain cases, the rules outperform the trained RNNs.
3
Socher, Richard, Andrej Karpathy, Quoc V. Le, Christopher D. Manning, and Andrew Y. Ng. "Grounded Compositional Semantics for Finding and Describing Images with Sentences." Transactions of the Association for Computational Linguistics 2 (December 2014): 207–18. http://dx.doi.org/10.1162/tacl_a_00177.
Full textAbstract:
Previous work on Recursive Neural Networks (RNNs) shows that these models can produce compositional feature vectors for accurately representing and classifying sentences or images. However, the sentence vectors of previous models cannot accurately represent visually grounded meaning. We introduce the DT-RNN model which uses dependency trees to embed sentences into a vector space in order to retrieve images that are described by those sentences. Unlike previous RNN-based models which use constituency trees, DT-RNNs naturally focus on the action and agents in a sentence. They are better able to abstract from the details of word order and syntactic expression. DT-RNNs outperform other recursive and recurrent neural networks, kernelized CCA and a bag-of-words baseline on the tasks of finding an image that fits a sentence description and vice versa. They also give more similar representations to sentences that describe the same image.
4
Cai, Guo-Rong, and Shui-Li Chen. "Recursive Neural Networks Based on PSO for Image Parsing." Abstract and Applied Analysis 2013 (2013): 1–7. http://dx.doi.org/10.1155/2013/617618.
Full textAbstract:
This paper presents an image parsing algorithm which is based on Particle Swarm Optimization (PSO) and Recursive Neural Networks (RNNs). State-of-the-art method such as traditional RNN-based parsing strategy uses L-BFGS over the complete data for learning the parameters. However, this could cause problems due to the nondifferentiable objective function. In order to solve this problem, the PSO algorithm has been employed to tune the weights of RNN for minimizing the objective. Experimental results obtained on the Stanford background dataset show that our PSO-based training algorithm outperforms traditional RNN, Pixel CRF, region-based energy, simultaneous MRF, and superpixel MRF.
5
Pike, Xander, and Jordan Cheer. "Active noise and vibration control of systems with primary path nonlinearities using FxLMS, Neural Networks and Recursive Neural Networks." Journal of the Acoustical Society of America 150, no. 4 (2021): A345. http://dx.doi.org/10.1121/10.0008532.
Full textAbstract:
Active control systems are often used to surmount the challenges associated with using passive control measures to control low frequencies, since they achieve control without the application of large or heavy control treatments. Historically, linear active control strategies have been used in feed-forward control systems to drive the control source to minimise the signal measured at the error sensor. However, when the response from noise source to error sensor becomes nonlinear, either in the primary or secondary path, the performance of such controllers can suffer. To overcome this limitation, it has previously been shown that Neural Networks (NNs) can outperform linear controllers. Furthermore, Recursive Neural Networks (RNNs) have been shown to outperform classical feed-forward networks in some cases. This is usually explained by the RNNs ability to form a rudimentary “memory.” This paper compares the behaviour of the linear FxLMS algorithm, a NN and an RNN through their application to the control of a simulated system with variable levels of saturation and hysteretic nonlinearities in the primary path. It is demonstrated that the NN is capable of greater control of saturation nonlinearities than FxLMS. Similarly, the RNN is capable of greater control of hysteretic nonlinearities than the NN.
6
Yang, Lei, Saddam Aziz, and Zhenyang Yu. "Cybersecurity Challenges in PV-Hydrogen Transport Networks: Leveraging Recursive Neural Networks for Resilient Operation." Energies 18, no. 9 (2025): 2262. https://doi.org/10.3390/en18092262.
Full textAbstract:
In the rapidly evolving landscape of transportation technologies, hydrogen vehicle networks integrated with photovoltaic (PV) systems represent a significant advancement toward sustainable mobility. However, the integration of such technologies also introduces complex cybersecurity challenges that must be meticulously managed to ensure operational integrity and system resilience. This paper explores the intricate dynamics of cybersecurity in PV-powered hydrogen vehicle networks, focusing on the real-time challenges posed by cyber threats such as False Data Injection Attacks (FDIAs) and their impact on network operations. Our research utilizes a novel hierarchical robust optimization model enhanced by Recursive Neural Networks (RNNs) to improve detection rates and response times to cyber incidents across various severity levels. The initial findings reveal that as the severity of incidents escalates from level 1 to 10, the response time significantly increases from an average of 7 min for low-severity incidents to over 20 min for high-severity scenarios, demonstrating the escalating complexity and resource demands of more severe incidents. Additionally, the study introduces an in-depth examination of the detection dynamics, illustrating that while detection rates generally decrease as incident frequency increases—due to system overload—the employment of advanced RNNs effectively mitigates this trend, sustaining high detection rates of up to 95% even under high-frequency scenarios. Furthermore, we analyze the cybersecurity risks specifically associated with the intermittency of PV-based hydrogen production, demonstrating how fluctuations in solar energy availability can create vulnerabilities that cyberattackers may exploit. We also explore the relationship between incident frequency, detection sensitivity, and the resulting false positive rates, revealing that the optimal adjustment of detection thresholds can reduce false positives by as much as 30%, even under peak load conditions. This paper not only provides a detailed empirical analysis of the cybersecurity landscape in PV-integrated hydrogen vehicle networks but also offers strategic insights into the deployment of AI-enhanced cybersecurity frameworks. The findings underscore the critical need for scalable, responsive cybersecurity solutions that can adapt to the dynamic threat environment of modern transport infrastructures, ensuring the sustainability and safety of solar-powered hydrogen mobility solutions.
7
Raghu, Nagashree, and Gowda Kishore. "Electric vehicle charging state predictions through hybrid deep learning: A review." GSC Advanced Research and Reviews 15, no. 1 (2023): 076–80. https://doi.org/10.5281/zenodo.7929676.
Full textAbstract:
This review paper discusses the application of hybrid deep learning techniques for predicting the charging state of electric vehicles. The paper highlights the importance of accurate predictions for the efficient management of electric vehicle charging stations. The review focuses on the use of recursive neural networks (RNNs) and the gated recurrent unit (GRU) framework in hybrid deep learning models, which have shown promising results in previous studies. In addition to hybrid deep learning, the paper also examines the use of support vector machines (SVMs) and artificial neural networks (ANNs) in charging state prediction. The strengths and weaknesses of these different approaches are analyzed and compared. The paper concludes that hybrid deep learning models, particularly those using RNNs and GRUs, are a promising approach for accurately predicting electric vehicle charging states. The paper also suggests potential areas for future research to further improve the accuracy and efficiency of charging state predictions.
8
Raghu Nagashree and Kishore Gowda. "Electric vehicle charging state predictions through hybrid deep learning: A review." GSC Advanced Research and Reviews 15, no. 1 (2023): 076–80. http://dx.doi.org/10.30574/gscarr.2023.15.1.0116.
Full textAbstract:
This review paper discusses the application of hybrid deep learning techniques for predicting the charging state of electric vehicles. The paper highlights the importance of accurate predictions for the efficient management of electric vehicle charging stations. The review focuses on the use of recursive neural networks (RNNs) and the gated recurrent unit (GRU) framework in hybrid deep learning models, which have shown promising results in previous studies. In addition to hybrid deep learning, the paper also examines the use of support vector machines (SVMs) and artificial neural networks (ANNs) in charging state prediction. The strengths and weaknesses of these different approaches are analyzed and compared. The paper concludes that hybrid deep learning models, particularly those using RNNs and GRUs, are a promising approach for accurately predicting electric vehicle charging states. The paper also suggests potential areas for future research to further improve the accuracy and efficiency of charging state predictions.
9
Hong, Chaoqun, Zhiqiang Zeng, Xiaodong Wang, and Weiwei Zhuang. "Multiple Network Fusion with Low-Rank Representation for Image-Based Age Estimation." Applied Sciences 8, no. 9 (2018): 1601. http://dx.doi.org/10.3390/app8091601.
Full textAbstract:
Image-based age estimation is a challenging task since there are ambiguities between the apparent age of face images and the actual ages of people. Therefore, data-driven methods are popular. To improve data utilization and estimation performance, we propose an image-based age estimation method. Theoretically speaking, the key idea of the proposed method is to integrate multi-modal features of face images. In order to achieve it, we propose a multi-modal learning framework, which is called Multiple Network Fusion with Low-Rank Representation (MNF-LRR). In this process, different deep neural network (DNN) structures, such as autoencoders, Convolutional Neural Networks (CNNs), Recursive Neural Networks (RNNs), and so on, can be used to extract semantic information of facial images. The outputs of these neural networks are then represented in a low-rank feature space. In this way, feature fusion is obtained in this space, and robust multi-modal image features can be computed. An experimental evaluation is conducted on two challenging face datasets for image-based age estimation extracted from the Internet Move Database (IMDB) and Wikipedia (WIKI). The results show the effectiveness of the proposed MNF-LRR.
10
S, Kalaivani, and Gopinath G. "MODIFIED BEE COLONY WITH BACTERIAL FORAGING OPTIMIZATION BASED HYBRID FEATURE SELECTION TECHNIQUE FOR INTRUSION DETECTION SYSTEM CLASSIFIER MODEL." ICTACT Journal on Soft Computing 10, no. 4 (2020): 2146–52. https://doi.org/10.21917/ijsc.2020.0305.
Full textAbstract:
Feature selection (FS) plays an essential role in creating machine learning models. The unrelated characteristics of the data disturb the precision of the perfection and upsurges the training time required to build the model. FS is a significant process in creating the Intrusion Detection System (IDS). In this document, we propose a technique for selecting container functions for IDS. To develop the performance capacity of the modified Artificial Bee Colony (ABC) procedure, a hybrid method is presented in which the swarm behavior of the Bacterial Foraging Optimization (BFO) algorithm is entered into the Modified Bee Colony (MBC) procedure to perform a local search. The proposed Hybrid MBC-BFO algorithm is analyzed with three different classification techniques which are separately analyzed to verify the proposed performance. The classification techniques are Artificial Neural Networks (ANN), Recursive Neural Network (ReNN), and Recurrent Neural Network (RNNs). The proposed algorithm has passed several algorithms for selecting advanced functions in terms of detection accuracy, recall, precision, false positive rate, and F-score.
11
Yang, Jiayi. "The application of machine learning algorithms in speech recognition error detection." Applied and Computational Engineering 16, no. 1 (2023): 191–95. http://dx.doi.org/10.54254/2755-2721/16/20230888.
Full textAbstract:
The field of speech recognition technology has experienced a significant progress in recent years, with error detection being a crucial research domain. The present study offers a comprehensive overview of this area. Prior to the emergence of neural networks, Hidden Markov models (HMM) have been widely employed as a primary framework for speech recognition. However, this model only achieves local optimality, leading to its gradual replacement by neural network models, which have attracted considerable attention from researchers aiming to enhance their recognition performance. Various models, such as variant RNNs, bidirectional recurrent neural networks, and Fastcorrect2 have been developed. This paper introduces HMM, followed by a presentation of several neural network models, which entail a detailed description of their respective framework, principle, and idea. The variant RNN model is designed to enhance the recursive connection between input and output layers, while the deep bidirectional recurrent neural network model simulates the nonlinear relationship between input feature vectors and output labels using two models, namely bidirectional and deep. Additionally, the FastCorrect2 model enhances the voting effect of candidate words and the alignment algorithm. Finally, the study highlights the application of speech recognition error detection in everyday life, emphasizing the importance of speech recognition.
12
Venturini, M. "Simulation of Compressor Transient Behavior Through Recurrent Neural Network Models." Journal of Turbomachinery 128, no. 3 (2005): 444–54. http://dx.doi.org/10.1115/1.2183315.
Full textAbstract:
In the paper, self-adapting models capable of reproducing time-dependent data with high computational speed are investigated. The considered models are recurrent feed-forward neural networks (RNNs) with one feedback loop in a recursive computational structure, trained by using a back-propagation learning algorithm. The data used for both training and testing the RNNs have been generated by means of a nonlinear physics-based model for compressor dynamic simulation, which was calibrated on a multistage axial-centrifugal small size compressor. The first step of the analysis is the selection of the compressor maneuver to be used for optimizing RNN training. The subsequent step consists in evaluating the most appropriate RNN structure (optimal number of neurons in the hidden layer and number of outputs) and RNN proper delay time. Then, the robustness of the model response towards measurement uncertainty is ascertained, by comparing the performance of RNNs trained on data uncorrupted or corrupted with measurement errors with respect to the simulation of data corrupted with measurement errors. Finally, the best RNN model is tested on field data taken on the axial-centrifugal compressor on which the physics-based model was calibrated, by comparing physics-based model and RNN predictions against measured data. The comparison between RNN predictions and measured data shows that the agreement can be considered acceptable for inlet pressure, outlet pressure and outlet temperature, while errors are significant for inlet mass flow rate.
13
Huang, Jianjun, Haoqiang Hu, and Li Kang. "Time Convolutional Network-Based Maneuvering Target Tracking with Azimuth–Doppler Measurement." Sensors 24, no. 1 (2024): 263. http://dx.doi.org/10.3390/s24010263.
Full textAbstract:
In the field of maneuvering target tracking, the combined observations of azimuth and Doppler may cause weak observation or non-observation in the application of traditional target-tracking algorithms. Additionally, traditional target tracking algorithms require pre-defined multiple mathematical models to accurately capture the complex motion states of targets, while model mismatch and unavoidable measurement noise lead to significant errors in target state prediction. To address those above challenges, in recent years, the target tracking algorithms based on neural networks, such as recurrent neural networks (RNNs), long short-term memory (LSTM) networks, and transformer architectures, have been widely used for their unique advantages to achieve accurate predictions. To better model the nonlinear relationship between the observation time series and the target state time series, as well as the contextual relationship among time series points, we present a deep learning algorithm called recursive downsample–convolve–interact neural network (RDCINN) based on convolutional neural network (CNN) that downsamples time series into subsequences and extracts multi-resolution features to enable the modeling of complex relationships between time series, which overcomes the shortcomings of traditional target tracking algorithms in using observation information inefficiently due to weak observation or non-observation. The experimental results show that our algorithm outperforms other existing algorithms in the scenario of strong maneuvering target tracking with the combined observations of azimuth and Doppler.
14
Priyadharshini, P., and B. S. E. Zoraida. "Hybrid Semantic Feature Descriptor and Fuzzy C-Means Clustering for Lung Cancer Detection and Classification." Journal of Computational and Theoretical Nanoscience 18, no. 4 (2021): 1263–69. http://dx.doi.org/10.1166/jctn.2021.9391.
Full textAbstract:
Lung cancer (LC) will decrease the yield, which will have a negative impact on the economy. Therefore, primary and accurate the attack finding is a priority for the agro-dependent state. In several modern technologies for early detection of LC, image processing has become a one of the essential tool so that it cannot only early to find the disease accurately, but also successfully measure it. Various approaches have been developed to detect LC based on background modelling. Most of them focus on temporal information but partially or completely ignore spatial information, making it sensitive to noise. In order to overcome these issues an improved hybrid semantic feature descriptor technique is introduced based on Gray-Level Co-Occurrence Matrix (GLCM), Local Binary Pattern (LBP) and histogram of oriented gradients (HOG) feature extraction algorithms. And also to improve the LC segmentation problems a fuzzy c-means clustering algorithm (FCM) is used. Experiments and comparisons on publically available LIDC-IBRI dataset. To evaluate the proposed feature extraction performance three different classifiers are analysed such as artificial neural networks (ANN), recursive neural network and recurrent neural networks (RNNs).
15
Zeybek, Sultan, Duc Truong Pham, Ebubekir Koç, and Aydın Seçer. "An Improved Bees Algorithm for Training Deep Recurrent Networks for Sentiment Classification." Symmetry 13, no. 8 (2021): 1347. http://dx.doi.org/10.3390/sym13081347.
Full textAbstract:
Recurrent neural networks (RNNs) are powerful tools for learning information from temporal sequences. Designing an optimum deep RNN is difficult due to configuration and training issues, such as vanishing and exploding gradients. In this paper, a novel metaheuristic optimisation approach is proposed for training deep RNNs for the sentiment classification task. The approach employs an enhanced Ternary Bees Algorithm (BA-3+), which operates for large dataset classification problems by considering only three individual solutions in each iteration. BA-3+ combines the collaborative search of three bees to find the optimal set of trainable parameters of the proposed deep recurrent learning architecture. Local learning with exploitative search utilises the greedy selection strategy. Stochastic gradient descent (SGD) learning with singular value decomposition (SVD) aims to handle vanishing and exploding gradients of the decision parameters with the stabilisation strategy of SVD. Global learning with explorative search achieves faster convergence without getting trapped at local optima to find the optimal set of trainable parameters of the proposed deep recurrent learning architecture. BA-3+ has been tested on the sentiment classification task to classify symmetric and asymmetric distribution of the datasets from different domains, including Twitter, product reviews, and movie reviews. Comparative results have been obtained for advanced deep language models and Differential Evolution (DE) and Particle Swarm Optimization (PSO) algorithms. BA-3+ converged to the global minimum faster than the DE and PSO algorithms, and it outperformed the SGD, DE, and PSO algorithms for the Turkish and English datasets. The accuracy value and F1 measure have improved at least with a 30–40% improvement than the standard SGD algorithm for all classification datasets. Accuracy rates in the RNN model trained with BA-3+ ranged from 80% to 90%, while the RNN trained with SGD was able to achieve between 50% and 60% for most datasets. The performance of the RNN model with BA-3+ has as good as for Tree-LSTMs and Recursive Neural Tensor Networks (RNTNs) language models, which achieved accuracy results of up to 90% for some datasets. The improved accuracy and convergence results show that BA-3+ is an efficient, stable algorithm for the complex classification task, and it can handle the vanishing and exploding gradients problem of deep RNNs.
16
Li, Ni. "Construction and Implementation of Ideological and Political Education Platforms Based on Artificial Intelligence Technology." International Journal of Web-Based Learning and Teaching Technologies 20, no. 1 (2025): 1–23. https://doi.org/10.4018/ijwltt.372072.
Full textAbstract:
In depth exploration of how the pandemic has reshaped the education ecosystem over the past three years, especially in the context of the surge in demand for online education courses and learning platforms, this article focuses on the field of student ideological and political education, and innovatively constructs a moral and political education platform that integrates efficiency, interactivity, and personalization. Through in-depth analysis of existing online education platforms in the market, we found that although these platforms have the potential for remote teaching in terms of technology. By introducing advanced artificial intelligence technologies, especially recursive neural networks (RNNs) and their variants in deep learning, traditional collaborative recommendation algorithms are revolutionized. This improvement not only enhances the algorithm's understanding of user behavior patterns, but also more accurately captures users' potential points of interest and changes in needs, thereby achieving more personalized content recommendations.
17
Wei, Siwei, Yanan Song, Donghua Liu, Sichen Shen, Rong Gao, and Chunzhi Wang. "Hierarchical Dynamic Spatio-Temporal Graph Convolutional Networks with Self-Supervised Learning for Traffic Flow Forecasting." Inventions 9, no. 5 (2024): 102. http://dx.doi.org/10.3390/inventions9050102.
Full textAbstract:
It is crucial for both traffic management organisations and individual commuters to be able to forecast traffic flows accurately. Graph neural networks made great strides in this field owing to their exceptional capacity to capture spatial correlations. However, existing approaches predominantly focus on local geographic correlations, ignoring cross-region interdependencies in a global context, which is insufficient to extract comprehensive semantic relationships, thereby limiting prediction accuracy. Additionally, most GCN-based models rely on pre-defined graphs and unchanging adjacency matrices to reflect the spatial relationships among node features, neglecting the dynamics of spatio-temporal features and leading to challenges in capturing the complexity and dynamic spatial dependencies in traffic data. To tackle these issues, this paper puts forward a fresh approach: a new self-supervised dynamic spatio-temporal graph convolutional network (SDSC) for traffic flow forecasting. The proposed SDSC model is a hierarchically structured graph–neural architecture that is intended to augment the representation of dynamic traffic patterns through a self-supervised learning paradigm. Specifically, a dynamic graph is created using a combination of temporal, spatial, and traffic data; then, a regional graph is constructed based on geographic correlation using clustering to capture cross-regional interdependencies. In the feature learning module, spatio-temporal correlations in traffic data are subjected to recursive extraction using dynamic graph convolution facilitated by Recurrent Neural Networks (RNNs). Furthermore, self-supervised learning is embedded within the network training process as an auxiliary task, with the objective of enhancing the prediction task by optimising the mutual information of the learned features across the two graph networks. The superior performance of the proposed SDSC model in comparison with SOTA approaches was confirmed by comprehensive experiments conducted on real road datasets, PeMSD4 and PeMSD8. These findings validate the efficacy of dynamic graph modelling and self-supervision tasks in improving the precision of traffic flow prediction.
18
Liao, Bolin, Cheng Hua, Xinwei Cao, Vasilios N. Katsikis, and Shuai Li. "Complex Noise-Resistant Zeroing Neural Network for Computing Complex Time-Dependent Lyapunov Equation." Mathematics 10, no. 15 (2022): 2817. http://dx.doi.org/10.3390/math10152817.
Full textAbstract:
Complex time-dependent Lyapunov equation (CTDLE), as an important means of stability analysis of control systems, has been extensively employed in mathematics and engineering application fields. Recursive neural networks (RNNs) have been reported as an effective method for solving CTDLE. In the previous work, zeroing neural networks (ZNNs) have been established to find the accurate solution of time-dependent Lyapunov equation (TDLE) in the noise-free conditions. However, noises are inevitable in the actual implementation process. In order to suppress the interference of various noises in practical applications, in this paper, a complex noise-resistant ZNN (CNRZNN) model is proposed and employed for the CTDLE solution. Additionally, the convergence and robustness of the CNRZNN model are analyzed and proved theoretically. For verification and comparison, three experiments and the existing noise-tolerant ZNN (NTZNN) model are introduced to investigate the effectiveness, convergence and robustness of the CNRZNN model. Compared with the NTZNN model, the CNRZNN model has more generality and stronger robustness. Specifically, the NTZNN model is a special form of the CNRZNN model, and the residual error of CNRZNN can converge rapidly and stably to order 10−5 when solving CTDLE under complex linear noises, which is much lower than order 10−1 of the NTZNN model. Analogously, under complex quadratic noises, the residual error of the CNRZNN model can converge to 2∥A∥F/ζ3 quickly and stably, while the residual error of the NTZNN model is divergent.
19
Venkitaraman, Ashwin Kavasseri, and Venkata Satya Rahul Kosuru. "Hybrid Deep Learning Mechanism for Charging Control and Management of Electric Vehicles." European Journal of Electrical Engineering and Computer Science 7, no. 1 (2023): 38–46. http://dx.doi.org/10.24018/ejece.2023.7.1.485.
Full textAbstract:
In perspective of their environmental friendliness and energy efficiency, Electric Vehicles (EVs) are posing a threat to traditional gasoline automobiles. Identifying the future charging needs of EV users may be aided by the forecasting of states linked to EV charging. It might deliver customized charge capacity statistics based on users' real-time locations as well as direct the operation and management of charging infrastructure. Consequently, an emergent problem is the effective model of EV charging state predictions. In this study, a hybrid deep learning approach is suggested to assure safe and dependable charging operations that prevent the battery from being overcharged or discharged. A Recursive Neural Networks (RNNs) for feature extraction process is suggested to acquire adequate feature information on the battery. The bidirectional gated recurrent unit framework (GRU) was then established by the study to predict the state of the EV. The GRU receives its input from the RNNs' output, which substantially enhances the effectiveness of the model. Because of its much simpler structure, the RNN-GRU has a lower computational performance. The experimental findings demonstrate the GRU method's ability to accurately track mileage of the electric vehicle. A hybrid deep learning-based prediction approach could give quick convergence speed less error rate in comparison to the appropriate method for obtaining state of charge estimate over conventional models, as demonstrated by the extensive real-world tests.
20
Pikus, Michal, and Jarosław Wąs. "Using Deep Neural Network Methods for Forecasting Energy Productivity Based on Comparison of Simulation and DNN Results for Central Poland—Swietokrzyskie Voivodeship." Energies 16, no. 18 (2023): 6632. http://dx.doi.org/10.3390/en16186632.
Full textAbstract:
Forecasting electricity demand is of utmost importance for ensuring the stability of the entire energy sector. However, predicting the future electricity demand and its value poses a formidable challenge due to the intricate nature of the processes influenced by renewable energy sources. Within this piece, we have meticulously explored the efficacy of fundamental deep learning models designed for electricity forecasting. Among the deep learning models, we have innovatively crafted recursive neural networks (RNNs) predominantly based on LSTM and combined architectures. The dataset employed was procured from a SolarEdge designer. The dataset encompasses daily records spanning the past year, encompassing an exhaustive collection of parameters extracted from solar farm (based on location in Central Europe (Poland Swietokrzyskie Voivodeship)). The experimental findings unequivocally demonstrated the exceptional superiority of the LSTM models over other counterparts concerning forecasting accuracy. Consequently, we compared multilayer DNN architectures with results provided by the simulator. The measurable results of both DNN models are multi-layer LSTM-only accuracy based on R2—0.885 and EncoderDecoderLSTM R2—0.812.
21
Abeltino, Alessio, Giada Bianchetti, Cassandra Serantoni, Alessia Riente, Marco De Spirito, and Giuseppe Maulucci. "Putting the Personalized Metabolic Avatar into Production: A Comparison between Deep-Learning and Statistical Models for Weight Prediction." Nutrients 15, no. 5 (2023): 1199. http://dx.doi.org/10.3390/nu15051199.
Full textAbstract:
Nutrition is a cross-cutting sector in medicine, with a huge impact on health, from cardiovascular disease to cancer. Employment of digital medicine in nutrition relies on digital twins: digital replicas of human physiology representing an emergent solution for prevention and treatment of many diseases. In this context, we have already developed a data-driven model of metabolism, called a “Personalized Metabolic Avatar” (PMA), using gated recurrent unit (GRU) neural networks for weight forecasting. However, putting a digital twin into production to make it available for users is a difficult task that as important as model building. Among the principal issues, changes to data sources, models and hyperparameters introduce room for error and overfitting and can lead to abrupt variations in computational time. In this study, we selected the best strategy for deployment in terms of predictive performance and computational time. Several models, such as the Transformer model, recursive neural networks (GRUs and long short-term memory networks) and the statistical SARIMAX model were tested on ten users. PMAs based on GRUs and LSTM showed optimal and stable predictive performances, with the lowest root mean squared errors (0.38 ± 0.16–0.39 ± 0.18) and acceptable computational times of the retraining phase (12.7 ± 1.42 s–13.5 ± 3.60 s) for a production environment. While the Transformer model did not bring a substantial improvement over RNNs in term of predictive performance, it increased the computational time for both forecasting and retraining by 40%. The SARIMAX model showed the worst performance in term of predictive performance, though it had the best computational time. For all the models considered, the extent of the data source was a negligible factor, and a threshold was established for the number of time points needed for a successful prediction.
22
Mishra, Nidhi, and Ghorpade Bipin Shivaji. "Integrated Deep Learning Framework for Electric Vehicle Charging Optimization and Management." E3S Web of Conferences 564 (2024): 02013. http://dx.doi.org/10.1051/e3sconf/202456402013.
Full textAbstract:
Vehicles that run on petrol face competition from electric vehicles (EVs), which are more environmentally friendly and consume less energy than gasoline-powered automobiles. If we can predict the states that have an effect on charging, we might be able to estimate how much charging electric vehicle owners will require in the future. It is also capable of operating and managing charging infrastructure, in addition to providing users with individualised charge capacity statistics based on where they are precisely at the moment. As a result of this, developing a reliable model that can accurately predict the charging state of an electric vehicle has become an important issue. Based on the findings of this study, it is recommended to employ a combination of machine learning and deep learning in order to guarantee that the charging process is both secure and dependable, and that the battery does not become overcharged or over-drained. It has been suggested that a process of feature extraction using Recursive Neural Networks (RNNs) be utilised in order to obtain sufficient feature information regarding the battery. The bidirectional gated recurrent unit framework (GRU) was then established in the research project in order to make an educated guess as to the state of the electric vehicle. It is because of the information that the GRU obtains from the output of the RNNs that the model is significantly more useful. As a result of its more straightforward structure, the RNN-GRU is less effective when it comes to computing. In light of the findings of the tests, it is clear that the GRU method is capable of accurately monitoring the mileage of an electric vehicle. Based on the results of numerous tests conducted in the real world, it has been demonstrated that a mixed deep learning-based prediction method has the potential to provide a faster convergence speed and a lower error rate than the conventional method of obtaining an estimate of the state of charge.
23
Suradhaniwar, Saurabh, Soumyashree Kar, Surya S. Durbha, and Adinarayana Jagarlapudi. "Time Series Forecasting of Univariate Agrometeorological Data: A Comparative Performance Evaluation via One-Step and Multi-Step Ahead Forecasting Strategies." Sensors 21, no. 7 (2021): 2430. http://dx.doi.org/10.3390/s21072430.
Full textAbstract:
High-frequency monitoring of agrometeorological parameters is quintessential in the domain of Precision Agriculture (PA), where timeliness of collected observations and the ability to generate ahead-of-time predictions can substantially impact the crop yield. In this context, state-of-the-art internet-of-things (IoT)-based sensing platforms are often employed to generate, pre-process and assimilate real-time data from heterogeneous sensors and streaming data sources. Simultaneously, Time-Series Forecasting Algorithms (TSFAs) are responsible for generating reliable forecasts with a pre-defined forecast horizon and confidence. These TSFAs often rely on modelling the correlation between endogenous variables, the impact of exogenous variables on latent form and structural properties of data such as autocorrelation, periodicity, trend, pattern, and causality to approximate the model parameters. Traditionally, TSFAs such as the Holt–Winters (HW) and Autoregressive family of models (ARIMA) apply a linear and parametric approach towards model approximation, whilst models like Support Vector Regression (SVRs) and Neural Networks (NNs) adhere to a non-linear, non-parametric approach for modelling the historical data. Recently, Deep-Learning-based TSFAs such as Recurrent Neural Networks (RNNs), and Long-Short-Term-Memory (LSTMS) have gained popularity due to their capability to model long sequences of highly non-linear and stochastic data effectively. However, the evolution of TSFAs for predicting agrometeorological parameters pivots around one-step-ahead forecasting, which often overestimates the performance metrics defined for validating forecast capabilities of potential TSFAs. Hence, this paper attempts to evaluate and compare the performance of different machine learning (ML) and deep learning (DL) based TSFAs under one-step and multi-step-ahead forecast scenarios, thereby estimating the generalization capabilities of TSFA models over unseen data. The data used in this study are collected from an Automatic Weather Station (AWS), sampled at an interval of 15 min, and range over one month. Temperature (T) and Humidity (H) observations from the AWS are further converted into univariate, supervised time-series diurnal data profiles. Finally, walk-forward validation is used to evaluate recursive one-step-ahead forecasts until the desired prediction horizon is achieved. The results show that the Seasonal Auto-Regressive Integrated Moving Average (SARIMA) and SVR models outperform their DL-based counterparts in one-step and multi-step ahead settings with a fixed forecast horizon. This work aims to present a baseline comparison between different TSFAs to assist the process of model selection and facilitate rapid ahead-of-time forecasting for end-user applications.
24
Ding, Haohan, Haoke Hou, Long Wang, Xiaohui Cui, Wei Yu, and David I. Wilson. "Application of Convolutional Neural Networks and Recurrent Neural Networks in Food Safety." Foods 14, no. 2 (2025): 247. https://doi.org/10.3390/foods14020247.
Full textAbstract:
This review explores the application of convolutional neural networks (CNNs) and recurrent neural networks (RNNs) in food safety detection and risk prediction. This paper highlights the advantages of CNNs in image processing and feature recognition, as well as the powerful capabilities of RNNs (especially their variant LSTM) in time series data modeling. This paper also makes a comparative analysis in many aspects: Firstly, the advantages and disadvantages of traditional food safety detection and risk prediction methods are compared with deep learning technologies such as CNNs and RNNs. Secondly, the similarities and differences between CNNs and fully connected neural networks in processing image data are analyzed. Furthermore, the advantages and disadvantages of RNNs and traditional statistical modeling methods in processing time series data are discussed. Finally, the application directions of CNNs in food safety detection and RNNs in food safety risk prediction are compared. This paper also discusses combining these deep learning models with technologies such as the Internet of Things (IoT), blockchain, and federated learning to improve the accuracy and efficiency of food safety detection and risk warning. Finally, this paper mentions the limitations of RNNs and CNNs in the field of food safety, as well as the challenges in the interpretability of the model, and suggests the use of interpretable artificial intelligence (XAI) technology to improve the transparency of the model.
25
Ma, Xiao, Peter Karkus, David Hsu, and Wee Sun Lee. "Particle Filter Recurrent Neural Networks." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (2020): 5101–8. http://dx.doi.org/10.1609/aaai.v34i04.5952.
Full textAbstract:
Recurrent neural networks (RNNs) have been extraordinarily successful for prediction with sequential data. To tackle highly variable and multi-modal real-world data, we introduce Particle Filter Recurrent Neural Networks (PF-RNNs), a new RNN family that explicitly models uncertainty in its internal structure: while an RNN relies on a long, deterministic latent state vector, a PF-RNN maintains a latent state distribution, approximated as a set of particles. For effective learning, we provide a fully differentiable particle filter algorithm that updates the PF-RNN latent state distribution according to the Bayes rule. Experiments demonstrate that the proposed PF-RNNs outperform the corresponding standard gated RNNs on a synthetic robot localization dataset and 10 real-world sequence prediction datasets for text classification, stock price prediction, etc.
26
Park, Sungrae, Kyungwoo Song, Mingi Ji, Wonsung Lee, and Il-Chul Moon. "Adversarial Dropout for Recurrent Neural Networks." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 4699–706. http://dx.doi.org/10.1609/aaai.v33i01.33014699.
Full textAbstract:
Successful application processing sequential data, such as text and speech, requires an improved generalization performance of recurrent neural networks (RNNs). Dropout techniques for RNNs were introduced to respond to these demands, but we conjecture that the dropout on RNNs could have been improved by adopting the adversarial concept. This paper investigates ways to improve the dropout for RNNs by utilizing intentionally generated dropout masks. Specifically, the guided dropout used in this research is called as adversarial dropout, which adversarially disconnects neurons that are dominantly used to predict correct targets over time. Our analysis showed that our regularizer, which consists of a gap between the original and the reconfigured RNNs, was the upper bound of the gap between the training and the inference phases of the random dropout. We demonstrated that minimizing our regularizer improved the effectiveness of the dropout for RNNs on sequential MNIST tasks, semi-supervised text classification tasks, and language modeling tasks.
27
Kao, Jonathan C. "Considerations in using recurrent neural networks to probe neural dynamics." Journal of Neurophysiology 122, no. 6 (2019): 2504–21. http://dx.doi.org/10.1152/jn.00467.2018.
Full textAbstract:
Recurrent neural networks (RNNs) are increasingly being used to model complex cognitive and motor tasks performed by behaving animals. RNNs are trained to reproduce animal behavior while also capturing key statistics of empirically recorded neural activity. In this manner, the RNN can be viewed as an in silico circuit whose computational elements share similar motifs with the cortical area it is modeling. Furthermore, because the RNN’s governing equations and parameters are fully known, they can be analyzed to propose hypotheses for how neural populations compute. In this context, we present important considerations when using RNNs to model motor behavior in a delayed reach task. First, by varying the network’s nonlinear activation and rate regularization, we show that RNNs reproducing single-neuron firing rate motifs may not adequately capture important population motifs. Second, we find that even when RNNs reproduce key neurophysiological features on both the single neuron and population levels, they can do so through distinctly different dynamical mechanisms. To distinguish between these mechanisms, we show that an RNN consistent with a previously proposed dynamical mechanism is more robust to input noise. Finally, we show that these dynamics are sufficient for the RNN to generalize to tasks it was not trained on. Together, these results emphasize important considerations when using RNN models to probe neural dynamics. NEW & NOTEWORTHY Artificial neurons in a recurrent neural network (RNN) may resemble empirical single-unit activity but not adequately capture important features on the neural population level. Dynamics of RNNs can be visualized in low-dimensional projections to provide insight into the RNN’s dynamical mechanism. RNNs trained in different ways may reproduce neurophysiological motifs but do so with distinctly different mechanisms. RNNs trained to only perform a delayed reach task can generalize to perform tasks where the target is switched or the target location is changed.
28
Wang, Rui. "Generalisation of Feed-Forward Neural Networks and Recurrent Neural Networks." Applied and Computational Engineering 40, no. 1 (2024): 242–46. http://dx.doi.org/10.54254/2755-2721/40/20230659.
Full textAbstract:
This paper presents an in-depth analysis of Feed-Forward Neural Networks (FNNs) and Recurrent Neural Networks (RNNs), two powerful models in the field of artificial intelligence. Understanding these models and their applications is crucial for harnessing their potential. The study addresses the need to comprehend the unique characteristics and architectures of FNNs and RNNs. These models excel at processing sequential and temporal data, making them indispensable in tasks. Furthermore, the paper emphasises the importance of variables in FNNs and proposes a novel method to rank the importance of independent variables in predicting the output variable. By understanding the relationship between inputs and outputs, valuable insights can be gained into the underlying patterns and mechanisms driving the system being modelled. Additionally, the research explores the impact of initial weights on model performance. Contrary to conventional beliefs, the study provides evidence that neural networks with random weights can achieve competitive performance, particularly in situations with limited training datasets. This finding challenges the traditional notion that careful initialization is necessary for neural networks to perform well. In summary, this paper provides a comprehensive analysis of FNNs and RNNs while highlighting the importance of understanding the relationship between variables and the impact of initial weights on model performance. By shedding light on these crucial aspects, this research contributes to the advancement and effective utilisation of neural networks, paving the way for improved predictions and insights in various domains.
29
Meng, Fandong, Jinchao Zhang, Yang Liu, and Jie Zhou. "Multi-Zone Unit for Recurrent Neural Networks." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (2020): 5150–57. http://dx.doi.org/10.1609/aaai.v34i04.5958.
Full textAbstract:
Recurrent neural networks (RNNs) have been widely used to deal with sequence learning problems. The input-dependent transition function, which folds new observations into hidden states to sequentially construct fixed-length representations of arbitrary-length sequences, plays a critical role in RNNs. Based on single space composition, transition functions in existing RNNs often have difficulty in capturing complicated long-range dependencies. In this paper, we introduce a new Multi-zone Unit (MZU) for RNNs. The key idea is to design a transition function that is capable of modeling multiple space composition. The MZU consists of three components: zone generation, zone composition, and zone aggregation. Experimental results on multiple datasets of the character-level language modeling task and the aspect-based sentiment analysis task demonstrate the superiority of the MZU.
30
Pauli, Patricia, Julian Berberich, and Frank Allgöwer. "Robustness analysis and training of recurrent neural networks using dissipativity theory." at - Automatisierungstechnik 70, no. 8 (2022): 730–39. http://dx.doi.org/10.1515/auto-2022-0032.
Full textAbstract:
Abstract Neural networks are widely applied in control applications, yet providing safety guarantees for neural networks is challenging due to their highly nonlinear nature. We provide a comprehensive introduction to the analysis of recurrent neural networks (RNNs) using robust control and dissipativity theory. Specifically, we consider H 2 {\mathcal{H}_{2}} -performance and the ℓ 2 {\ell _{2}} -gain to quantify the robustness of dynamic RNNs with respect to input perturbations. First, we analyze the robustness of RNNs using the proposed robustness certificates and then, we present linear matrix inequality constraints to be used in training of RNNs to enforce robustness. Finally, we illustrate in a numerical example that the proposed approach enhances the robustness of RNNs.
31
Chen, Yingyi, Qianqian Cheng, Yanjun Cheng, Hao Yang, and Huihui Yu. "Applications of Recurrent Neural Networks in Environmental Factor Forecasting: A Review." Neural Computation 30, no. 11 (2018): 2855–81. http://dx.doi.org/10.1162/neco_a_01134.
Full textAbstract:
Analysis and forecasting of sequential data, key problems in various domains of engineering and science, have attracted the attention of many researchers from different communities. When predicting the future probability of events using time series, recurrent neural networks (RNNs) are an effective tool that have the learning ability of feedforward neural networks and expand their expression ability using dynamic equations. Moreover, RNNs are able to model several computational structures. Researchers have developed various RNNs with different architectures and topologies. To summarize the work of RNNs in forecasting and provide guidelines for modeling and novel applications in future studies, this review focuses on applications of RNNs for time series forecasting in environmental factor forecasting. We present the structure, processing flow, and advantages of RNNs and analyze the applications of various RNNs in time series forecasting. In addition, we discuss limitations and challenges of applications based on RNNs and future research directions. Finally, we summarize applications of RNNs in forecasting.
32
Chen, Dechao, Shuai Li, and Qing Wu. "A Review on Neural Dynamics for Robot Autonomy." International Journal of Robotics and Control 1, no. 1 (2018): 20. http://dx.doi.org/10.5430/ijrc.v1n1p20.
Full textAbstract:
Exploiting neural networks to solve control problems of robots is becoming commonly and effectively in academia and engineering. Due to the remarkable features like distributed storage, parallelism, easy implementation by hardware, adaptive self-learning capability, and free of off-line training, the solutions of neural networks break the bottlenecks of serial-processing strategies and methods, and serve as significant alternatives for robotic engineers and researchers. Especially, various types and branches of recurrent neural networks (RNNs) have been sequentially developed since the seminal works by Hopfield and Tank. Successively, many classes and branches of RNNs such as primal-dual neural networks (PDNNs), zeroing neural networks (ZNNs) and gradient neural networks (GNNs) are proposed, investigated, developed and applied to the robot autonomy. The objective of this paper is to present a comprehensive review of the research on neural networks (especially RNNs) for control problems solving of different kinds of robots. Specifically, the state-of-the-art research of RNNs, PDNNs, ZNNs and GNNs in different robot control problems solving are detailedly revisited and reported. The readers can readily find many effective and valuable solutions on the basis of neural networks for the robot autonomy in this paper.
33
Jacobsson, Henrik. "Rule Extraction from Recurrent Neural Networks: ATaxonomy and Review." Neural Computation 17, no. 6 (2005): 1223–63. http://dx.doi.org/10.1162/0899766053630350.
Full textAbstract:
Rule extraction (RE) from recurrent neural networks (RNNs) refers to finding models of the underlying RNN, typically in the form of finite state machines, that mimic the network to a satisfactory degree while having the advantage of being more transparent. RE from RNNs can be argued to allow a deeper and more profound form of analysis of RNNs than other, more or less ad hoc methods. RE may give us understanding of RNNs in the intermediate levels between quite abstract theoretical knowledge of RNNs as a class of computing devices and quantitative performance evaluations of RNN instantiations. The development of techniques for extraction of rules from RNNs has been an active field since the early 1990s. This article reviews the progress of this development and analyzes it in detail. In order to structure the survey and evaluate the techniques, a taxonomy specifically designed for this purpose has been developed. Moreover, important open research issues are identified that, if addressed properly, possibly can give the field a significant push forward.
34
Du, Xiaoli, Hongwei Zeng, Shengbo Chen, and Zhou Lei. "RNNCon: Contribution Coverage Testing for Stacked Recurrent Neural Networks." Entropy 25, no. 3 (2023): 520. http://dx.doi.org/10.3390/e25030520.
Full textAbstract:
Recurrent Neural Networks (RNNs) are applied in safety-critical fields such as autonomous driving, aircraft collision detection, and smart credit. They are highly susceptible to input perturbations, but little research on RNN-oriented testing techniques has been conducted, leaving a threat to a large number of sequential application domains. To address these gaps, improve the test adequacy of RNNs, find more defects, and improve the performance of RNNs models and their robustness to input perturbations. We aim to propose a test coverage metric for the underlying structure of RNNs, which is used to guide the generation of test inputs to test RNNs. Although coverage metrics have been proposed for RNNs, such as the hidden state coverage in RNN-Test, they ignore the fact that the underlying structure of RNNs is still a fully connected neural network but with an additional “delayer” that records the network state at the time of data input. We use the contributions, i.e., the combination of the outputs of neurons and the weights they emit, as the minimum computational unit of RNNs to explore the finer-grained logical structure inside the recurrent cells. Compared to existing coverage metrics, our research covers the decision mechanism of RNNs in more detail and is more likely to generate more adversarial samples and discover more flaws in the model. In this paper, we redefine the contribution coverage metric applicable to Stacked LSTMs and Stacked GRUs by considering the joint effect of neurons and weights in the underlying structure of the neural network. We propose a new coverage metric, RNNCon, which can be used to guide the generation of adversarial test inputs. And we design and implement a test framework prototype RNNCon-Test. 2 datasets, 4 LSTM models, and 4 GRU models are used to verify the effectiveness of RNNCon-Test. Compared to the current state-of-the-art study RNN-Test, RNNCon can cover a deeper decision logic of RNNs. RNNCon-Test is not only effective in identifying defects in Deep Learning (DL) systems but also in improving the performance of the model if the adversarial inputs generated by RNNCon-Test are filtered and added to the training set to retrain the model. In the case where the accuracy of the model is already high, RNNCon-Test is still able to improve the accuracy of the model by up to 0.45%.
35
Tito Ayyalasomayajula, Madan Mohan, and Sailaja Ayyalasomayajula. "Improving Machine Reliability with Recurrent Neural Networks." International Journal for Research Publication and Seminar 11, no. 4 (2020): 253–79. http://dx.doi.org/10.36676/jrps.v11.i4.1500.
Full textAbstract:
This study explores the application of recurrent neural networks (RNNs) to enhance machine reliability in industrial settings, specifically in predictive maintenance systems. Predictive maintenance uses previous sensor data to identify abnormalities and forecast machine breakdowns before they occur, lowering downtime and maintenance costs. RNNs are ideal with their unique capacity to handle sequential input while capturing temporal relationships. RNN-based models may reliably foresee machine breakdowns and detect early malfunction indicators, allowing for appropriate interventions. The paper investigates key RNN architectures, such as Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), that have proven effective in addressing the limitations of traditional machine learning models, particularly in dealing with long-term dependencies and avoiding the vanishing gradient issue. LSTMs and GRUs are noted for their excellent performance in predictive maintenance, which requires precise failure predictions. However, obstacles persist, notably regarding data quality—sensor data is often noisy, missing, or inconsistent—and model interpretability since RNNs' "black-box" nature makes comprehending predictions challenging. Addressing these difficulties is critical for effective adoption in industrial settings. Future directions include integrating domain knowledge to improve model accuracy and creating hybrid models that combine RNNs with machine learning techniques, such as convolutional neural networks (CNNs) or support vector machines (SVMs), to improve predictive maintenance systems' robustness and scalability. These developments might considerably impact equipment dependability and operational efficiency in production.
36
Lalapura, Varsha S., Veerender Reddy Bhimavarapu, J. Amudha, and Hariram Selvamurugan Satheesh. "A Systematic Evaluation of Recurrent Neural Network Models for Edge Intelligence and Human Activity Recognition Applications." Algorithms 17, no. 3 (2024): 104. http://dx.doi.org/10.3390/a17030104.
Full textAbstract:
The Recurrent Neural Networks (RNNs) are an essential class of supervised learning algorithms. Complex tasks like speech recognition, machine translation, sentiment classification, weather prediction, etc., are now performed by well-trained RNNs. Local or cloud-based GPU machines are used to train them. However, inference is now shifting to miniature, mobile, IoT devices and even micro-controllers. Due to their colossal memory and computing requirements, mapping RNNs directly onto resource-constrained platforms is arcane and challenging. The efficacy of edge-intelligent RNNs (EI-RNNs) must satisfy both performance and memory-fitting requirements at the same time without compromising one for the other. This study’s aim was to provide an empirical evaluation and optimization of historic as well as recent RNN architectures for high-performance and low-memory footprint goals. We focused on Human Activity Recognition (HAR) tasks based on wearable sensor data for embedded healthcare applications. We evaluated and optimized six different recurrent units, namely Vanilla RNNs, Long Short-Term Memory (LSTM) units, Gated Recurrent Units (GRUs), Fast Gated Recurrent Neural Networks (FGRNNs), Fast Recurrent Neural Networks (FRNNs), and Unitary Gated Recurrent Neural Networks (UGRNNs) on eight publicly available time-series HAR datasets. We used the hold-out and cross-validation protocols for training the RNNs. We used low-rank parameterization, iterative hard thresholding, and spare retraining compression for RNNs. We found that efficient training (i.e., dataset handling and preprocessing procedures, hyperparameter tuning, and so on, and suitable compression methods (like low-rank parameterization and iterative pruning) are critical in optimizing RNNs for performance and memory efficiency. We implemented the inference of the optimized models on Raspberry Pi.
37
Sav, Sinem, Abdulrahman Diaa, Apostolos Pyrgelis, Jean-Philippe Bossuat, and Jean-Pierre Hubaux. "Privacy-Preserving Federated Recurrent Neural Networks." Proceedings on Privacy Enhancing Technologies 2023, no. 4 (2023): 500–521. http://dx.doi.org/10.56553/popets-2023-0122.
Full textAbstract:
We present RHODE, a novel system that enables privacy-preserving training of and prediction on Recurrent Neural Networks (RNNs) in a cross-silo federated learning setting by relying on multiparty homomorphic encryption. RHODE preserves the confidentiality of the training data, the model, and the prediction data; and it mitigates federated learning attacks that target the gradients under a passive-adversary threat model. We propose a packing scheme, multi-dimensional packing, for a better utilization of Single Instruction, Multiple Data (SIMD) operations under encryption. With multi-dimensional packing, RHODE enables the efficient processing, in parallel, of a batch of samples. To avoid the exploding gradients problem, RHODE provides several clipping approximations for performing gradient clipping under encryption. We experimentally show that the model performance with RHODE remains similar to non-secure solutions both for homogeneous and heterogeneous data distributions among the data holders. Our experimental evaluation shows that RHODE scales linearly with the number of data holders and the number of timesteps, sub-linearly and sub-quadratically with the number of features and the number of hidden units of RNNs, respectively. To the best of our knowledge, RHODE is the first system that provides the building blocks for the training of RNNs and its variants, under encryption in a federated learning setting.
38
Mienye, Ibomoiye Domor, Theo G. Swart, and George Obaido. "Recurrent Neural Networks: A Comprehensive Review of Architectures, Variants, and Applications." Information 15, no. 9 (2024): 517. http://dx.doi.org/10.3390/info15090517.
Full textAbstract:
Recurrent neural networks (RNNs) have significantly advanced the field of machine learning (ML) by enabling the effective processing of sequential data. This paper provides a comprehensive review of RNNs and their applications, highlighting advancements in architectures, such as long short-term memory (LSTM) networks, gated recurrent units (GRUs), bidirectional LSTM (BiLSTM), echo state networks (ESNs), peephole LSTM, and stacked LSTM. The study examines the application of RNNs to different domains, including natural language processing (NLP), speech recognition, time series forecasting, autonomous vehicles, and anomaly detection. Additionally, the study discusses recent innovations, such as the integration of attention mechanisms and the development of hybrid models that combine RNNs with convolutional neural networks (CNNs) and transformer architectures. This review aims to provide ML researchers and practitioners with a comprehensive overview of the current state and future directions of RNN research.
39
Tiňo, Peter, and Barbara Hammer. "Architectural Bias in Recurrent Neural Networks: Fractal Analysis." Neural Computation 15, no. 8 (2003): 1931–57. http://dx.doi.org/10.1162/08997660360675099.
Full textAbstract:
We have recently shown that when initialized with “small” weights, recurrent neural networks (RNNs) with standard sigmoid-type activation functions are inherently biased toward Markov models; even prior to any training, RNN dynamics can be readily used to extract finite memory machines (Hammer & Tiňo, 2002; Tiňo, Čerňanský, &Beňušková, 2002a, 2002b). Following Christiansen and Chater (1999), we refer to this phenomenon as the architectural bias of RNNs. In this article, we extend our work on the architectural bias in RNNs by performing a rigorous fractal analysis of recurrent activation patterns. We assume the network is driven by sequences obtained by traversing an underlying finite-state transition diagram&a scenario that has been frequently considered in the past, for example, when studying RNN-based learning and implementation of regular grammars and finite-state transducers. We obtain lower and upper bounds on various types of fractal dimensions, such as box counting and Hausdorff dimensions. It turns out that not only can the recurrent activations inside RNNs with small initial weights be explored to build Markovian predictive models, but also the activations form fractal clusters, the dimension of which can be bounded by the scaled entropy of the underlying driving source. The scaling factors are fixed and are given by the RNN parameters.
40
Pan, Yu, Jing Xu, Maolin Wang, et al. "Compressing Recurrent Neural Networks with Tensor Ring for Action Recognition." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 4683–90. http://dx.doi.org/10.1609/aaai.v33i01.33014683.
Full textAbstract:
Recurrent Neural Networks (RNNs) and their variants, such as Long-Short Term Memory (LSTM) networks, and Gated Recurrent Unit (GRU) networks, have achieved promising performance in sequential data modeling. The hidden layers in RNNs can be regarded as the memory units, which are helpful in storing information in sequential contexts. However, when dealing with high dimensional input data, such as video and text, the input-to-hidden linear transformation in RNNs brings high memory usage and huge computational cost. This makes the training of RNNs very difficult. To address this challenge, we propose a novel compact LSTM model, named as TR-LSTM, by utilizing the low-rank tensor ring decomposition (TRD) to reformulate the input-to-hidden transformation. Compared with other tensor decomposition methods, TR-LSTM is more stable. In addition, TR-LSTM can complete an end-to-end training and also provide a fundamental building block for RNNs in handling large input data. Experiments on real-world action recognition datasets have demonstrated the promising performance of the proposed TR-LSTM compared with the tensor-train LSTM and other state-of-the-art competitors.
41
Khan, Hania Nawaz, Sibghatullah Bazai, Zubair Zaland, et al. "A Comparative Study of Convolutional Neural Networks and Recurrent Neural Networks for Chord Recognition." International Journal of Membrane Science and Technology 10, no. 2 (2023): 1617–30. http://dx.doi.org/10.15379/ijmst.v10i2.1837.
Full textAbstract:
Using Mel-spectrograms, this study evaluates the effectiveness of Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNNs). Mel-spectrograms are justified by their non- linearity and similarity to the human hearing system. This study uses over 200 tracks by The Beatles and Queen collected through the Music Information Retrieval Evaluation Exchange. Data augmentation approaches are used to increase accuracy on unusual chords. This paper presents a 3-layer 2D CNN model trained on major and minor chords and then expanded to different types of chords. The dataset demonstrates that both models can recognize musical chords across various genres. We compare the proposed results to the existing literature and demonstrate the effectiveness of the proposed methodology. As a result of our analysis, we found that the CNN and RNN models were 79% and 76% accurate, respectively. The presented findings suggest that CNNs and RNNs are suitable models for chord recognition using Mel-spectrograms. Data augmentation can be an effective technique for improving accuracy on rare chords.
42
SATO, SHOZO, and KAZUTOSHI GOHARA. "FRACTAL TRANSITION IN CONTINUOUS RECURRENT NEURAL NETWORKS." International Journal of Bifurcation and Chaos 11, no. 02 (2001): 421–34. http://dx.doi.org/10.1142/s0218127401002158.
Full textAbstract:
A theory for continuous dynamical systems stochastically excited by temporal external inputs has been presented. The theory suggests that the dynamics of continuous-time recurrent neural networks (RNNs) is generally characterized by a set of continuous trajectories with a fractal-like structure in hyper-cylindrical phase space. We refer to this dynamics as the fractal transition. In this paper, three types of numerical experiments are discussed in order to investigate the learning process and noise effects in terms of the fractal transition. First, to analyze how an RNN learns desired input–output transformations, a simple example with a single state was examined in detail. A fractal structure similar to a Cantor set was clearly observed in the learning process. This finding sheds light on the learning of RNNs, i.e. it suggests that the learning is a process of adjusting the fractal dimension. Second, input noise effects on the fractal structure were investigated. The results show that small-scale hierarchical structures are broken by noise. Third, using a network with twenty states, we show that fractal transition is a universal characteristic of RNNs driven by switching inputs.
43
Zou, Xuetian, Kangli Wang, Jiawei Lu, and Dili Wu. "Time Series Forecasting of Emission Trends Using Recurrent Neural Networks." Computer Life 12, no. 3 (2024): 12–18. http://dx.doi.org/10.54097/ezvnav34.
Full textAbstract:
This paper explores the application of Recurrent Neural Networks (RNNs) for forecasting emission trends, a critical aspect of addressing climate change and formulating effective environmental policies. Traditional forecasting methods, such as Autoregressive Integrated Moving Average (ARIMA) and Exponential Smoothing, often fail to capture the complex nonlinear relationships and temporal dependencies inherent in emission data. In contrast, RNNs, particularly Long Short-Term Memory (LSTM) networks, are designed to recognize patterns in sequential data, making them well-suited for time series forecasting tasks. This study employs a comprehensive methodology that includes data collection from reputable sources, feature selection, RNN model design, and rigorous evaluation using various performance metrics. The results indicate that RNNs significantly outperform traditional forecasting techniques in terms of accuracy, providing valuable insights into future emission trajectories. The findings underscore the potential of RNNs as powerful tools for policymakers and researchers, facilitating more informed decision-making in the fight against climate change.
44
Bucci, Andrea. "Realized Volatility Forecasting with Neural Networks." Journal of Financial Econometrics 18, no. 3 (2020): 502–31. http://dx.doi.org/10.1093/jjfinec/nbaa008.
Full textAbstract:
Abstract In the last few decades, a broad strand of literature in finance has implemented artificial neural networks as a forecasting method. The major advantage of this approach is the possibility to approximate any linear and nonlinear behaviors without knowing the structure of the data generating process. This makes it suitable for forecasting time series which exhibit long-memory and nonlinear dependencies, like conditional volatility. In this article, the predictive performance of feed-forward and recurrent neural networks (RNNs) was compared, particularly focusing on the recently developed long short-term memory (LSTM) network and nonlinear autoregressive model process with eXogenous input (NARX) network, with traditional econometric approaches. The results show that RNNs are able to outperform all the traditional econometric methods. Additionally, capturing long-range dependence through LSTM and NARX models seems to improve the forecasting accuracy also in a highly volatile period.
45
B.Venkateswarlu and Dr. C. Gulzar. "Spam Classification using Recurrent Neural Networks." international journal of engineering technology and management sciences 9, no. 2 (2025): 684–89. https://doi.org/10.46647/ijetms.2025.v09i02.087.
Full textAbstract:
Spam classification is a critical task in email filtering systems to distinguish between legitimate andspam emails. Traditional machine learning methods have been used for this purpose, but they oftenstruggle to capture the complex patterns and variations in spam emails. In this paper, we propose anovel approach using Recurrent Neural Networks (RNNs) for spam classification. RNNs are wellsuited for sequence modeling tasks like this, as they can capture dependencies between words in anemail. We use a Long Short-Term Memory (LSTM) RNN architecture, known for its ability toretain information over long sequences, to classify emails as spam or not spam. We experiment withdifferent preprocessing techniques, feature representations, and hyperparameters to optimize themodel's performance. Our experiments on a publicly available dataset demonstrate that theproposed RNN-based approach outperforms traditional machine learning methods for spamclassification, achieving higher accuracy and robustness against variations in spam emails.
46
Liu, Shiwei, Iftitahu Ni’mah, Vlado Menkovski, Decebal Constantin Mocanu, and Mykola Pechenizkiy. "Efficient and effective training of sparse recurrent neural networks." Neural Computing and Applications 33, no. 15 (2021): 9625–36. http://dx.doi.org/10.1007/s00521-021-05727-y.
Full textAbstract:
AbstractRecurrent neural networks (RNNs) have achieved state-of-the-art performances on various applications. However, RNNs are prone to be memory-bandwidth limited in practical applications and need both long periods of training and inference time. The aforementioned problems are at odds with training and deploying RNNs on resource-limited devices where the memory and floating-point operations (FLOPs) budget are strictly constrained. To address this problem, conventional model compression techniques usually focus on reducing inference costs, operating on a costly pre-trained model. Recently, dynamic sparse training has been proposed to accelerate the training process by directly training sparse neural networks from scratch. However, previous sparse training techniques are mainly designed for convolutional neural networks and multi-layer perceptron. In this paper, we introduce a method to train intrinsically sparse RNN models with a fixed number of parameters and floating-point operations (FLOPs) during training. We demonstrate state-of-the-art sparse performance with long short-term memory and recurrent highway networks on widely used tasks, language modeling, and text classification. We simply use the results to advocate that, contrary to the general belief that training a sparse neural network from scratch leads to worse performance than dense networks, sparse training with adaptive connectivity can usually achieve better performance than dense models for RNNs.
47
Wang, Zengkai, Weizhi Liao, Youzhen Jin, and Zijia Wang. "Performance Guarantees of Recurrent Neural Networks for the Subset Sum Problem." Biomimetics 10, no. 4 (2025): 231. https://doi.org/10.3390/biomimetics10040231.
Full textAbstract:
The subset sum problem is a classical NP-hard problem. Various methods have been developed to address this issue, including backtracking techniques, dynamic programming approaches, branch-and-bound strategies, and Monte Carlo methods. In recent years, researchers have proposed several neural network-based methods for solving combinatorial optimization problems, which have shown commendable performance. However, there has been limited research on the performance guarantees of recurrent neural networks (RNNs) when applied to the subset sum problem. In this paper, we conduct a novel investigation into the performance guarantees of RNNs to solve the subset sum problem for the first time. A construction method for RNNs is developed to compute both exact and approximate solutions of subset sum problems, and the mathematical model of each hidden layer in RNNs is rigorously defined. Furthermore, the correctness of the proposed RNNs is strictly proven through mathematical reasoning, and their performance is thoroughly analyzed. In particular, we prove wNN≥wOPT(1−ε) mathematically, i.e., the errors between the approximate solutions obtained by the proposed ASS-NN model and the actual optimal solutions are relatively small and highly consistent with theoretical expectations. Finally, the validity of RNNs is verified through a series of examples, where the actual error value of the approximate solution aligns closely with the theoretical error value. Additionally, our research reveals that recurrence relations in dynamic programming can effectively simulate the process of constructing solutions.
48
Schmidhuber, Jürgen, Daan Wierstra, Matteo Gagliolo, and Faustino Gomez. "Training Recurrent Networks by Evolino." Neural Computation 19, no. 3 (2007): 757–79. http://dx.doi.org/10.1162/neco.2007.19.3.757.
Full textAbstract:
In recent years, gradient-based LSTM recurrent neural networks (RNNs) solved many previously RNN-unlearnable tasks. Sometimes, however, gradient information is of little use for training RNNs, due to numerous local minima. For such cases, we present a novel method: EVOlution of systems with LINear Outputs (Evolino). Evolino evolves weights to the nonlinear, hidden nodes of RNNs while computing optimal linear mappings from hidden state to output, using methods such as pseudo-inverse-based linear regression. If we instead use quadratic programming to maximize the margin, we obtain the first evolutionary recurrent support vector machines. We show that Evolino-based LSTM can solve tasks that Echo State nets (Jaeger, 2004a) cannot and achieves higher accuracy in certain continuous function generation tasks than conventional gradient descent RNNs, including gradient-based LSTM.
49
Szkoła, Jarosław, Krzysztof Pancerz, and Jan Warchoł. "Recurrent Neural Networks in Computer-Based Clinical Decision Support for Laryngopathies: An Experimental Study." Computational Intelligence and Neuroscience 2011 (2011): 1–8. http://dx.doi.org/10.1155/2011/289398.
Full textAbstract:
The main goal of this paper is to give the basis for creating a computer-based clinical decision support (CDS) system for laryngopathies. One of approaches which can be used in the proposed CDS is based on the speech signal analysis using recurrent neural networks (RNNs). RNNs can be used for pattern recognition in time series data due to their ability of memorizing some information from the past. The Elman networks (ENs) are a classical representative of RNNs. To improve learning ability of ENs, we may modify and combine them with another kind of RNNs, namely, with the Jordan networks. The modified Elman-Jordan networks (EJNs) manifest a faster and more exact achievement of the target pattern. Validation experiments were carried out on speech signals of patients from the control group and with two kinds of laryngopathies.
50
Perevedentsev, O. V., О. I. Orlov, and R. V. Chernogorov. "APPLICATION OF RECURRENT NEURON NETWORKS IN PROGNOSTIC ASSESSMENT OF MEDICAL MONITORING DATA FROM PARTICIPANTS IN ISOLATION STUDY «SIRIUS-21»." Aerospace and Environmental Medicine 57, no. 2 (2023): 33–38. http://dx.doi.org/10.21687/0233-528x-2023-57-2-33-38.
Full textAbstract:
The paper deals with analysis of the possibility to apply artificial neural networks for health evaluation in the series of isolation studies launched at IBMP in 2017 (SIRIUS project). Indeed, recurrent neural networks (RNNs) with an original architecture increased precision of forecasting health dynamics. Daily self check-ups by spacecrew members with the use of RNNs may help detect negative trends in organism and undertake preventive actions.