To see the other types of publications on this topic, follow the link: Adam optimization.
Journal articles on the topic 'Adam optimization'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Adam optimization.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Shao, Yichuan, Jiantao Wang, Haijing Sun, et al. "An Improved BGE-Adam Optimization Algorithm Based on Entropy Weighting and Adaptive Gradient Strategy." Symmetry 16, no. 5 (2024): 623. http://dx.doi.org/10.3390/sym16050623.
Full textAbstract:
This paper introduces an enhanced variant of the Adam optimizer—the BGE-Adam optimization algorithm—that integrates three innovative technologies to augment the adaptability, convergence, and robustness of the original algorithm under various training conditions. Firstly, the BGE-Adam algorithm incorporates a dynamic β parameter adjustment mechanism that utilizes the rate of gradient variations to dynamically adjust the exponential decay rates of the first and second moment estimates (β1 and β2), the adjustment of β1 and β2 is symmetrical, which means that the rules that the algorithm considers when adjusting β1 and β2 are the same. This design helps to maintain the consistency and balance of the algorithm, allowing the optimization algorithm to adaptively capture the trending movements of gradients. Secondly, it estimates the direction of future gradients by a simple gradient prediction model, combining historic gradient information with the current gradient. Lastly, entropy weighting is integrated into the gradient update step. This strategy enhances the model’s exploratory nature by introducing a certain amount of noise, thereby improving its adaptability to complex loss surfaces. Experimental results on classical datasets, MNIST and CIFAR10, and gastrointestinal disease medical datasets demonstrate that the BGE-Adam algorithm has improved convergence and generalization capabilities. In particular, on the specific medical image gastrointestinal disease test dataset, the BGE-Adam optimization algorithm achieved an accuracy of 69.36%, a significant improvement over the 67.66% accuracy attained using the standard Adam algorithm; on the CIFAR10 test dataset, the accuracy of the BGE-Adam algorithm reached 71.4%, which is higher than the 70.65% accuracy of the Adam optimization algorithm; and on the MNIST dataset, the BGE-Adam algorithm’s accuracy was 99.34%, surpassing the Adam optimization algorithm’s accuracy of 99.23%. The BGE-Adam optimization algorithm exhibits better convergence and robustness. This research not only demonstrates the effectiveness of the combination of these three technologies but also provides new perspectives for the future development of deep learning optimization algorithms.
2
Shao, Yichuan, Jiapeng Yang, Wen Zhou, et al. "An Improvement of Adam Based on a Cyclic Exponential Decay Learning Rate and Gradient Norm Constraints." Electronics 13, no. 9 (2024): 1778. http://dx.doi.org/10.3390/electronics13091778.
Full textAbstract:
Aiming at a series of limitations of the Adam algorithm, such as hyperparameter sensitivity and unstable convergence, in this paper, an improved optimization algorithm, the Cycle-Norm-Adam (CN-Adam) algorithm, is proposed. The algorithm integrates the ideas of a cyclic exponential decay learning rate (CEDLR) and gradient paradigm constraintsand accelerates the convergence speed of the Adam model and improves its generalization performance by dynamically adjusting the learning rate. In order to verify the effectiveness of the CN-Adam algorithm, we conducted extensive experimental studies. The CN-Adam algorithm achieved significant performance improvementsin both standard datasets. The experimental results show that the CN-Adam algorithm achieved 98.54% accuracy in the MNIST dataset and 72.10% in the CIFAR10 dataset. Due to the complexity and specificity of medical images, the algorithm was tested in a medical dataset and achieved an accuracy of 78.80%, which was better than the other algorithms. The experimental results show that the CN-Adam optimization algorithm provides an effective optimization strategy for improving model performance and promoting medical research.
3
Jais, Imran Khan Mohd, Amelia Ritahani Ismail, and Syed Qamrun Nisa. "Adam Optimization Algorithm for Wide and Deep Neural Network." Knowledge Engineering and Data Science 2, no. 1 (2019): 41. http://dx.doi.org/10.17977/um018v2i12019p41-46.
Full textAbstract:
The objective of this research is to evaluate the effects of Adam when used together with a wide and deep neural network. The dataset used was a diagnostic breast cancer dataset taken from UCI Machine Learning. Then, the dataset was fed into a conventional neural network for a benchmark test. Afterwards, the dataset was fed into the wide and deep neural network with and without Adam. It was found that there were improvements in the result of the wide and deep network with Adam. In conclusion, Adam is able to improve the performance of a wide and deep neural network.
4
Eren Dio Sefrila, Basuki Rahmat, and Andreas Nugroho Sihananto. "Implementasi Arsitektur Inception V3 Dengan Optimasi Adam, SGD dan RMSP Pada Klasifikasi Penyakit Malaria." Bridge : Jurnal publikasi Sistem Informasi dan Telekomunikasi 2, no. 2 (2024): 69–84. http://dx.doi.org/10.62951/bridge.v2i2.62.
Full textAbstract:
In the current era of technological advancement, deep learning has become a widely discussed and utilized topic, particularly in image classification, object detection, and natural language processing. A significant development in deep learning is the Convolutional Neural Network (CNN), which is enhanced with various optimizations such as Adam, RMSProp, and SGD. This thesis implements the Inception v3 architecture for the deep learning model, utilizing these three optimization methods to classify malaria disease. The study aims to evaluate performance and determine the best optimization based on classification accuracy. The results indicate that the SGD optimization with a learning rate of 0.001 achieved an accuracy of 94%, RMSProp with learning rates of 0.001 and 0.0001 achieved an accuracy of 96%, and Adam with learning rates of 0.001 and 0.0001 achieved an accuracy of 95%.
5
Ma, Jerry, and Denis Yarats. "On the Adequacy of Untuned Warmup for Adaptive Optimization." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 10 (2021): 8828–36. http://dx.doi.org/10.1609/aaai.v35i10.17069.
Full textAbstract:
Adaptive optimization algorithms such as Adam (Kingma and Ba, 2014) are widely used in deep learning. The stability of such algorithms is often improved with a warmup schedule for the learning rate. Motivated by the difficulty of choosing and tuning warmup schedules, recent work proposes automatic variance rectification of Adam's adaptive learning rate, claiming that this rectified approach ("RAdam") surpasses the vanilla Adam algorithm and reduces the need for expensive tuning of Adam with warmup. In this work, we refute this analysis and provide an alternative explanation for the necessity of warmup based on the magnitude of the update term, which is of greater relevance to training stability. We then provide some "rule-of-thumb" warmup schedules, and we demonstrate that simple untuned warmup of Adam performs more-or-less identically to RAdam in typical practical settings. We conclude by suggesting that practitioners stick to linear warmup with Adam, with a sensible default being linear warmup over 2 / (1 - β₂) training iterations.
6
Kamble, Arvind, and Virendra S. Malemath. "Adam Improved Rider Optimization-Based Deep Recurrent Neural Network for the Intrusion Detection in Cyber Physical Systems." International Journal of Swarm Intelligence Research 13, no. 3 (2022): 1–22. http://dx.doi.org/10.4018/ijsir.304402.
Full textAbstract:
This paper designed the intrusion detection systems for determining the intrusions. Here, Adam Improved rider optimization approach (Adam IROA) is newly developed for detecting the intrusion in intrusion detection. Accordingly, the training of DeepRNN is done by proposed Adam IROA, which is designed by combining the Adam optimization algorithm with IROA. Thus, the newly developed Adam IROA is applied for intrusion detection. Overall, two phases are included in the proposed intrusion detection system, which involves feature selection and classification. Here, the features selection is done using proposed WWIROA to select significant features from the input data. The proposed WWIROA is developed by combining WWO and IROA. The obtained features are fed to the classification module for discovering the intrusions present in the network. Here, the classification is progressed using Adam IROA-based DeepRNN. The proposed Adam IROA-based DeepRNN achieves maximal accuracy of 0.937, maximal sensitivity of 0.952, and maximal specificity of 0.908 based on SCADA dataset.
7
Su, Stephanie S. W., and Sie Long Kek. "An Improvement of Stochastic Gradient Descent Approach for Mean-Variance Portfolio Optimization Problem." Journal of Mathematics 2021 (March 25, 2021): 1–10. http://dx.doi.org/10.1155/2021/8892636.
Full textAbstract:
In this paper, the current variant technique of the stochastic gradient descent (SGD) approach, namely, the adaptive moment estimation (Adam) approach, is improved by adding the standard error in the updating rule. The aim is to fasten the convergence rate of the Adam algorithm. This improvement is termed as Adam with standard error (AdamSE) algorithm. On the other hand, the mean-variance portfolio optimization model is formulated from the historical data of the rate of return of the S&P 500 stock, 10-year Treasury bond, and money market. The application of SGD, Adam, adaptive moment estimation with maximum (AdaMax), Nesterov-accelerated adaptive moment estimation (Nadam), AMSGrad, and AdamSE algorithms to solve the mean-variance portfolio optimization problem is further investigated. During the calculation procedure, the iterative solution converges to the optimal portfolio solution. It is noticed that the AdamSE algorithm has the smallest iteration number. The results show that the rate of convergence of the Adam algorithm is significantly enhanced by using the AdamSE algorithm. In conclusion, the efficiency of the improved Adam algorithm using the standard error has been expressed. Furthermore, the applicability of SGD, Adam, AdaMax, Nadam, AMSGrad, and AdamSE algorithms in solving the mean-variance portfolio optimization problem is validated.
8
Irfan, Desi, Teddy Surya Gunawan, and Wanayumini Wanayumini. "Comparison Of SGD, Rmsprop, and Adam Optimation In Animal Classification Using CNNs." International Conference on Information Science and Technology Innovation (ICoSTEC) 2, no. 1 (2023): 45–51. http://dx.doi.org/10.35842/icostec.v2i1.35.
Full textAbstract:
Many measures have been taken to protect endangered species by using "camera trap" technology which is widespread in the field of technology-based nature protection field research. In this study, a machine learning-based approach is presented to identify endangered wildlife images with a data set containing 5000 images taken from Kaggle and some other sources. The Gradient Descent optimization method is often used for Artificial Neural Network (ANN) training. This method plays a role in finding the weight values that give the best output value. Three optimization methods have been implemented, namely Stochastic Gradient Descent (SGD), ADADELTA, and Adam on the Artificial Neural Network system for animal data classification. In some of the studies reviewed there are differences in the results of SGD and ADAM, which on the one hand SGD is superior, and on the one hand ADAM is superior with the appropriate learning rate. The results of this study show that the CNN method with the Adam optimization function produces the highest accuracy compared to the SGD and RMSprop optimization methods. The model trained using Adam's optimization function achieved an accuracy of 89.81% on the test, showing the feasibility of the approach.
9
Wang, Yijun, Pengyu Zhou, and Wenya Zhong. "An Optimization Strategy Based on Hybrid Algorithm of Adam and SGD." MATEC Web of Conferences 232 (2018): 03007. http://dx.doi.org/10.1051/matecconf/201823203007.
Full textAbstract:
Despite superior training outcomes, adaptive optimization methods such as Adam, Adagrad or RMSprop have been found to generalize poorly compared to stochastic gradient descent (SGD). So scholars (Nitish Shirish Keskar et al., 2017) proposed a hybrid strategy to start training with Adam and switch to SGD at the right time. In the learning task with a large output space, it was observed that Adam could not converge to an optimal solution (or could not converge to an extreme point in a non-convex scene) [1]. Therefore, this paper proposes a new variant of the ADAM algorithm (AMSGRAD), which not only solves the convergence problem, but also improves the empirical performance.
10
Zhang, Can, Yichuan Shao, Haijing Sun, Lei Xing, Qian Zhao, and Le Zhang. "The WuC-Adam algorithm based on joint improvement of Warmup and cosine annealing algorithms." Mathematical Biosciences and Engineering 21, no. 1 (2023): 1270–85. http://dx.doi.org/10.3934/mbe.2024054.
Full textAbstract:
<abstract> <p>The Adam algorithm is a common choice for optimizing neural network models. However, its application often brings challenges, such as susceptibility to local optima, overfitting and convergence problems caused by unstable learning rate behavior. In this article, we introduce an enhanced Adam optimization algorithm that integrates Warmup and cosine annealing techniques to alleviate these challenges. By integrating preheating technology into traditional Adam algorithms, we systematically improved the learning rate during the initial training phase, effectively avoiding instability issues. In addition, we adopt a dynamic cosine annealing strategy to adaptively adjust the learning rate, improve local optimization problems and enhance the model's generalization ability. To validate the effectiveness of our proposed method, extensive experiments were conducted on various standard datasets and compared with traditional Adam and other optimization methods. Multiple comparative experiments were conducted using multiple optimization algorithms and the improved algorithm proposed in this paper on multiple datasets. On the MNIST, CIFAR10 and CIFAR100 datasets, the improved algorithm proposed in this paper achieved accuracies of 98.87%, 87.67% and 58.88%, respectively, with significant improvements compared to other algorithms. The experimental results clearly indicate that our joint enhancement of the Adam algorithm has resulted in significant improvements in model convergence speed and generalization performance. These promising results emphasize the potential of our enhanced Adam algorithm in a wide range of deep learning tasks.</p> </abstract>
11
S, Selvakumari, and Durairaj M. "A Comparative Study of Optimization Techniques in Deep Learning Using the MNIST Dataset." Indian Journal of Science and Technology 18, no. 10 (2025): 803–10. https://doi.org/10.17485/IJST/v18i10.121.
Full textAbstract:
Abstract <strong>Objectives:</strong> To evaluate the performance of four standard optimization algorithms - Stochastic Gradient Descent, RMSProp, Adam, and AdamW – on the MNIST image classification benchmark using Convolutional Neural Networks (CNNs). It examines the effects of L2 regularization on convergence speed, test accuracy, and generalizability, focusing on the AdamW optimizer. Finally, this study investigates the sensitivity of every optimization algorithm for different learning rates. <strong>Methods:</strong> CNNs were trained on the MNIST dataset using each of the four optimization algorithms, and experiments were performed. Once the best hyperparameter was found, L2 regularization was used to determine how the performance metrics would be affected, which is one specific focus for AdamW. Different learning rates were tested to test the robustness and adaptability of each optimizer. <strong>Findings:</strong> The results revealed that AdamW with weight decay outperformed the remaining optimizer-based training regarding test accuracy and validation loss. AdamW had the best generalization performance despite its suitable regularization and SGD and RMSProp being competitive. Moreover, AdamW was more robust over a broader range of learning rates than the other optimization algorithms. <strong>Novelty:</strong> This work underscores the importance of a lesson that gives more attention to kernel methods in that the role of regularization must not be ignored when describing the generalization performance of deep learning tasks. It analyzes various optimizers against each other at different learning rates and shows that AdamW utilizes L2 regularization to achieve even better performance and robustness. <strong>Keywords:</strong> Stochastic Gradient Descent, RMSProp, Adam, AdamW, Convolutional Neural Networks
12
Rizky Amalia and Febriyanti Panjaitan. "Mask Detection Using Convolutional Neural Network Algorithm." Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) 6, no. 4 (2022): 639–47. http://dx.doi.org/10.29207/resti.v6i4.4276.
Full textAbstract:
The World Health Organizations and the Ministry of Health of the Republic of Indonesia have required the use of masks to suppress the spread of COVID-19. WHO provides guidance on how to use a good mask to cover the mouth and nose. This study aims to detect the correct use of masks using the Convolutional Neural Network. CNN is a popular Deep Learning algorithm for image data classification problems. The Mask Usage Detector is built with the help of a pre-trained MobileNetV2 model with an architecture that supports media that has minimum computations. This study will also compare the performance of three optimization methods from CNN, namely Adam, SGD, and RMSprop in detecting the use of masks. Performance will be seen from the test results by analyzing the values of accuracy, precision, and recall. The dataset used is in the form of image data of 2,029 images for 2 categories, namely "masked" and "unmasked". A total of 1,623 images were used as training data and 406 images for test data. Based on the testing process, the accuracy of each optimization is 93.84% with Adam optimization, 84.48% with SGD optimization, and 93.10% with RMSprop optimization. With the proposed model, this study obtains the performance results of the three CNN optimizations, and it is concluded that adam's optimization gives better performance results than the other two optimizations.
13
Song, Ci. "The performance analysis of Adam and SGD in image classification and generation tasks." Applied and Computational Engineering 5, no. 1 (2023): 757–63. http://dx.doi.org/10.54254/2755-2721/5/20230697.
Full textAbstract:
Optimization problems have a very important leading position in machine learning. A great deal of machine learning algorithms ends up solving optimization problems. Among all the optimization algorithms, gradient methods are the simplest and most commonly used compared to algorithms like Particle Swarm Optimization and Ant Colony Optimization. In the gradient methods, Adaptive Moment Estimation (Adam) and stochastic parallel gradient descent (SGD) are both outstanding algorithms that have helped solve all kinds of deep learning tasks. But which one is better in some certain conditions is still unknown, which means programmers need to try many of the optimizers to have the best choice. Based on some previous researches, this paper study the impact of L2 regularization and weight decay in Adam and SGD with momentum, which turns out in adaptive methods, L2 regularization is not as effective as it is in SGD. It gives the intuition that SGD should outperform Adam in image classification tasks. However, this paper finds things go the other way around by running an experiment using Lenet-5 on MINST. Besides, this paper describes an experiment on Fashion-MINST using DCGAN with both Adam and SGD as optimizers in Generator and Discriminator. The result shows that the generator with SGD produces fake images with higher quality.
14
Rahman, Alrafiful, Lucia Sri Istiyowati, Valentinus Valentinus, Ivan Ivan, and Zainal Azis. "IMPLEMENTASI DATA MINING DALAM PREDIKSI HARGA SAHAM BBNI DENGAN PEMODELAN MATEMATIKA MENGGUNAKAN METODE LSTM DENGAN OPTIMASI ADAM." JUTECH : Journal Education and Technology 5, no. 2 (2024): 427–39. https://doi.org/10.31932/jutech.v5i2.4137.
Full textAbstract:
Stock price prediction plays a crucial role in investment decision-making, allowing investors to maximize profits and minimize risks. This study implements the Long Short-Term Memory (LSTM) method with Adam optimization to predict the stock price of Bank Negara Indonesia (BBNI) based on historical stock price data from the Indonesia Stock Exchange (2001-2023). LSTM is chosen for its ability to handle sequential data and identify long-term patterns in time series. Meanwhile, the Adam optimization algorithm is used to accelerate model convergence and improve prediction accuracy. The data used includes daily stock prices (closing prices), and the research process involves data collection, preprocessing, LSTM model creation, Adam optimization, training, evaluation, and prediction. The experimental results show that the model with a batch size of 64 and 100 epochs yields an R² of 0.9928 and a MAPE of 1.53%, indicating a very high prediction accuracy. With an accuracy of 98.46%, the LSTM model with Adam optimization proves to be effective in predicting stock prices, providing excellent results for applications in investment strategies. This study demonstrates the great potential of applying data mining and machine learning techniques in more informed and data-driven stock market analysis.
15
Tang, Qiaoyue, Frederick Shpilevskiy, and Mathias Lécuyer. "DP-AdamBC: Your DP-Adam Is Actually DP-SGD (Unless You Apply Bias Correction)." Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 14 (2024): 15276–83. http://dx.doi.org/10.1609/aaai.v38i14.29451.
Full textAbstract:
The Adam optimizer is a popular choice in contemporary deep learning due to its strong empirical performance. However we observe that in privacy sensitive scenarios, the traditional use of Differential Privacy (DP) with the Adam optimizer leads to sub-optimal performance on several tasks. We find that this performance degradation is due to a DP bias in Adam's second moment estimator, introduced by the addition of independent noise in the gradient computation to enforce DP guarantees. This DP bias leads to a different scaling for low variance parameter updates, that is inconsistent with the behavior of non-private Adam, and Adam's sign descent interpretation. We propose the DP-AdamBC optimization algorithm, which corrects for the bias in the second moment estimation and retrieves the expected behaviour of Adam. Empirically, DP-AdamBC significantly improves the optimization performance of DP-Adam by up to 3.5% in final accuracy in image, text, and graph node classification tasks.
16
Liu, Zhipeng, Rui Feng, Xiuhan Li, Wei Wang, and Xiaoling Wu. "Gradient-Sensitive Optimization for Convolutional Neural Networks." Computational Intelligence and Neuroscience 2021 (March 22, 2021): 1–16. http://dx.doi.org/10.1155/2021/6671830.
Full textAbstract:
Convolutional neural networks (CNNs) are effective models for image classification and recognition. Gradient descent optimization (GD) is the basic algorithm for CNN model optimization. Since GD appeared, a series of improved algorithms have been derived. Among these algorithms, adaptive moment estimation (Adam) has been widely recognized. However, local changes are ignored in Adam to some extent. In this paper, we introduce an adaptive learning rate factor based on current and recent gradients. According to this factor, we can dynamically adjust the learning rate of each independent parameter to adaptively adjust the global convergence process. We use the factor to adjust the learning rate for each parameter. The convergence of the proposed algorithm is proven by using the regret bound approach of the online learning framework. In the experimental section, comparisons are conducted between the proposed algorithm and other existing algorithms, such as AdaGrad, RMSprop, Adam, diffGrad, and AdaHMG, on test functions and the MNIST dataset. The results show that Adam and RMSprop combined with our algorithm can not only find the global minimum faster in the experiment using the test function but also have a better convergence curve and higher test set accuracy in experiments using datasets. Our algorithm is a supplement to the existing gradient descent algorithms, which can be combined with many other existing gradient descent algorithms to improve the efficiency of iteration, speed up the convergence of the cost function, and improve the final recognition rate.
17
Moraes, André Magalhães, Luiz Felipe Pugliese, Rafael Francisco dos Santos, Giovani Bernardes Vitor, Rodrigo Aparecido da Silva Braga, and Fernanda Rodrigues da Silva. "Effectiveness of YOLO Architectures in Tree Detection: Impact of Hyperparameter Tuning and SGD, Adam, and AdamW Optimizers." Standards 5, no. 1 (2025): 9. https://doi.org/10.3390/standards5010009.
Full textAbstract:
This study investigates the optimization of tree detection in static images using YOLOv5, YOLOv8, and YOLOv11 models, leveraging a custom non-standard image bank created exclusively for this research. Objectives: To enhance tree detection in static images by comparing the performance of YOLOv5, YOLOv8, and YOLOv11 models. The comparison involved hyperparameter tuning and the application of various optimizers, aiming to improve model performance in terms of precision, recall, F1, and mean average precision (mAP). Design/Methodology/Approach: A custom image bank was utilized to train YOLOv5, YOLOv8, and YOLOv11 models. During training, the hyperparameters’ learning rate and momentum were tuned in combination with the optimizers SGD, Adam, and AdamW. Performance metrics, including precision, recall, F1, and mAP, were analyzed for each configuration. Key Results: The optimization process achieved precision values of 100% with Adam for YOLOv8 and SGD for YOLOv11, and recall of 91.5% with AdamW on YOLOv8. Additionally, mAP values reached 95.6% for AdamW on YOLOv8 and 95.2% for SGD on YOLOv11. Convergence times for mAP were also significantly reduced, demonstrating faster training and enhanced overall model performance. Originality/Research gap: This study addresses a gap in tree detection using YOLO models trained on non-standard image banks, a topic that is less commonly explored in the literature. The exclusive development of a custom image bank further adds novelty to the research. Practical Implications: The findings underscore the effectiveness of model optimization in tree detection tasks using custom datasets. This methodology could be extended to other applications requiring object detection in non-standard image banks. Limitations of the investigation: This study is limited to tree detection within a single custom dataset and does not evaluate the generalizability of these optimizations to other datasets or object detection tasks.
18
T S, Steni Mol, and P. S. Sreeja. "Adam Adadelta Optimization based bidirectional encoder representations from transformers model for fake news detection on social media." Multiagent and Grid Systems 19, no. 3 (2023): 271–87. http://dx.doi.org/10.3233/mgs-230033.
Full textAbstract:
Social platform have disseminated the news in rapid speed and has been considered an important news resource for many people over worldwide because of easy access and less cost benefits when compared with the traditional news organizations. Fake news is the news deliberately written by bad writers that manipulates the original contents and this rapid dissemination of fake news may mislead the people in the society. As a result, it is critical to investigate the veracity of the data leaked via social media platforms. Even so, the reliability of information reported via this platform is still doubtful and remains a significant obstacle. As a result, this study proposes a promising technique for identifying fake information in social media called Adam Adadelta Optimization based Deep Long Short-Term Memory (Deep LSTM). The tokenization operation in this case is carried out with the Bidirectional Encoder Representations from Transformers (BERT) approach. The measurement of the features is reduced with the assistance of Kernel Linear Discriminant Analysis (LDA), and Singular Value Decomposition (SVD) and the top-N attributes are chosen by employing Renyi joint entropy. Furthermore, the LSTM is applied to identify false information in social media, with Adam Adadelta Optimization, which comprises a combo of Adam Optimization and Adadelta Optimization . The Deep LSTM based on Adam Adadelta Optimization achieved maximum accuracy, sensitivity, specificity of 0.936, 0.942, and 0.925.
19
Mandasari, Sartika, Desi Irfan, Wanayumini Wanayumini, and Rika Rosnelly. "COMPARISON OF SGD, ADADELTA, ADAM OPTIMIZATION IN GENDER CLASSIFICATION USING CNN." JURTEKSI (Jurnal Teknologi dan Sistem Informasi) 9, no. 3 (2023): 345–54. http://dx.doi.org/10.33330/jurteksi.v9i3.2067.
Full textAbstract:
Abstract: Gender classification is one of the most important tasks of video analysis. A machine learning-based approach was presented to identify male and female facial images with a data set of 2000 images taken from kaggles. This method plays a role in finding the weight value that gives the best output value. This study uses the most appropriate learning rate of each optimization method as a criterion for stopping training. The results showed that the Artificial Neural Network with Adam optimization produced the highest accuracy, which was 91.5% compared to the SGD and ADADELTA optimization methods. Deep Learning techniques that are applied extensively to image recognition used utilize Adam's optimizer method. Keywords: artificial neural networks; adadelta; adam; gender; sgm; Abstrak: Klasifikasi gender adalah salah satu tugas analisis video yang paling penting. Pendekatan berbasis machine learning disajikan untuk mengidentifikasi gambar wajah Pria dan Wanita dengan kumpulan data sebanyak 2000 gambar yang diambil dari kaggle. Metode ini berperan dalam menemukan nilai bobot yang memberikan nilai keluaran terbaik. Penelitian ini menggunakan learning rate yang paling sesuai dari masing-masing metode optimasi sebagai kriteria pemberhentian pelatihan. Hasil penelitian menunjukkan Jaringan Saraf Tiruan dengan optimasi Adam menghasilkan akurasi tertinggi yaitu 91,5 % dibandingkan dengan dengan metode optimasi SGD dan ADADELTA. Teknik Deep Learning yang diterapkan secara ekstensif pada pengenalan gambar yang digunakan memanfaatkan metode optimizer Adam. Kata kunci: Adadelta; Adam; Jaringan Syaraf; Gender; Tiruan; SGM;
20
Irfan, Desi, Rika Rosnelly, Masri Wahyuni, Jaka Tirta Samudra, and Aditia Rangga. "PERBANDINGAN OPTIMASI SGD, ADADELTA, DAN ADAM DALAM KLASIFIKASI HYDRANGEA MENGGUNAKAN CNN." JOURNAL OF SCIENCE AND SOCIAL RESEARCH 5, no. 2 (2022): 244. http://dx.doi.org/10.54314/jssr.v5i2.789.
Full textAbstract:
Abstract - invasive species are threatening indigenous species habitat in many countries around the world. Nowadays, the monitoring method relies on scientists. Scientists are skilled to see the determined areas and record the living species. Applying high skill labors requires high cost, inefficient time and limited scope as the large area cannot be reached by the man. In this research, engine based learning approach was presented to identify the image of invasive hydrangea (indigenous species from Asia) with data collection around 800 images taken form the Brazil national forest and Hydrangea appears in some images. Gradient Descent optimization method is frequently used for artificial neural network. This method roles to discover standard grade for the best output. The Gradient Descent method role play is minimizing the cost function grade by changing the parameter grade step by step. Three optimization methods have been implemented namely Stochastic Gradient Descent (SGD), ADADELTA, and Adam in the artificial neural network (Ann) for classifying aritmia data [32]. This research used the most suitable error grade limitation from each optimization method as the indicators at the end of the training. The result of this research showed that artificial nerve tissue using Adam optimization gets the highest accuration compared with SDG and ADADELTA optimization methods. Deep Learning Technique applied extensively in image introduction is Adam optimization. The training model has reached accuration to 83, 5 % and showed properness of approach conducted. Keyword: SGD, Adadelta, Adam, Optimizer FunctionAbstrak— Spesies invasif mengancam habitat spesies asli di banyak negara di dunia. Saat ini dalam metode pemantauan mereka tergantung pada pengetahuan ahli. Ilmuwan terlatih mengunjungi area yang ditentukan dan mencatat spesies yang menghuninya. Menggunakan tenaga kerja berkualifikasi tinggi seperti itu membutuhkan biaya yang mahal, tidak efisien waktu dan jangkauan yang terbatas karena manusia tidak dapat mencakup area yang luas. Dalam makalah ini, pendekatan berbasis pembelajaran mesin disajikan untuk mengidentifikasi gambar hydrangea invasif (spesies invasif asli Asia) dengan kumpulan data yang berisi sekitar 800 gambar yang diambil di hutan nasional Brasil dan di beberapa gambar terdapat Hydrangea.  Metode optimasi Gradient Descent sering digunakan untuk pelatihan Jaringan Syaraf Tiruan (JST). Metode ini berperan dalam menemukan nilai bobot yang memberikan nilai keluaran terbaik. Prinsip kerja metode Gradient Descent adalah memperkecil nilai fungsi biaya dengan mengubah nilai parameter selangkah demi selangkah. Telah diimplementasikan tiga buah metode optimasi yaitu Stochastic Gradient Descent (SGD), ADADELTA, dan Adam pada sistem Jaringan Saraf Tiruan untuk klasifikasi data aritmia [32]. Penelitian ini menggunakan batas nilai kesalahan yang paling sesuai dari masing-masing metode optimasi  sebagai kriteria pemberhentian pelatihan. Hasil penelitian menunjukkan Jaringan Saraf Tiruan dengan optimasi Adam menghasilkan akurasi tertinggi dibandingkan dengan dengan metode optimasi SGD dan ADADELTA.Teknik Deep Learning  yang diterapkan secara ekstensif pada pengenalan gambar yang digunakan memanfaatkan metode optimizer Adam  . Model yang kita latih menggunakan fungsi optimisasi Adam mencapai akurasi 83,5% pada tes yang lakukan, menunjukkan kelayakan pada  pendekatan yang dilakukan .Kata Kunci— SGD, Adadelta, Adam, Fungsi Optimasi
21
Hidajat, Moch Sjamsul, and Dibyo Adi Wibowo. "Covid-19 Classification using Convolutional Neural Networks Based on Adam, RMSP, and SGD Optimalization." Journal of Applied Intelligent System 8, no. 3 (2023): 442–50. http://dx.doi.org/10.33633/jais.v8i3.9492.
Full textAbstract:
In this comprehensive study, a meticulous analysis of the application of Convolutional Neural Network (CNN) methodologies in the classification of Covid-19 and non-Covid-19 cases was conducted. Leveraging diverse optimization techniques such as RMS, SGD, and Adam, the research systematically evaluated the performance of the CNN model in accurately discerning intricate patterns and distinct features associated with Covid-19 pathology. the implementation of the RMS and Adam optimization methods resulted in the highest accuracy levels, with both models achieving an impressive 98% accuracy in the classification of Covid-19 and non-Covid-19 cases. Leveraging the robust capabilities of these optimization techniques, the study successfully demonstrated the effectiveness of the RMS and Adam models in enhancing the precision and reliability of the Convolutional Neural Network (CNN) for the accurate identification and differentiation of Covid-19 patterns within the medical imaging datasets. The notable achievement of 98% accuracy further emphasizes the potential of these optimization methods in advancing the capabilities of CNN-based diagnostic tools, thus contributing significantly to the ongoing efforts in Covid-19 diagnosis and management.
22
Сіряк, Р. В., І. С. Скарга-Бандурова, and T. O. Білобородова. "Towards an empirical hyperparameters optimization in CNN." ВІСНИК СХІДНОУКРАЇНСЬКОГО НАЦІОНАЛЬНОГО УНІВЕРСИТЕТУ імені Володимира Даля, no. 5(253) (September 5, 2019): 87–91. http://dx.doi.org/10.33216/1998-7927-2019-253-5-87-91.
Full textAbstract:
The necessity of creating a model of recognition of gestures based on convolutional neural network that effective not only in pattern recognition, but also in terms of learning speed and resource intensity, is substantiated. In this regard, the work solved the problem of optimization of hyperparameters and the selection of the best optimizer backpropagation errors. To implement the tasks, a model was created that can recognize hand gestures, both from a single image and from streaming video.When choosing an optimizer, two adaptive methods were tested - Adadelta and Adam. The experiments confirmed the high efficiency of Adadelta, however, when compared with Adam, it showed more than twice as long network training.
23
Ajmi, Nouf Ali AL, and Muhammad Shoaib. "Optimization Strategies in Quantum Machine Learning: A Performance Analysis." Applied Sciences 15, no. 8 (2025): 4493. https://doi.org/10.3390/app15084493.
Full textAbstract:
This study presents a comprehensive comparison of multiple optimization algorithms applied to a quantum classification model, utilizing the Cleveland dataset. Specifically, the research focuses on three prominent optimizers—COBYLA, L-BFGS-B, and ADAM—each employing distinct methodologies and widely recognized in the domain of quantum machine learning. The performance of predictive models using these optimizers is rigorously evaluated through key metrics, including accuracy, precision, recall, and F1 score. The findings reveal that the COBYLA optimizer outperforms the L-BFGS-B and ADAM optimizers across all performance metrics, achieving an accuracy of 92%, precision of 89%, recall of 97%, and F1 score of 93%. Furthermore, the COBYLA optimizer exhibits superior computational efficiency, requiring only 1 min of training time compared to 6 min for L-BFGS-B and 10 min for ADAM. These results underscore the critical role played by optimizer selection in enhancing model performance and efficiency in quantum machine learning applications, offering valuable insights for practitioners in the field.
24
Guoxing Si. "Research on the Data-Driven Differential Equation-Solving Algorithm Based on Artificial Intelligence." Journal of Electrical Systems 20, no. 6s (2024): 145–57. http://dx.doi.org/10.52783/jes.2624.
Full textAbstract:
Data-driven DEs has gained popularity in the past few years. This work proposes the new framework, named Adam Gannet Optimization Algorithm (AdamGOA), that combines Adam Optimization and Gannet Optimization Algorithm (GOA) to improve a stability, solve higher order Differential Equations (DE) and accuracy of DE. Adam is a first-order gradient-based methods, optimizes stochastic objectives using adaptive lower-order moments. In contrast, GOA represents a different distinct action of a gannets mathematically during foraging and is employed to facilitate exploitation and exploration. In addition, a Shepard Convolutional Neural Network (ShCNN) processed data to construct meta-data and estimate derivatives. After that, the unified integral form is established to determine optimal structure. Heterogeneous parameters are used to estimate and are labeled as constants or variables. Furthermore, the experimental findings showed that the AdamGOA_ ShCNN beat leading models in Accuracy, Convergence, and Mean Square Error (MSE), with values of 0.989, 4, and 0.539, respectively.
25
Mehmood, Faisal, Shabir Ahmad, and Taeg Keun Whangbo. "An Efficient Optimization Technique for Training Deep Neural Networks." Mathematics 11, no. 6 (2023): 1360. http://dx.doi.org/10.3390/math11061360.
Full textAbstract:
Deep learning is a sub-branch of artificial intelligence that acquires knowledge by training a neural network. It has many applications in the field of banking, automobile industry, agriculture, and healthcare industry. Deep learning has played a significant role in solving complex tasks related to computer vision, such as image classification, natural language processing, and object detection. On the other hand, optimizers also play an intrinsic role in training the deep learning model. Recent studies have proposed many deep learning models, such as VGG, ResNet, DenseNet, and ImageNet. In addition, there are many optimizers such as stochastic gradient descent (SGD), Adam, AdaDelta, Adabelief, and AdaMax. In this study, we have selected those models that require lower hardware requirements and shorter training times, which facilitates the overall training process. We have modified the Adam based optimizers and minimized the cyclic path. We have removed an additional hyper-parameter from RMSProp and observed that the optimizer works with various models. The learning rate is set to minimum and constant. The initial weights are updated after each epoch, which helps to improve the accuracy of the model. We also changed the position of the epsilon in the default Adam optimizer. By changing the position of the epsilon, it accumulates the updating process. We used various models with SGD, Adam, RMSProp, and the proposed optimization technique. The results indicate that the proposed method is effective in achieving the accuracy and works well with the state-of-the-art architectures.
26
Sun, Haijing, Hao Yu, Yichuan Shao, et al. "An Improved Adam’s Algorithm for Stomach Image Classification." Algorithms 17, no. 7 (2024): 272. http://dx.doi.org/10.3390/a17070272.
Full textAbstract:
Current stomach disease detection and diagnosis is challenged by data complexity and high dimensionality and requires effective deep learning algorithms to improve diagnostic accuracy. To address this challenge, in this paper, an improved strategy based on the Adam algorithm is proposed, which aims to alleviate the influence of local optimal solutions, overfitting, and slow convergence rates by controlling the restart strategy and the gradient norm joint clipping technique. This improved algorithm is abbreviated as the CG-Adam algorithm. The control restart strategy performs a restart operation by periodically checking the number of steps and once the number of steps reaches a preset restart period. After the restart is completed, the algorithm will restart the optimization process. It helps the algorithm avoid falling into the local optimum and maintain convergence stability. Meanwhile, gradient norm joint clipping combines both gradient clipping and norm clipping techniques, which can avoid gradient explosion and gradient vanishing problems and help accelerate the convergence of the optimization process by restricting the gradient and norm to a suitable range. In order to verify the effectiveness of the CG-Adam algorithm, experimental validation is carried out on the MNIST, CIFAR10, and Stomach datasets and compared with the Adam algorithm as well as the current popular optimization algorithms. The experimental results demonstrate that the improved algorithm proposed in this paper achieves an accuracy of 98.59%, 70.7%, and 73.2% on the MNIST, CIFAR10, and Stomach datasets, respectively, surpassing the Adam algorithm. The experimental results not only prove the significant effect of the CG-Adam algorithm in accelerating the model convergence and improving generalization performance but also demonstrate its wide potential and practical application value in the field of medical image recognition.
27
Aldo, Dasril, Adanti Wido Paramadini, and Muhammad Afrizal Amrustian. "Performance Comparison of LSTM Models with Various Optimizers and Activation Functions for Garlic Bulb Price Prediction Using Deep Learning." Jurnal Teknik Informatika (Jutif) 6, no. 2 (2025): 905–22. https://doi.org/10.52436/1.jutif.2025.6.2.4412.
Full textAbstract:
Accurate commodity price forecasting is crucial for market stability and decision-making. This study evaluates the performance of the Long Short-Term Memory (LSTM) model using various activation functions and optimization algorithms for predicting garlic bulb prices. Historical price data was collected from panelharga.badanpangan.go.id and preprocessed through normalization and dataset splitting into training, validation, and test sets. The model was trained for 200 epochs using activation functions ReLU, Sigmoid, and Tanh, combined with optimization algorithms Adam, RMSprop, SGD, Adagrad, Adadelta, Nadam, and AdamW. Experimental results indicate that ReLU + Adam achieves the best performance with Final Epoch Loss of 0.001789, RMSE of 0.701632, MAPE of 0.009593, and R² of 0.909794, followed by Sigmoid + Nadam and Tanh + Adam, which also yielded high accuracy. These findings reinforce prior research, highlighting Adam and its momentum-based variants as effective optimizers for LSTM training. This study provides insights into selecting optimal activation functions and optimizers for commodity price forecasting. Future work may explore hybrid models and external factors, such as global market trends, to enhance predictive accuracy in time series data analysis.
28
Malau, Fransiscus Rolanda. "OPTIMIZING CNN PERFORMANCE FOR AI-GENERATED IMAGE CLASSIFICATION: A COMPARATIVE STUDY OF ARCHITECTURES AND OPTIMIZERS USING K-FOLD CROSS-VALIDATION." Jurnal INSTEK (Informatika Sains dan Teknologi) 9, no. 2 (2025): 385–97. https://doi.org/10.24252/instek.v9i2.54193.
Full textAbstract:
This study investigates CNN optimization for classifying AI-generated images. Using the CIFAKE dataset (60,000 real and 60,000 AI-generated images), we evaluated four CNN configurations with varying complexity and four optimization algorithms through 5-fold cross-validation. Our findings show Configuration 4 (4 Conv, 2 MaxPool) with Adam optimizer achieved the highest validation accuracy (0.8368±0.0135). Adam demonstrated consistent performance across architectures, while SGD showed strong but variable results improving with model complexity. Adagrad and Adadelta consistently underperformed. The final model achieved 85.28% test accuracy with balanced precision (0.8531) and recall (0.8528). Results indicate more complex architectures combined with adaptive optimizers like Adam provide superior performance for AI-generated image classification, with the balance between model complexity and optimizer selection being crucial. The consistent performance across real and fake categories demonstrates this approach's robustness for deepfake detection applications.
29
Krutikov, Vladimir, Elena Tovbis, and Lev Kazakovtsev. "Adam Algorithm with Step Adaptation." Algorithms 18, no. 5 (2025): 268. https://doi.org/10.3390/a18050268.
Full textAbstract:
Adam (Adaptive Moment Estimation) is a well-known algorithm for the first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. As shown by computational experiments, with an increase in the degree of conditionality of the problem and in the presence of interference, Adam is prone to looping, which is associated with difficulties in step adjusting. In this paper, an algorithm for step adaptation for the Adam method is proposed. The principle of the step adaptation scheme used in the paper is based on reproducing the state in which the descent direction and the new gradient are found during one-dimensional descent. In the case of exact one-dimensional descent, the angle between these directions is right. In case of inexact descent, if the angle between the descent direction and the new gradient is obtuse, then the step is large and should be reduced; if the angle is acute, then the step is small and should be increased. For the experimental analysis of the new algorithm, test functions of a certain degree of conditionality with interference on the gradient and learning problems with mini-batches for calculating the gradient were used. As the computational experiment showed, in stochastic optimization problems, the proposed Adam modification with step adaptation turned out to be significantly more efficient than both the standard Adam algorithm and the other methods with step adaptation that are studied in the work.
30
Jin, Long, Han Nong, Liangming Chen, and Zhenming Su. "A Method for Enhancing Generalization of Adam by Multiple Integrations." Proceedings of the AAAI Conference on Artificial Intelligence 39, no. 4 (2025): 4147–55. https://doi.org/10.1609/aaai.v39i4.32435.
Full textAbstract:
The insufficient generalization of adaptive moment estimation (Adam) has hindered its broader application. Recent studies have shown that flat minima in loss landscapes are highly associated with improved generalization. Inspired by the filtering effect of integration operations on high-frequency signals, we propose multiple integral Adam (MIAdam), a novel optimizer that integrates a multiple integral term into Adam. This multiple integral term effectively filters out sharp minima encountered during optimization, guiding the optimizer towards flatter regions and thereby enhancing generalization capability. We provide a theoretical explanation for the improvement in generalization through the diffusion theory framework and analyze the impact of the multiple integral term on the optimizer's convergence. Experimental results demonstrate that MIAdam not only enhances generalization and robustness against label noise but also maintains the rapid convergence characteristic of Adam, outperforming Adam and its variants in state-of-the-art benchmarks.
31
Prihandoko, P., and Putrama Alkhairi. "Optimasi JST Backpropagation dengan Adaptive Learning Rate Dalam Memprediksi Hasil Panen Padi." Jurasik (Jurnal Riset Sistem Informasi dan Teknik Informatika) 10, no. 1 (2025): 441. https://doi.org/10.30645/jurasik.v10i1.887.
Full textAbstract:
Artificial Neural Networks (ANN) with the Backpropagation algorithm have been widely applied across various domains, including data prediction tasks. However, one of the primary challenges in implementing Backpropagation is the selection of an optimal learning rate. A learning rate that is too high can lead to unstable convergence, while one that is too low can significantly slow down the training process. To address this issue, this study proposes an optimization of Backpropagation using an Adaptive Learning Rate through the implementation of the Adam optimizer. The objective of this research is to analyze the performance comparison between Standard Backpropagation and Backpropagation with the Adam optimizer in predicting rice harvest yields based on rainfall, temperature, and humidity variables. The dataset consists of 100 synthetic samples generated based on a normal distribution to resemble real-world data. The results show that the use of the Adam optimizer improves the performance of the ANN model compared to the Standard Backpropagation method. Model accuracy increased from 92.04% to 92.99%, while the values of loss, Mean Squared Error (MSE), and Root Mean Squared Error (RMSE) decreased significantly, indicating that the model optimized with Adam is more stable and yields lower prediction errors. Therefore, Adaptive Learning Rate optimization using the Adam optimizer is proven to be effective in enhancing both the accuracy and efficiency of ANN in data prediction tasks.
32
Liu, Songang. "An improvement on common optimization methods based on SuperstarGAN." Applied and Computational Engineering 50, no. 1 (2024): 46–51. http://dx.doi.org/10.54254/2755-2721/50/20241169.
Full textAbstract:
Image processing has long been a focal point of research, offering avenues to enhance image clarity and transfer image features. Over the past decade, Generative Adversarial Networks (GANs) have played a pivotal role in the field of image conversion. This study delves into the world of GANs, focusing on the SuperstarGAN model and its optimization techniques. SuperstarGAN, an evolution of the well-known StarGAN, excels in multi-domain image-to-image conversion, overcoming limitations and offering versatility. To better understand its optimization, this study explored the effects of different optimizers, such as Adam, SGD, and Nadam, on SuperstarGAN's performance. Using the CelebA Face Dataset with 200 million images and 40 features, this study conducted experiments to compare these optimizers. The results revealed that while SGD and Nadam can achieve comparable results to Adam, they require more iterations and careful tuning, with SGD showing slower convergence. Nadam, with its oscillatory nature, shows promise but requires proper learning rate adjustments. This research sheds light on the critical role of optimizer choice in training SuperstarGAN. Adam emerges as the most efficient and stable option, but further exploration of Nadam's potential is warranted. This study contributes to advancing the understanding of optimization techniques for generative adversarial networks, with implications for high-quality facial image generation and beyond.
33
Septiadi, Jaka, Budi Warsito, and Adi Wibowo. "Human Activity Prediction using Long Short Term Memory." E3S Web of Conferences 202 (2020): 15008. http://dx.doi.org/10.1051/e3sconf/202020215008.
Full textAbstract:
Early symptoms of dementia is one of the causes decrease in quality of life. Human activity recognition (HAR) system is proposed to recognize the daily routines which has an important role in detecting early symptoms of dementia. Long Short Term Memory (LSTM) is very useful for sequence analysis that can find the pattern of activities that carried out in daily routines. However, the LSTM model is slow to achieving convergence and take a long time during training. In this paper, we investigated the sequence of actions recorded in smart home sensors data using LSTM model, then the model will be optimized using several optimization methods. The optimization methods were Stochastic Gradient Descent (SGD), Adagrad, Adadelta, RMSProp, and Adam. The results showed that using Adam to optimized LSTM is better than other optimization methods.
34
Kandhro, I. A., S. Z. Jumani, F. Ali, Z. U. Shaikh, M. A. Arain, and A. A. Shaikh. "Performance Analysis of Hyperparameters on a Sentiment Analysis Model." Engineering, Technology & Applied Science Research 10, no. 4 (2020): 6016–20. http://dx.doi.org/10.48084/etasr.3549.
Full textAbstract:
This paper focuses on the performance analysis of hyperparameters of the Sentiment Analysis (SA) model of a course evaluation dataset. The performance was analyzed regarding hyperparameters such as activation, optimization, and regularization. In this paper, the activation functions used were adam, adagrad, nadam, adamax, and hard_sigmoid, the optimization functions were softmax, softplus, sigmoid, and relu, and the dropout values were 0.1, 0.2, 0.3, and 0.4. The results indicate that parameters adam and softmax with dropout value 2.0 are effective when compared to other combinations of the SA model. The experimental results reveal that the proposed model outperforms the state-of-the-art deep learning classifiers.
35
Kandhro, I. A., S. Z. Jumani, F. Ali, Z. U. Shaikh, M. A. Arain, and A. A. Shaikh. "Performance Analysis of Hyperparameters on a Sentiment Analysis Model." Engineering, Technology & Applied Science Research 10, no. 4 (2020): 6016–20. https://doi.org/10.5281/zenodo.4016212.
Full textAbstract:
This paper focuses on the performance analysis of hyperparameters of the Sentiment Analysis (SA) model of a course evaluation dataset. The performance was analyzed regarding hyperparameters such as activation, optimization, and regularization. In this paper, the activation functions used were adam, adagrad, nadam, adamax, and hard_sigmoid, the optimization functions were softmax, softplus, sigmoid, and relu, and the dropout values were 0.1, 0.2, 0.3, and 0.4. The results indicate that parameters adam and softmax with dropout value 2.0 are effective when compared to other combinations of the SA model. The experimental results reveal that the proposed model outperforms the state-of-the-art deep learning classifiers.
36
Nartsev, D. Yu, and A. N. Gneushev. "Adam Optimization Method Modifications Comparison in the Regression Models Parameters Evaluation Tasks." INFORMACIONNYE TEHNOLOGII 27, no. 9 (2021): 461–69. http://dx.doi.org/10.17587/it.27.461-469.
Full textAbstract:
There is considered a problem of optimization methods comparing for the neural network regression task. Various optimization methods, such as stochastic gradient descent with momentum (SGDM), Adam and its modifications, AdamW and RAdam, were considered. To compare optimization method two regression task were formulated. Both tasks are connected with the preprocessing subtasks in the field of image analysis. The first considered task was the filtering blurred eye images on which confident recognition cannot be achieved. The training samples were generated by Gaussian blurring of the images. The blurring degree was estimated. The test and training sample for the assessment problem was formed on the basis of the BATH and CASSIA eye image databases. The second task was aligning faces in assessment image in face recognition systems. The training samples were generated by rotating face images, and rotation angle was estimated. To solve these tasks the direct estimation of parameters by solving the image regression problem by training neural network models is proposed. The adequate accuracy was acquired with all considered optimization methods for both tasks. Modifications of Adam algorithm show better results than original method. Both AdamW and RAdam methods reduced the error twice in comparison with Adam. The modification of the RAdam algorithm proposed in the work reduced the error by more than 1.5 times in comparison with the model trained by the original algorithm.
37
Cheng, Haoyuan, and Qian Ai. "A Cost Optimization Method Based on Adam Algorithm for Integrated Demand Response." Electronics 12, no. 23 (2023): 4731. http://dx.doi.org/10.3390/electronics12234731.
Full textAbstract:
Demand response has gradually evolved into integrated demand response (IDR) as energy integration technology develops in integrated energy systems (IESs). The IES has a large amount of data interaction and an increasing concern for users’ privacy protection. Based on the combined cooling, heating, and power model, our study established an IDR management model considering demand-side energy coupling, focusing on cost optimization. In terms of privacy protection in the IDR management process, an optimization method based on the Adam algorithm was proposed. Only nonsensitive data, such as gradients, were transmitted during the processing of the Adam-based method by relying on a centralized iterative computing architecture similar to federated learning. Thus, privacy protection was achieved. The final simulation results proved that the proposed IDR management model had a cost reduction of more than 9% compared with a traditional power demand response. Further simulations based on this model showed that the efficiency and accuracy of the proposed Adam-based method are better than those of other distributed computing algorithms.
38
Adigun, Taiwo, Adedeji Adegbenle, Oludele Awodele, and Chibueze Ogbonna. "Optimizing Recurrent Neural Network with Bayesian Algorithm for Behavioural Authentication System." International Journal of Engineering and Computer Science 13, no. 09 (2024): 26373–90. http://dx.doi.org/10.18535/ijecs/v13i09.4886.
Full textAbstract:
Current studies on behavioural biometrics authentication have been focused on the use of deep learning and keystroke dynamics but the aspect of conscious optimization of the algorithm in order to obtain best outcome has not been considered. This study employed and incorporated Bayesian algorithm into Recurrent Neural Network to build a Keystroke Behavioural Biometric (KBB) authentication model used against social engineering attacks. The model begins with importing the keylogging dataset for data pre-processing, feature extraction, and RNN algorithm was used to build the KBB model. Hyperparameter tuning was done to achieve optimal results. A traditional optimizer called Adaptive Momentum Estimation (Adam) was used and evaluated so as to estimate the impact of optimization in model inferencing. RNN model result with Bayesian optimization technique shows a better performance than the result of RNN model with ADAM optimization. The essence of incorporating and evaluating the best optimization technique is to come up with an effective and accurate model for behavioural biometric authentication, that could mitigate effectively against social engineering attacks.
39
Ismanto, Edi, and Noverta Effendi. "An LSTM-based prediction model for gradient-descending optimization in virtual learning environments." Computer Science and Information Technologies 4, no. 3 (2024): 199–207. http://dx.doi.org/10.11591/csit.v4i3.pp199-207.
Full textAbstract:
A virtual learning environment (VLE) is an online learning platform that allows many students, even millions, to study according to their interests without being limited by space and time. Online learning environments have many benefits, but they also have some drawbacks, such as high dropout rates, low engagement, and students' self-regulated behavior. Evaluating and analyzing the students' data generated from online learning platforms can help instructors to understand and monitor students learning progress. In this study, we suggest a predictive model for assessing student success in online learning. We investigate the effect of hyperparameters on the prediction of student learning outcomes in VLEs by the long short-term memory (LSTM) model. A hyperparameter is a parameter that has an impact on prediction results. Two optimization algorithms, adaptive moment estimation (Adam) and Nesterov-accelerated adaptive moment estimation (Nadam), were used to modify the LSTM model's hyperparameters. Based on the findings of research done on the optimization of the LSTM model using the Adam and Nadam algorithm. The average accuracy of the LSTM model using Nadam optimization is 89%, with a maximum accuracy of 93%. The LSTM model with Nadam optimisation performs better than the model with Adam optimisation when predicting students in online learning.
40
Ismanto, Edi, and Noverta Effendi. "An LSTM-based prediction model for gradient-descending optimization in virtual learning environments." Computer Science and Information Technologies 4, no. 3 (2023): 199–207. http://dx.doi.org/10.11591/csit.v4i3.p199-207.
Full textAbstract:
A virtual learning environment (VLE) is an online learning platform that allows many students, even millions, to study according to their interests without being limited by space and time. Online learning environments have many benefits, but they also have some drawbacks, such as high dropout rates, low engagement, and students' self-regulated behavior. Evaluating and analyzing the students' data generated from online learning platforms can help instructors to understand and monitor students learning progress. In this study, we suggest a predictive model for assessing student success in online learning. We investigate the effect of hyperparameters on the prediction of student learning outcomes in VLEs by the long short-term memory (LSTM) model. A hyperparameter is a parameter that has an impact on prediction results. Two optimization algorithms, adaptive moment estimation (Adam) and Nesterov-accelerated adaptive moment estimation (Nadam), were used to modify the LSTM model's hyperparameters. Based on the findings of research done on the optimization of the LSTM model using the Adam and Nadam algorithm. The average accuracy of the LSTM model using Nadam optimization is 89%, with a maximum accuracy of 93%. The LSTM model with Nadam optimisation performs better than the model with Adam optimisation when predicting students in online learning.
41
Ismanto, Edi, and Noverta Effendi. "An LSTM-based prediction model for gradient-descending optimization in virtual learning environments." Computer Science and Information Technologies 4, no. 3 (2023): 199–207. https://doi.org/10.11591/csit.v4i3.pp199-207.
Full textAbstract:
A virtual learning environment (VLE) is an online learning platform that allows many students, even millions, to study according to their interests without being limited by space and time. Online learning environments have many benefits, but they also have some drawbacks, such as high dropout rates, low engagement, and students' self-regulated behavior. Evaluating and analyzing the students' data generated from online learning platforms can help instructors to understand and monitor students learning progress. In this study, we suggest a predictive model for assessing student success in online learning. We investigate the effect of hyperparameters on the prediction of student learning outcomes in VLEs by the long short-term memory (LSTM) model. A hyperparameter is a parameter that has an impact on prediction results. Two optimization algorithms, adaptive moment estimation (Adam) and Nesterov-accelerated adaptive moment estimation (Nadam), were used to modify the LSTM model's hyperparameters. Based on the findings of research done on the optimization of the LSTM model using the Adam and Nadam algorithm. The average accuracy of the LSTM model using Nadam optimization is 89%, with a maximum accuracy of 93%. The LSTM model with Nadam optimisation performs better than the model with Adam optimisation when predicting students in online learning.
42
Zhou, Yangfan, Mingchuan Zhang, Junlong Zhu, Ruijuan Zheng, and Qingtao Wu. "A Randomized Block-Coordinate Adam online learning optimization algorithm." Neural Computing and Applications 32, no. 16 (2020): 12671–84. http://dx.doi.org/10.1007/s00521-020-04718-9.
Full text
43
Huang, Wendi. "Implementation of Parallel Optimization Algorithms for NLP: Mini-batch SGD, SGD with Momentum, AdaGrad Adam." Applied and Computational Engineering 81, no. 1 (2024): 226–33. http://dx.doi.org/10.54254/2755-2721/81/20241146.
Full textAbstract:
Abstract. With the rapid development of machine learning technology, optimization algorithms and optimizers have become key to the development of related technologies contemporarily. Models need the help of optimizers to meet other performance indicators while saving computing resources. This research focuses on comparisons between optimizers, in the context of text sentiment classification tasks. The optimizers mainly compared in this article are mini batch SGD, momentum SGD, Adagrad and Adam. Through comparative experiments, it was found that SGD and its variants have a high dependence on the initial learning rate setting, while the performance of Adagrad and Adam is relatively balanced. Although the training time of Adagrad is shorter than that of Adam, its principal formula has flaws, which are not reflected in this task. The conclusions drawn in this article through comparison can point out the advantages and disadvantages of each optimizer, and can help realize better optimizers in subsequent research.
44
Okonkwo, S. J., and Z. H. Mshelia. "Estimating Aboveground Biomass Using Allometric Models And Adaptive Learning Rate Optimization Algorithms." Journal of Applied Sciences and Environmental Management 25, no. 7 (2021): 1139–46. http://dx.doi.org/10.4314/jasem.v25i7.6.
Full textAbstract:

 
 
 Forest aboveground biomass (AGB) is imperative in the study of climate change and the carbon cycle in the global terrestrial ecosystem. Developing a credible approach to estimate forest biomass and carbon stocks is essential. Four allometric models were used with two optimization algorithms; Modified Root Mean Square Propagation (Modified RMSProp) and Modified Adaptive Moment Estimation (Modified Adam) were also used to train each model. Convergence was achieved after 1000 iterations of Modified RMSProp and 200 iterations of Modified Adam for all the models. A learning rate of 0.01 and exponential decay rates of 0.9 and 0.999 for the first and second momentum. A loss function of 0.5 Mean Square Error (0.5 MSE) was used and Root Mean Square Error (RMSE) was used to judge the accuracy of the models. The study showed that the optimization algorithms were both able to accurately optimize three of the four allometric models. While Modified Adam was the more efficient optimizer, it had the highest RMSE value 2.3910 and Modified RMSProp had the least RMSE value 0.37381. However, there was no statistically significant difference between the accuracy of the models optimized by both algorithms.
 
 
45
Albayati, Abdulhakeem Qusay, Sarmad A. Jameel Altaie, Wasseem N. Ibrahem Al-Obaydy, and Farah Alkhalid. "Performance analysis of optimization algorithms for convolutional neural network-based handwritten digit recognition." IAES International Journal of Artificial Intelligence (IJ-AI) 13, no. 1 (2024): 563. http://dx.doi.org/10.11591/ijai.v13.i1.pp563-571.
Full textAbstract:
Handwritten digit recognition has been widely researched by the recognition society during the last decades. Deep convolutional neural networks (CNN) have been exploited to propose efficient handwritten digit recognition approaches. However, the CNN model may need an optimization algorithm to achieve satisfactory performance. In this work, a performance evaluation of seven optimization methods applied in a straightforward CNN architecture is presented. The inspected algorithms are stochastic gradient descent (SGD), adaptive gradient (AdaGrad), adaptive delta (AdaDelta), adaptive moment estimation (ADAM), maximum adaptive moment estimation (AdaMax), nesterov-accelerated adaptive moment estimation (Nadam), and root mean square propagation (RMSprop). Experimental results carried out on MNIST and EMNIST digit datasets have shown the superior performance of RMSprop and Adam algorithms over the peer methods, respectively.
46
Subagyo, Ageng Ramdhan, and Theopilus Bayu Sasongko. "Implementasi Algoritma Transformers BART dan Penggunaan Metode Optimasi Adam Untuk Klasifikasi Judul Berita Palsu." JURNAL MEDIA INFORMATIKA BUDIDARMA 8, no. 3 (2024): 1768. http://dx.doi.org/10.30865/mib.v8i3.7852.
Full textAbstract:
Classification is a process of identifying new data provided based on validation of previous data. One classification process that can be used is fake news classification. The classification process requires as little time as possible to get maximum results, so a faster method is needed to classify news. The BART algorithm can be a method that can be used to carry out classification and use Adam optimization to improve the performance of the algorithm. The aim of this research is to classify fake news, whether the BART algorithm and Adam optimization are able to provide good results and to label whether the news is fake or not. The results of this process are based on the use of a dataset of 65% for training, 30% for validation, and 5% to produce 2 BART models. With the additional use of Adam optimization and several other parameters for the training process, the first model was able to provide accuracy performance of 92.88%, training loss reached 12.2%, and validation loss reached 28.4% and the second model produced an accuracy of 92.63 %, training loss 15% and validation loss reaching 20.2%. In the first model, it can predict 105 data labeled negative and 1306 positive data. Meanwhile, the second model was able to predict 128 data labeled negative and 1283 positive data.
47
Raharjo, Mokhamad Ramdhani, Indra Riyana Rahadjeng, Muhammad Noor Hasan Siregar, and Putrama Alkhairi. "Optimasi Fungsi Aktivasi pada Artificial Neural Network untuk Prediksi Gagal Jantung Secara Akurat." Explorer 5, no. 1 (2025): 42–51. https://doi.org/10.47065/explorer.v5i1.1840.
Full textAbstract:
Heart failure is one of the major health problems that can be fatal if not diagnosed properly and quickly. Therefore, early prediction using artificial intelligence models, especially Artificial Neural Network (ANN), is needed to improve the accuracy in detecting heart failure. This study aims to optimize the activation function in ANN to predict heart failure accurately. Several optimization algorithms tested, namely Adam, RMSprop, SGD, Adagrad, and Adadelta, were used to evaluate model performance in terms of accuracy, precision, recall, and F1-score. The results showed that the Adam optimization algorithm provided the best performance with an accuracy of 86.74%, precision of 75.12%, recall of 66.67%, and F1-score of 70.64%. Meanwhile, other algorithms such as RMSprop, SGD, Adagrad, and Adadelta showed lower performance, with some metrics reaching 0%. This study shows that proper activation function optimization in ANN is very important to improve the model's ability to predict heart failure with a high level of accuracy.
48
Amryliana, Septia, Syamsul Bahri, and Muhammad Rijal Alfian. "Optimization of Long Short Term Memory Model for Gold Price Prediction Using Adaptive Moment Estimation." Jurnal Matematika, Statistika dan Komputasi 21, no. 3 (2025): 669–83. https://doi.org/10.20956/j.v21i3.42872.
Full textAbstract:
The era of globalization and rapidly evolving economic dynamics place the financial sector at the center of attention for market participants and investors. Financial instruments such as gold play a crucial role as hedging tools and portfolio diversification, yet face significant challenges due to complex and unpredictable price fluctuations. Artificial intelligence technology, particularly Long Short Term Memory (LSTM) models and Adaptive Moment Estimation (ADAM), offers relevant solutions for predicting financial asset prices with strong temporal fluctuations, such as gold prices. This research aims to optimize the LSTM model using the ADAM technique to enhance the accuracy of gold price predictions. The research findings indicate that the LSTM model optimized with ADAM can provide highly accurate gold price predictions with low error rates. The LSTM model used has 3 layers with 128, 64, and 32 units, and uses 100 epochs in the model training process. At the 100th epoch, the final loss obtained was 0,000336. Model evaluation results showed a MAPE of around 0,0108 or 1,08% an accuracy rate of about 98,92%, and a low loss value of 0,00025.
49
Pooja, Choudhary, and Garg Kanwal. "Tensor Data Imputation by PARAFAC with Updated Chaotic Biases by Adam Optimizer." International Journal of Recent Technology and Engineering (IJRTE) 9, no. 6 (2021): 30–38. https://doi.org/10.35940/ijrte.E5291.039621.
Full textAbstract:
<strong>ABSTRACT:</strong> The big data pattern analysis suffers from incorrect responses due to missing data entries in the real world. Data collected for digital movie platforms like Netflix and intelligent transportation systems is Spatio-temporal data. Extracting the latent and explicit features from this data is a challenge. We present the high dimensional data imputation problem as a higher-order tensor decomposition. The regularized and biased PARAFAC decomposition is proposed to generate the missing data entries. The biases are created and updated by a chaotic exponential factor in Adam's optimization, which reduces the imputation error. This chaotic perturbed exponentially update in the learning rate replaces the fixed learning rate in the bias update by Adam optimization. The idea has experimented with Netflix and traffic datasets from Guangzhou, China
50
Yang, Luyi. "Theoretical Analysis of Adam Optimizer in the Presence of Gradient Skewness." International Journal of Applied Science 7, no. 2 (2024): p27. http://dx.doi.org/10.30560/ijas.v7n2p27.
Full textAbstract:
The Adam optimizer has become a cornerstone in deep learning, widely adopted for its adaptive learning rates and momentumbased updates. However, its behavior under non-standard conditions, particularly skewed gradient distributions, remains underexplored. This paper presents a novel theoretical analysis of the Adam optimizer in the presence of skewed gradients, a scenario frequently encountered in real-world applications due to imbalanced datasets or inherent problem characteristics. We extend the standard convergence analysis of Adam to explicitly account for gradient skewness, deriving new bounds that characterize the optimizer’s performance under these conditions. Our main contributions include: (1) a formal proof of Adam’s convergence under skewed gradient distributions, (2) quantitative error bounds that capture the impact of skewness on optimization outcomes, and (3) insights into how skewness affects Adam’s adaptive learning rate mechanism. We demonstrate that gradient skewness can lead to biased parameter updates and potentially slower convergence compared to scenarios with symmetric distributions. Additionally, we provide practical recommendations for mitigating these effects, including adaptive gradient clipping and distribution-aware hyperparameter tuning. Our findings bridge a critical gap between Adam’s empirical success and its theoretical underpinnings, offering valuable insights for practitioners dealing with non-standard optimization landscapes in deep learning.