Log in

Relevant bibliographies by topics / Rectified Linear Unit (ReLU) / Journal articles

To see the other types of publications on this topic, follow the link: Rectified Linear Unit (ReLU).

Journal articles on the topic 'Rectified Linear Unit (ReLU)'

Author: Grafiati

Published: 5 June 2025

Last updated: 1 August 2025

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles

Select a source type:

Consult the top 50 journal articles for your research on the topic 'Rectified Linear Unit (ReLU).'

Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.

You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.

Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.

1

Abdeljawad, Ahmed, and Philipp Grohs. "Approximations with deep neural networks in Sobolev time-space." Analysis and Applications 20, no. 03 (2022): 499–541. http://dx.doi.org/10.1142/s0219530522500014.

Full text

Abstract:

Solutions of the evolution equation generally lie in certain Bochner–Sobolev spaces, in which the solutions may have regularity and integrability properties for the time variable that can be different for the space variables. Therefore, in this paper, we develop a framework that shows that deep neural networks can approximate Sobolev-regular functions with respect to Bochner–Sobolev spaces. In our work, we use the so-called Rectified Cubic Unit (ReCU) as an activation function in our networks. This activation function allows us to deduce approximation results of the neural networks while avoid

APA, Harvard, Vancouver, ISO, and other styles

2

Bai, Yuhan. "RELU-Function and Derived Function Review." SHS Web of Conferences 144 (2022): 02006. http://dx.doi.org/10.1051/shsconf/202214402006.

Full text

Abstract:

The activation function plays an important role in training and improving performance in deep neural networks (dnn). The rectified linear unit (relu) function provides the necessary non-linear properties in the deep neural network (dnn). However, few papers sort out and compare various relu activation functions. Most of the paper focuses on the efficiency and accuracy of certain activation functions used by the model, but does not pay attention to the nature and differences of these activation functions. Therefore, this paper attempts to organize the RELU-function and derived function in this

APA, Harvard, Vancouver, ISO, and other styles

3

Garg, Shruti, Soumyajit Behera, K. Rahul Patro, and Ashwani Garg. "Deep Neural Network for Electroencephalogram based Emotion Recognition." IOP Conference Series: Materials Science and Engineering 1187, no. 1 (2021): 012012. http://dx.doi.org/10.1088/1757-899x/1187/1/012012.

Full text

Abstract:

Abstract Emotion recognition using electroencephalogram (EEG) signals is an aspect of affective computing. The EEG refers to recording brain responses via electrical signals by showing external stimuli to the participants. This paper proposes the prediction of valence, arousal, dominance and liking for EEG signals using a deep neural network (DNN). The EEG data is obtained from the AMIGOS dataset, a publicly available dataset for mood and personality research. Two features, normalized and power and normalized wavelet energy, are extracted using Fourier and wavelet transform, respectively. A DN

APA, Harvard, Vancouver, ISO, and other styles

4

Katende, Ronald, Henry Kasumba, Godwin Kakuba, and John M. Mango. "A proof of convergence and equivalence for 1D finite element methods and ReLU neural networks." Annals of Mathematics and Computer Science 25 (November 16, 2024): 97–111. http://dx.doi.org/10.56947/amcs.v25.392.

Full text

Abstract:

This paper investigates the convergence and equivalence properties of the Finite Element Method (FEM) and Rectified Linear Unit Neural Networks (ReLU NNs) in solving differential equations. We provide a detailed comparison of the two approaches, highlighting their mutual capabilities in function space approximation. Our analysis proves the subset and superset inclusions that establish the equivalence between FEM and ReLU NNs for approximating piecewise linear functions. Furthermore, a comprehensive numerical evaluation is presented, demonstrating the error convergence behavior of ReLU NNs as t

APA, Harvard, Vancouver, ISO, and other styles

5

McCarty, Sarah. "Piecewise linear functions representable with infinite width shallow ReLU neural networks." Proceedings of the American Mathematical Society, Series B 10, no. 27 (2023): 296–310. http://dx.doi.org/10.1090/bproc/186.

Full text

Abstract:

This paper analyzes representations of continuous piecewise linear functions with infinite width, finite cost shallow neural networks using the rectified linear unit (ReLU) as an activation function. Through its integral representation, a shallow neural network can be identified by the corresponding signed, finite measure on an appropriate parameter space. We map these measures on the parameter space to measures on the projective n n -sphere cross R \mathbb {R} , allowing points in the parameter space to be bijectively mapped to hyperplanes in the domain of the function. We prove a conjecture

APA, Harvard, Vancouver, ISO, and other styles

6

Tedyyana, Agus, Osman Ghazali, Onno W. Purbo, and Mohamad Amir Abu Seman. "Enhancing intrusion detection system using rectified linear unit function in pigeon inspired optimization algorithm." IAES International Journal of Artificial Intelligence (IJ-AI) 13, no. 2 (2024): 1526. http://dx.doi.org/10.11591/ijai.v13.i2.pp1526-1534.

Full text

Abstract:

The increasing rate of cybercrime in the digital world highlights the importance of having a reliable intrusion detection system (IDS) to detect unauthorized attacks and notify administrators. IDS can leverage machine learning techniques to identify patterns of attacks and provide real-time notifications. In building a successful IDS, selecting the right features is crucial as it determines the accuracy of the predictions made by the model. This paper presents a new IDS algorithm that combines the rectified linear unit (ReLU) activation function with a pigeon-inspired optimizer in feature sele

APA, Harvard, Vancouver, ISO, and other styles

7

Agus, Tedyyana, Ghazali Osman, and W. Purbo Onno. "Enhancing intrusion detection system using rectified linear unit function in pigeon inspired optimization algorithm." IAES International Journal of Artificial Intelligence (IJ-AI) 13, no. 2 (2024): 1526–34. https://doi.org/10.11591/ijai.v13.i2.pp1526-1534.

Full text

Abstract:

The increasing rate of cybercrime in the digital world highlights the importance of having a reliable intrusion detection system (IDS) to detect unauthorized attacks and notify administrators. IDS can leverage machine learning techniques to identify patterns of attacks and provide real-time notifications. In building a successful IDS, selecting the right features is crucial as it determines the accuracy of the predictions made by the model. This paper presents a new IDS algorithm that combines the rectified linear unit (ReLU) activation function with a pigeon-inspired optimizer in feature sele

APA, Harvard, Vancouver, ISO, and other styles

8

Gühring, Ingo, Gitta Kutyniok, and Philipp Petersen. "Error bounds for approximations with deep ReLU neural networks in Ws,p norms." Analysis and Applications 18, no. 05 (2019): 803–59. http://dx.doi.org/10.1142/s0219530519410021.

Full text

Abstract:

We analyze to what extent deep Rectified Linear Unit (ReLU) neural networks can efficiently approximate Sobolev regular functions if the approximation error is measured with respect to weaker Sobolev norms. In this context, we first establish upper approximation bounds by ReLU neural networks for Sobolev regular functions by explicitly constructing the approximate ReLU neural networks. Then, we establish lower approximation bounds for the same type of function classes. A trade-off between the regularity used in the approximation norm and the complexity of the neural network can be observed in

APA, Harvard, Vancouver, ISO, and other styles

9

Almatroud, Othman Abdullah, Viet-Thanh Pham, and Karthikeyan Rajagopal. "A Rectified Linear Unit-Based Memristor-Enhanced Morris–Lecar Neuron Model." Mathematics 12, no. 19 (2024): 2970. http://dx.doi.org/10.3390/math12192970.

Full text

Abstract:

This paper introduces a modified Morris–Lecar neuron model that incorporates a memristor with a ReLU-based activation function. The impact of the memristor on the dynamics of the ML neuron model is analyzed using bifurcation diagrams and Lyapunov exponents. The findings reveal chaotic behavior within specific parameter ranges, while increased magnetic strength tends to maintain periodic dynamics. The emergence of various firing patterns, including periodic and chaotic spiking as well as square-wave and triangle-wave bursting is also evident. The modified model also demonstrates multistability

APA, Harvard, Vancouver, ISO, and other styles

10

Noprisson, Handrie, Vina Ayumi, Mariana Purba, and Nur Ani. "MOBILENET PERFORMANCE IMPROVEMENTS FOR DEEPFAKE IMAGE IDENTIFICATION USING ACTIVATION FUNCTION AND REGULARIZATION." JITK (Jurnal Ilmu Pengetahuan dan Teknologi Komputer) 10, no. 2 (2024): 441–48. http://dx.doi.org/10.33480/jitk.v10i2.5798.

Full text

Abstract:

Deepfake images are often used to spread false information, manipulate public opinion, and harm individuals by creating fake content, making developing deepfake detection technology essential to mitigate these potential dangers. This study utilized the MobileNet architecture by applying regularization and activation function methods to improve detection accuracy. ReLU (Rectified Linear Unit) enhances the model's efficiency and ability to capture non-linear features, while Dropout and L2 regularization help reduce overfitting by penalizing large weights, thereby improving generalization. Based

APA, Harvard, Vancouver, ISO, and other styles

11

Phua, Yeong Tsann, Sujata Navaratnam, Chon-Moy Kang, and Wai-Seong Che. "Sequence-to-sequence neural machine translation for English-Malay." IAES International Journal of Artificial Intelligence (IJ-AI) 11, no. 2 (2022): 658. http://dx.doi.org/10.11591/ijai.v11.i2.pp658-665.

Full text

Abstract:

Machine translation aims to translate text from a specific language into another language using computer software. In this work, we performed neural machine translation with attention implementation on English-Malay parallel corpus. We attempt to improve the model performance by rectified linear unit (ReLU) attention alignment. Different sequence-to-sequence models were trained. These models include long-short term memory (LSTM), gated recurrent unit (GRU), bidirectional LSTM (Bi-LSTM) and bidirectional GRU (Bi-GRU). In the experiment, both bidirectional models, Bi-LSTM and Bi-GRU yield a conv

APA, Harvard, Vancouver, ISO, and other styles

12

Yeong-Tsann, Phua, Navaratnam Sujata, Kang Chon-Moy, and Chew Wai-Seong. "Sequence-to-sequence neural machine translation for English-Malay." International Journal of Artificial Intelligence (IJ-AI) 11, no. 2 (2022): 658–65. https://doi.org/10.11591/ijai.v11.i2.pp658-665.

Full text

Abstract:

Machine translation aims to translate text from a specific language into another language using computer software. In this work, we performed neural machine translation with attention implementation on English-Malay parallel corpus. We attempt to improve the model performance by rectified linear unit (ReLU) attention alignment. Different sequence-to-sequence models were trained. These models include long-short term memory (LSTM), gated recurrent unit (GRU), bidirectional LSTM (Bi-LSTM) and bidirectional GRU (Bi-GRU). In the experiment, both bidirectional models, Bi-LSTM and Bi-GRU yield a conv

APA, Harvard, Vancouver, ISO, and other styles

13

Purnawansyah, Purnawansyah, Haviluddin Haviluddin, Herdianti Darwis, Huzain Azis, and Yulita Salim. "Backpropagation Neural Network with Combination of Activation Functions for Inbound Traffic Prediction." Knowledge Engineering and Data Science 4, no. 1 (2021): 14. http://dx.doi.org/10.17977/um018v4i12021p14-28.

Full text

Abstract:

Predicting network traffic is crucial for preventing congestion and gaining superior quality of network services. This research aims to use backpropagation to predict the inbound level to understand and determine internet usage. The architecture consists of one input layer, two hidden layers, and one output layer. The study compares three activation functions: sigmoid, rectified linear unit (ReLU), and hyperbolic Tangent (tanh). Three learning rates: 0.1, 0.5, and 0.9 represent low, moderate, and high rates, respectively. Based on the result, in terms of a single form of activation function, a

APA, Harvard, Vancouver, ISO, and other styles

14

Razali, Noor Fadzilah, Iza Sazanita Isa, Siti Noraini Sulaiman, Muhammad Khusairi Osman, Noor Khairiah A. Karim, and Dayang Suhaida Awang Damit. "Genetic algorithm-adapted activation function optimization of deep learning framework for breast mass cancer classification in mammogram images." International Journal of Electrical and Computer Engineering (IJECE) 15, no. 3 (2025): 2820. https://doi.org/10.11591/ijece.v15i3.pp2820-2833.

Full text

Abstract:

The convolutional neural network (CNN) has been explored for mammogram cancer classification to aid radiologists. CNNs require multiple convolution and non-linearity repetitions to learn data sparsity, but deeper networks often face the vanishing gradient effect, which hinders effective learning. The rectified linear unit (ReLU) activation function activates neurons only when the output exceeds zero, limiting activation and potentially lowering performance. This study proposes an adaptive ReLU based on a genetic algorithm (GA) to determine the optimal threshold for neuron activation, thus impr

APA, Harvard, Vancouver, ISO, and other styles

15

Gu, Jinlong, Zhiyi Li, Lijuan Zhang, et al. "Research on the Quality Grading Method of Ginseng with Improved DenseNet121 Model." Electronics 13, no. 22 (2024): 4504. http://dx.doi.org/10.3390/electronics13224504.

Full text

Abstract:

Ginseng is an important medicinal plant widely used in traditional Chinese medicine. Traditional methods for evaluating the visual quality of ginseng have limitations. This study presents a new method for grading ginseng’s appearance quality using an improved DenseNet121 model. We enhance the network’s capability to recognize various channel features by integrating a CA (Coordinate Attention) mechanism. We also use grouped convolution instead of standard convolution in dense layers to lower the number of model parameters and improve efficiency. Additionally, we substitute the ReLU (Rectified L

APA, Harvard, Vancouver, ISO, and other styles

16

Naufal, Budiman, Adi Kusworo, and Wibowo Adi. "Impact of Activation Function on the Performance of Convolutional Neural Network in Identifying Oil Palm Fruit Ripeness." International Journal of Mathematics and Computer Research 13, no. 04 (2025): 5107–13. https://doi.org/10.5281/zenodo.15261476.

Full text

Abstract:

Activation functions play a crucial role in Convolutional Neural Networks (CNN), particularly in enabling the model to recognize and represent complex patterns in digital images. In image classification tasks, the choice of activation function can significantly impact the accuracy and overall performance of the model. The Rectified Linear Unit (ReLU) is the most commonly used activation function due to its simplicity; however, it has a limitation in discarding information from negative input values. To address this issue, alternative functions such as Leaky ReLU and Gaussian Error Linear Unit

APA, Harvard, Vancouver, ISO, and other styles

17

Pattanaik, Abhipsa, and Leena Das. "DeepSkinNet: A Deep Learning Induced Skin Lesion Extraction System from Dermoscopic Images." International Journal of Online and Biomedical Engineering (iJOE) 21, no. 07 (2025): 15–28. https://doi.org/10.3991/ijoe.v21i07.54621.

Full text

Abstract:

In this work, a DeepSkinNet model was developed based on an encoder-decoder type framework. The designed encoder incorporates three blocks where each block sandwiches convolution, rectified linear unit (ReLU), and maxpooling layers to retain the prominent details. The developed DIL (dilated convolution + instance normalization + Leaky ReLU) module comprises three branches, where each branch consists of an atrous convolution layer with a sampling rate of two, followed by instance normalization and a Leaky ReLU activation function to retain the subtle details accurately. Further, the proposed de

APA, Harvard, Vancouver, ISO, and other styles

18

Anjum, Rizwan, Mujahid Abbas, Hira Safdar, Muhammad Din, Mi Zhou, and Stojan Radenović. "Application to Activation Functions through Fixed-Circle Problems with Symmetric Contractions." Symmetry 16, no. 1 (2024): 69. http://dx.doi.org/10.3390/sym16010069.

Full text

Abstract:

In this paper, our main aim is to present innovative fixed-point theorems that provide solutions to the fixed-circle problem with symmetric contractions. We accomplish this by employing operator enrichment techniques within the context of Banach spaces. Furthermore, we demonstrate the practical application of these theorems by showcasing their relevance to the rectified linear unit (ReLU) activation function. By exploring the connection between fixed points and activation functions, our work contributes to a deeper understanding of the behavior and properties of these fundamental mathematical

APA, Harvard, Vancouver, ISO, and other styles

19

Zheng, Jing, Shuaishuai Shen, Tianqi Jiang, and Weiqiang Zhu. "Deep neural networks design and analysis for automatic phase pickers from three-component microseismic recordings." Geophysical Journal International 220, no. 1 (2019): 323–34. http://dx.doi.org/10.1093/gji/ggz441.

Full text

Abstract:

SUMMARY It is essential to pick P-wave and S-wave arrival times rapidly and accurately for the microseismic monitoring systems. Meanwhile, it is not easy to identify the arrivals at a true phase automatically using traditional picking method. This is one of the reasons that many researchers are trying to introduce deep neural networks to solve these problems. Convolutional neural networks (CNNs) are very attractive for designing automatic phase pickers especially after introducing the fundamental network structure from semantic segmentation field, which can give the probability outputs for eve

APA, Harvard, Vancouver, ISO, and other styles

20

Kulathunga, Nalinda, Nishath Rajiv Ranasinghe, Daniel Vrinceanu, Zackary Kinsman, Lei Huang, and Yunjiao Wang. "Effects of Nonlinearity and Network Architecture on the Performance of Supervised Neural Networks." Algorithms 14, no. 2 (2021): 51. http://dx.doi.org/10.3390/a14020051.

Full text

Abstract:

The nonlinearity of activation functions used in deep learning models is crucial for the success of predictive models. Several simple nonlinear functions, including Rectified Linear Unit (ReLU) and Leaky-ReLU (L-ReLU) are commonly used in neural networks to impose the nonlinearity. In practice, these functions remarkably enhance the model accuracy. However, there is limited insight into the effects of nonlinearity in neural networks on their performance. Here, we investigate the performance of neural network models as a function of nonlinearity using ReLU and L-ReLU activation functions in the

APA, Harvard, Vancouver, ISO, and other styles

21

Aloufi, Nasser, Abdulaziz Alnori, Vijey Thayananthan, and Abdullah Basuhail. "Object Detection Performance Evaluation for Autonomous Vehicles in Sandy Weather Environments." Applied Sciences 13, no. 18 (2023): 10249. http://dx.doi.org/10.3390/app131810249.

Full text

Abstract:

In order to reach the highest level of automation, autonomous vehicles (AVs) are required to be aware of surrounding objects and detect them even in adverse weather. Detecting objects is very challenging in sandy weather due to characteristics of the environment, such as low visibility, occlusion, and changes in lighting. In this paper, we considered the You Only Look Once (YOLO) version 5 and version 7 architectures to evaluate the performance of different activation functions in sandy weather. In our experiments, we targeted three activation functions: Sigmoid Linear Unit (SiLU), Rectified L

APA, Harvard, Vancouver, ISO, and other styles

22

Madhu, Golla, Sandeep Kautish, Khalid Abdulaziz Alnowibet, Hossam M. Zawbaa, and Ali Wagdy Mohamed. "NIPUNA: A Novel Optimizer Activation Function for Deep Neural Networks." Axioms 12, no. 3 (2023): 246. http://dx.doi.org/10.3390/axioms12030246.

Full text

Abstract:

In recent years, various deep neural networks with different learning paradigms have been widely employed in various applications, including medical diagnosis, image analysis, self-driving vehicles and others. The activation functions employed in deep neural networks have a huge impact on the training model and the reliability of the model. The Rectified Linear Unit (ReLU) has recently emerged as the most popular and extensively utilized activation function. ReLU has some flaws, such as the fact that it is only active when the units are positive during back-propagation and zero otherwise. This

APA, Harvard, Vancouver, ISO, and other styles

23

Suresh, Bharathwaj, Kamlesh Pillai, Gurpreet Singh Kalsi, Avishaii Abuhatzera, and Sreenivas Subramoney. "Early Prediction of DNN Activation Using Hierarchical Computations." Mathematics 9, no. 23 (2021): 3130. http://dx.doi.org/10.3390/math9233130.

Full text

Abstract:

Deep Neural Networks (DNNs) have set state-of-the-art performance numbers in diverse fields of electronics (computer vision, voice recognition), biology, bioinformatics, etc. However, the process of learning (training) from the data and application of the learnt information (inference) process requires huge computational resources. Approximate computing is a common method to reduce computation cost, but it introduces loss in task accuracy, which limits their application. Using an inherent property of Rectified Linear Unit (ReLU), a popular activation function, we propose a mathematical model t

APA, Harvard, Vancouver, ISO, and other styles

24

Favorskaya, M. N., and V. V. Andreev. "THE STUDY OF ACTIVATION FUNCTIONS IN DEEP LEARNING FOR PEDESTRIAN DETECTION AND TRACKING." ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLII-2/W12 (May 9, 2019): 53–59. http://dx.doi.org/10.5194/isprs-archives-xlii-2-w12-53-2019.

Full text

Abstract:

<p><strong>Abstract.</strong> Pedestrian detection and tracking remains a highlight research topic due to its paramount importance in the fields of video surveillance, human-machine interaction, and tracking analysis. At present time, pedestrian detection is still an open problem because of many challenges of image representation in the outdoor and indoor scenes. In recent years, deep learning, in particular Convolutional Neural Networks (CNNs) became the state-of-the-art in terms of accuracy in many computer vision tasks. The unsupervised learning of CNNs is still an open is

APA, Harvard, Vancouver, ISO, and other styles

25

Moon, Sunghwan. "ReLU Network with Bounded Width Is a Universal Approximator in View of an Approximate Identity." Applied Sciences 11, no. 1 (2021): 427. http://dx.doi.org/10.3390/app11010427.

Full text

Abstract:

Deep neural networks have shown very successful performance in a wide range of tasks, but a theory of why they work so well is in the early stage. Recently, the expressive power of neural networks, important for understanding deep learning, has received considerable attention. Classic results, provided by Cybenko, Barron, etc., state that a network with a single hidden layer and suitable activation functions is a universal approximator. A few years ago, one started to study how width affects the expressiveness of neural networks, i.e., a universal approximation theorem for a deep neural networ

APA, Harvard, Vancouver, ISO, and other styles

26

Sana, Danish, Ul Rahman Jamshaid, and Haider Gulfam. "Performance analysis of convolutional neural networks for image classification with appropriate optimizers." i-manager’s Journal on Mathematics 12, no. 1 (2023): 1. http://dx.doi.org/10.26634/jmat.12.1.19398.

Full text

Abstract:

Optimizers in Convolutional Neural Networks play an important role in many advanced deep learning models. Studies on advanced optimizers and modifications of existing optimizers continue to hold significant importance in the study of machine tools and algorithms. There are a number of studies to defend and the selection of these optimizers illustrate some of the challenges on the effectiveness of these optimizers. Comprehensive analysis on the optimizers and alteration with famous activation function Rectified Linear Unit (ReLU) offered to protect effectiveness. Significance is determined base

APA, Harvard, Vancouver, ISO, and other styles

27

BENCHAMA, Asmaa, and Khalid ZEBBARA. "Optimizing CNN-BiGRU Performance: Mish Activation and Comparative Analysis." International journal of Computer Networks & Communications 16, no. 3 (2024): 69–87. http://dx.doi.org/10.5121/ijcnc.2024.16305.

Full text

Abstract:

Deep learning is currently extensively employed across a range of research domains. The continuous advancements in deep learning techniques contribute to solving intricate challenges. Activation functions (AF) are fundamental components within neural networks, enabling them to capture complex patterns and relationships in the data. By introducing non-linearities, AF empowers neural networks to model and adapt to the diverse and nuanced nature of real-world data, enhancing their ability to make accurate predictions across various tasks. In the context of intrusion detection, the Mish, a recent

APA, Harvard, Vancouver, ISO, and other styles

28

Yan, Zhiqi, Shisheng Zhong, Lin Lin, and Zhiquan Cui. "Adaptive Levenberg–Marquardt Algorithm: A New Optimization Strategy for Levenberg–Marquardt Neural Networks." Mathematics 9, no. 17 (2021): 2176. http://dx.doi.org/10.3390/math9172176.

Full text

Abstract:

Engineering data are often highly nonlinear and contain high-frequency noise, so the Levenberg–Marquardt (LM) algorithm may not converge when a neural network optimized by the algorithm is trained with engineering data. In this work, we analyzed the reasons for the LM neural network’s poor convergence commonly associated with the LM algorithm. Specifically, the effects of different activation functions such as Sigmoid, Tanh, Rectified Linear Unit (RELU) and Parametric Rectified Linear Unit (PRLU) were evaluated on the general performance of LM neural networks, and special values of LM neural n

APA, Harvard, Vancouver, ISO, and other styles

29

Benatti, F., S. Mancini, and S. Mangini. "Continuous variable quantum perceptron." International Journal of Quantum Information 17, no. 08 (2019): 1941009. http://dx.doi.org/10.1142/s0219749919410090.

Full text

Abstract:

We present a model of Continuous Variable Quantum Perceptron (CVQP), also referred to as neuron in the following, whose architecture implements a classical perceptron. The necessary nonlinearity is obtained via measuring the output qubit and using the measurement outcome as input to an activation function. The latter is chosen to be the so-called Rectified linear unit (ReLu) activation function by virtue of its practical feasibility and the advantages it provides in learning tasks. The encoding of classical data into realistic finitely squeezed states and the use of superposed (entangled) inpu

APA, Harvard, Vancouver, ISO, and other styles

30

Daróczy, Bálint. "Gaussian Perturbations in ReLU Networks and the Arrangement of Activation Regions." Mathematics 10, no. 7 (2022): 1123. http://dx.doi.org/10.3390/math10071123.

Full text

Abstract:

Recent articles indicate that deep neural networks are efficient models for various learning problems. However, they are often highly sensitive to various changes that cannot be detected by an independent observer. As our understanding of deep neural networks with traditional generalisation bounds still remains incomplete, there are several measures which capture the behaviour of the model in case of small changes at a specific state. In this paper we consider Gaussian perturbations in the tangent space and suggest tangent sensitivity in order to characterise the stability of gradient updates.

APA, Harvard, Vancouver, ISO, and other styles

31

Chieng, Hock Hung, Noorhaniza Wahid, Ong Pauline, and Sai Raj Kishore Perla. "Flatten-T Swish: a thresholded ReLU-Swish-like activation function for deep learning." International Journal of Advances in Intelligent Informatics 4, no. 2 (2018): 76. http://dx.doi.org/10.26555/ijain.v4i2.249.

Full text

Abstract:

Activation functions are essential for deep learning methods to learn and perform complex tasks such as image classification. Rectified Linear Unit (ReLU) has been widely used and become the default activation function across the deep learning community since 2012. Although ReLU has been popular, however, the hard zero property of the ReLU has heavily hindering the negative values from propagating through the network. Consequently, the deep neural network has not been benefited from the negative representations. In this work, an activation function called Flatten-T Swish (FTS) that leverage th

APA, Harvard, Vancouver, ISO, and other styles

32

Ghanimi, Hayder M. A., T. Gopalakrishnan, G. Joel Sunny Deol, K. Amarendra, Pankaj Dadheech, and Sudhakar Sengan. "Chebyshev polynomial approximation in CNN for zero-knowledge encrypted data analysis." Journal of Discrete Mathematical Sciences and Cryptography 27, no. 2 (2024): 203–14. http://dx.doi.org/10.47974/jdmsc-1880.

Full text

Abstract:

Integrating Deep Learning (DL) techniques in Convolutional Neural Networks (CNNs) with encrypted data analysis is an emerging field for enhancing data privacy and security. A significant challenge in this domain is the incompatibility of standard non-linear Activation Functions (AF) like Rectified Linear Unit (ReLU) and Hyperbolic Tangent (tanh) with Zero-Knowledge (ZK) encrypted data, which impacts computational efficiency and data privacy. Addressing this, our paper introduces the novel application of Chebyshev Polynomial Approximation (CPA) to adapt these AF to process encrypted data effect

APA, Harvard, Vancouver, ISO, and other styles

33

Guo, Aihua. "ReLU-FCM trained by quasi-oppositional bare bone imperialist competition algorithm for predicting employment rate." PLOS ONE 17, no. 8 (2022): e0272624. http://dx.doi.org/10.1371/journal.pone.0272624.

Full text

Abstract:

Fuzzy cognitive maps (FCMs) are a powerful tool for system modeling, which can be used for static and dynamic analysis. However, traditional FCMs are usually learned by gradient-based methods, and the adopted sigmoid nonlinear activation function frequently causes gradient saturation. These two shortcomings set a limit on the modeling accuracy. To overcome those problems, we propose in this paper a new FCM with two improvements. First, the rectified linear unit (ReLu) activation function is adopted to replace the sigmoid function. Second, a newly proposed quasi-oppositional bare bone imperiali

APA, Harvard, Vancouver, ISO, and other styles

34

Cordova, Ronald S., Rolou Lyn R. Maata, Malik Jawarneh, Marwan I. Alshar'e, and Oliver C. Agustin. "Accurate prediction of chronic diseases using deep learning algorithms." IAES International Journal of Artificial Intelligence (IJ-AI) 14, no. 1 (2025): 570. http://dx.doi.org/10.11591/ijai.v14.i1.pp570-583.

Full text

Abstract:

In this paper, the researchers studied the effects of different activation functions in hidden layers and how they impact the overfitting or underfitting of the model in the multiclass prediction of chronic diseases. This paper also evaluated the effects of varying the number of layers, and hyperparameters and its impact on the accuracy of the model and its generalization capabilities. It was found that exponential linear unit (ELU) does not have a significant advantage over rectified linear unit (ReLU) when used as an activation function in the hidden layer. Additionally, the performance of s

APA, Harvard, Vancouver, ISO, and other styles

35

Ronald, S. Cordova, Lyn R. Maata Rolou, Jawarneh Malik, I. Alshar'e Marwan, and C. Agustin Oliver. "Accurate prediction of chronic diseases using deep learning algorithms." IAES International Journal of Artificial Intelligence (IJ-AI) 14, no. 1 (2025): 570–83. https://doi.org/10.11591/ijai.v14.i1.pp570-583.

Full text

Abstract:

In this paper, the researchers studied the effects of different activation functions in hidden layers and how they impact the overfitting or underfitting of the model in the multiclass prediction of chronic diseases. This paper also evaluated the effects of varying the number of layers, and hyperparameters and its impact on the accuracy of the model and its generalization capabilities. It was found that exponential linear unit (ELU) does not have a significant advantage over rectified linear unit (ReLU) when used as an activation function in the hidden layer. Additionally, the performance of s

APA, Harvard, Vancouver, ISO, and other styles

36

Li, Qiang, Rui Wang, Guowei Li, and Zhengwei Xu. "Depth-to-basement Estimation of Basin Relief Using the BP Network." Journal of Physics: Conference Series 2651, no. 1 (2023): 012159. http://dx.doi.org/10.1088/1742-6596/2651/1/012159.

Full text

Abstract:

Abstract Nonlinear gravity inversion is a popular method for determining basin bottom relief and delineating basin configuration. However, traditional gravity inversion presents certain challenges, including the complexity and time demand of calculating and transforming large matrices, as well as instability and non-uniqueness caused by the inherently ill-posed nature of inversion problems. Over the past decade, deep learning, a subset of machine learning, has seen successful applications in geophysical interpretation and exploration. In this study, we propose an innovative method for estimati

APA, Harvard, Vancouver, ISO, and other styles

37

Vashisht, Priyanka, and Anvesha Katti. "Creative Image Synthesis Using Neural Style Transfer Techniques." International Journal of Innovative Research in Engineering and Management 12, no. 2 (2025): 145–53. https://doi.org/10.55524/ijirem.2025.12.2.23.

Full text

Abstract:

First Art generation, a profound display of human creativity, evolves with technological advancements, notably in deep learning. One striking innovation is Neural Style Transfer (NST), blending artistical flair with technological prowess. NST employs convolutional neural networks, such as VGG19, to fuse the content of one image with the style of another, yielding captivating artworks. VGG19, a seminal CNN architecture, features 3x3 convolutional filters, max-pooling layers, and Rectified Linear Unit (ReLU) activations, enabling it to discern both low-level and high-level features. Through rigo

APA, Harvard, Vancouver, ISO, and other styles

38

Lee, Hyeonjeong, Jaewon Lee, and Miyoung Shin. "Using Wearable ECG/PPG Sensors for Driver Drowsiness Detection Based on Distinguishable Pattern of Recurrence Plots." Electronics 8, no. 2 (2019): 192. http://dx.doi.org/10.3390/electronics8020192.

Full text

Abstract:

This paper aims to investigate the robust and distinguishable pattern of heart rate variability (HRV) signals, acquired from wearable electrocardiogram (ECG) or photoplethysmogram (PPG) sensors, for driver drowsiness detection. As wearable sensors are so vulnerable to slight movement, they often produce more noise in signals. Thus, from noisy HRV signals, we need to find good traits that differentiate well between drowsy and awake states. To this end, we explored three types of recurrence plots (RPs) generated from the R–R intervals (RRIs) of heartbeats: Bin-RP, Cont-RP, and ReLU-RP. Here Bin-

APA, Harvard, Vancouver, ISO, and other styles

39

AbdulQader, Dina A., Asma T. Saadoon, Marwa T. Naser, and Ali Hassan Jabbar. "Classification of COVID-19 from CT chest images using Convolutional Wavelet Neural Network." International Journal of Electrical and Computer Engineering (IJECE) 13, no. 1 (2023): 1078. http://dx.doi.org/10.11591/ijece.v13i1.pp1078-1085.

Full text

Abstract:

<p>Analyzing X-rays and computed tomography-scan (CT scan) images using a convolutional neural network (CNN) method is a very interesting subject, especially after coronavirus disease 2019 (COVID-19) pandemic. In this paper, a study is made on 423 patients’ CT scan images from Al-Kadhimiya (Madenat Al Emammain Al Kadhmain) hospital in Baghdad, Iraq, to diagnose if they have COVID or not using CNN. The total data being tested has 15000 CT-scan images chosen in a specific way to give a correct diagnosis. The activation function used in this research is the wavelet function, which differs f

APA, Harvard, Vancouver, ISO, and other styles

40

Dina, A. AbdulQader, T. Saadoon Asma, T. Naser Marwa, and Hassan Jabbar Ali. "Classification of COVID-19 from CT chest images using convolutional wavelet neural network." International Journal of Electrical and Computer Engineering (IJECE) 13, no. 1 (2023): 1078–85. https://doi.org/10.11591/ijece.v13i1.pp1078-1085.

Full text

Abstract:

Analyzing X-rays and computed tomography-scan (CT scan) images using a convolutional neural network (CNN) method is a very interesting subject, especially after coronavirus disease 2019 (COVID-19) pandemic. In this paper, a study is made on 423 patients’ CT scan images from Al-Kadhimiya (Madenat Al Emammain Al Kadhmain) hospital in Baghdad, Iraq, to diagnose if they have COVID or not using CNN. The total data being tested has 15000 CT-scan images chosen in a specific way to give a correct diagnosis. The activation function used in this research is the wavelet function, which differs from

APA, Harvard, Vancouver, ISO, and other styles

41

Hu, Baoquan, Jun Liu, Rongzhen Zhao, Yue Xu, and Tianlong Huo. "A New Fault Diagnosis Method for Unbalanced Data Based on 1DCNN and L2-SVM." Applied Sciences 12, no. 19 (2022): 9880. http://dx.doi.org/10.3390/app12199880.

Full text

Abstract:

In general, the measured health condition data from rolling bearings usually exhibit imbalanced distribution. However, traditional intelligent fault diagnosis methods usually assume that the data categories are balanced. To improve the diagnosis accuracy of unbalanced datasets, a new fault diagnosis method for unbalanced data based on 1DCNN and L2-SVM is proposed in this paper. Firstly, to prevent the minority class samples from being heavily suppressed by the rectified linear unit (ReLU) activation function in the traditional convolutional neural network (CNN), ReLU is improved by linear and

APA, Harvard, Vancouver, ISO, and other styles

42

Butt, F. M., L. Hussain, S. H. M. Jafri, et al. "Optimizing Parameters of Artificial Intelligence Deep Convolutional Neural Networks (CNN) to improve Prediction Performance of Load Forecasting System." IOP Conference Series: Earth and Environmental Science 1026, no. 1 (2022): 012028. http://dx.doi.org/10.1088/1755-1315/1026/1/012028.

Full text

Abstract:

Abstract Load Forecasting is an approach that is implemented to foresee the future load demand projected on some physical parameters such as loading on lines, temperature, losses, pressure, and weather conditions etc. This study is specifically aimed to optimize the parameters of deep convolutional neural networks (CNN) to improve the short-term load forecasting (STLF) and Medium-term load forecasting (MTLF) i.e. one day, one week, one month and three months. The models were tested based on the real-world case by conducting detailed experiments to validate their stability and practicality. The

APA, Harvard, Vancouver, ISO, and other styles

43

Xiangyang, Lin, Qinghua Xing, Zhang Han, and Chen Feng. "A Novel Activation Function of Deep Neural Network." Scientific Programming 2023 (August 4, 2023): 1–12. http://dx.doi.org/10.1155/2023/3873561.

Full text

Abstract:

In deep neural networks, the activation function is an important component. The most popular activation functions at the moment are Sigmoid, Sin, rectified linear unit (ReLU), and some variants of ReLU. However, each of them has its own weakness. To improve the network fitting and generalization ability, a new activation function, TSin, is designed. The basic design idea for TSin function is to rotate the Sin function 45° counterclockwise and then finetune it to give it multiple better properties needed as an activation function, such as nonlinearity, global differentiability, unsaturated prop

APA, Harvard, Vancouver, ISO, and other styles

44

Chen, Zhixian, Jialin Tang, Xueyuan Gong, and Qinglang Su. "Security of E-Health Systems Using Face Recognition Based on Convolutional Neural Network." International Journal of Extreme Automation and Connectivity in Healthcare 2, no. 2 (2020): 37–41. http://dx.doi.org/10.4018/ijeach.2020070104.

Full text

Abstract:

In order to improve the low accuracy of the face recognition methods in the case of e-health, this paper proposed a novel face recognition approach, which is based on convolutional neural network (CNN). In detail, through resolving the convolutional kernel, rectified linear unit (ReLU) activation function, dropout, and batch normalization, this novel approach reduces the number of parameters of the CNN model, improves the non-linearity of the CNN model, and alleviates overfitting of the CNN model. In these ways, the accuracy of face recognition is increased. In the experiments, the proposed ap

APA, Harvard, Vancouver, ISO, and other styles

45

Lim, Jaehyuk, Hogeun Yoo, Euihyuk Lee, et al. "Analysis of the Applicability of Blind Beamforming Techniques using Deep Neural Network to Defense Systems." Journal of the Korea Institute of Military Science and Technology 28, no. 1 (2025): 44–51. https://doi.org/10.9766/kimst.2025.28.1.044.

Full text

Abstract:

This paper compares the performance of deep neural networks(DNN) applied to antenna arrays with different element counts(8, 16, 32). The DNNs were designed using fully connected(FC) layers, comprising input, hidden, and output layers. Each hidden layer includes a single FC layer and a rectified linear unit(ReLU) activation function. Results indicate that blind beamforming with DNNs performs well with fewer elements but degrades as the number of elements increases due to increased nonlinearity, complicating training. State-of-the-art defense radar systems require many array elements, making cur

APA, Harvard, Vancouver, ISO, and other styles

46

Siti Shofiah and Brasie Pradana Sela Bunga Reska Ayu. "Pengenalan Wajah dengan Viola Jones." Jurnal Penelitian Rumpun Ilmu Teknik 3, no. 4 (2024): 89–95. https://doi.org/10.55606/juprit.v3i4.4363.

Full text

Abstract:

The Viola-Jones algorithm in OpenCV is efficient for detecting faces. The study is the accuracy of face detection using Viola-Jones on FCNN. The data is divided into training, testing, and validation sets. The FCNN model achieves high accuracy but suffers from overfitting. Techniques such as regularization and dropout can improve performance. The training duration is relatively short. FCNN machine learning model. The first layer is a hidden layer with 128 neurons and uses the ReLU (Rectified Linear Unit) activation function. The second layer is the output layer with ten drilled neurons showing

APA, Harvard, Vancouver, ISO, and other styles

47

Djarum, D. H., Z. Ahmad, and J. Zhang. "Performance Analysis of Neural Network Architecture in Developing Real-Time Malaysian River Water Quality Model." IOP Conference Series: Materials Science and Engineering 1257, no. 1 (2022): 012022. http://dx.doi.org/10.1088/1757-899x/1257/1/012022.

Full text

Abstract:

Abstract According to the world health organization, 485,000 people died each year due to water-related health diseases which are mainly contributed by poor river water quality. As a result, water quality monitoring stations have been deployed across the world. Unfortunately, due to the complex nature of the off-site water quality parameters, the water quality index (WQI) cannot be assessed in real-time. This has led to a significant push for the scientific community to develop an accurate and robust water quality prediction model. The dynamic and nonlinear nature of water quality parameters a

APA, Harvard, Vancouver, ISO, and other styles

48

Zheng, Shuxin, Qi Meng, Huishuai Zhang, Wei Chen, Nenghai Yu, and Tie-Yan Liu. "Capacity Control of ReLU Neural Networks by Basis-Path Norm." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 5925–32. http://dx.doi.org/10.1609/aaai.v33i01.33015925.

Full text

Abstract:

Recently, path norm was proposed as a new capacity measure for neural networks with Rectified Linear Unit (ReLU) activation function, which takes the rescaling-invariant property of ReLU into account. It has been shown that the generalization error bound in terms of the path norm explains the empirical generalization behaviors of the ReLU neural networks better than that of other capacity measures. Moreover, optimization algorithms which take path norm as the regularization term to the loss function, like Path-SGD, have been shown to achieve better generalization performance. However, the path

APA, Harvard, Vancouver, ISO, and other styles

49

Handayanto, Rahmadya Trias, and Herlawati Herlawati. "Prediksi Kelas Jamak dengan Deep Learning Berbasis Graphics Processing Units." Jurnal Kajian Ilmiah 20, no. 1 (2020): 67–76. http://dx.doi.org/10.31599/jki.v20i1.71.

Full text

Abstract:

For the first time, machine learning did the classical classification process using two classes (bi-class) such as class -1 and class +1, 0 and 1, or the form of categories such as true and false. Famous methods used are Artificial Neural Networks (ANN) and Support Vector Machine (SVM). The current development was a problem with more than two classes, known as multi-class classes. For SVM sometimes the plural classes are overcome by doing a gradual process like a decision tree (DT) method. Meanwhile, ANN has experienced rapid development and is currently being developed with a large number of

APA, Harvard, Vancouver, ISO, and other styles

50

Ali, Saqib, Jianqiang Li, Yan Pei, Muhammad Saqlain Aslam, Zeeshan Shaukat, and Muhammad Azeem. "An Effective and Improved CNN-ELM Classifier for Handwritten Digits Recognition and Classification." Symmetry 12, no. 10 (2020): 1742. http://dx.doi.org/10.3390/sym12101742.

Full text

Abstract:

Optical character recognition is gaining immense importance in the domain of deep learning. With each passing day, handwritten digits (0–9) data are increasing rapidly, and plenty of research has been conducted thus far. However, there is still a need to develop a robust model that can fetch useful information and investigate self-build handwritten digit data efficiently and effectively. The convolutional neural network (CNN) models incorporating a sigmoid activation function with a large number of derivatives have low efficiency in terms of feature extraction. Here, we designed a novel CNN mo

APA, Harvard, Vancouver, ISO, and other styles

We offer discounts on all premium plans for authors whose works are included in thematic literature selections. Contact us to get a unique promo code!