To see the other types of publications on this topic, follow the link: Deep neural networks architecture.
Journal articles on the topic 'Deep neural networks architecture'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Deep neural networks architecture.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Suk-Hwan, Jung, and Chung Yong-Joo. "Sound event detection using deep neural networks." TELKOMNIKA Telecommunication, Computing, Electronics and Control 18, no. 5 (2020): 2587~2596. https://doi.org/10.12928/TELKOMNIKA.v18i5.14246.
Full textAbstract:
We applied various architectures of deep neural networks for sound event detection and compared their performance using two different datasets. Feed forward neural network (FNN), convolutional neural network (CNN), recurrent neural network (RNN) and convolutional recurrent neural network (CRNN) were implemented using hyper-parameters optimized for each architecture and dataset. The results show that the performance of deep neural networks varied significantly depending on the learning rate, which can be optimized by conducting a series of experiments on the validation data over predetermined ranges. Among the implemented architectures, the CRNN performed best under all testing conditions, followed by CNN. Although RNN was effective in tracking the time-correlation information in audio signals, it exhibited inferior performance compared to the CNN and the CRNN. Accordingly, it is necessary to develop more optimization strategies for implementing RNN in sound event detection.
2
Laveglia, Vincenzo, and Edmondo Trentin. "Downward-Growing Neural Networks." Entropy 25, no. 5 (2023): 733. http://dx.doi.org/10.3390/e25050733.
Full textAbstract:
A major issue in the application of deep learning is the definition of a proper architecture for the learning machine at hand, in such a way that the model is neither excessively large (which results in overfitting the training data) nor too small (which limits the learning and modeling capabilities of the automatic learner). Facing this issue boosted the development of algorithms for automatically growing and pruning the architectures as part of the learning process. The paper introduces a novel approach to growing the architecture of deep neural networks, called downward-growing neural network (DGNN). The approach can be applied to arbitrary feed-forward deep neural networks. Groups of neurons that negatively affect the performance of the network are selected and grown with the aim of improving the learning and generalization capabilities of the resulting machine. The growing process is realized via replacement of these groups of neurons with sub-networks that are trained relying on ad hoc target propagation techniques. In so doing, the growth process takes place simultaneously in both the depth and width of the DGNN architecture. We assess empirically the effectiveness of the DGNN on several UCI datasets, where the DGNN significantly improves the average accuracy over a range of established deep neural network approaches and over two popular growing algorithms, namely, the AdaNet and the cascade correlation neural network.
3
Svitlana, Shapovalova, and Moskalenko Yurii. "METHODS FOR INCREASING THE CLASSIFICATION ACCURACY BASED ON MODIFICATIONS OF THE BASIC ARCHITECTURE OF CONVOLUTIONAL NEURAL NETWORKS." ScienceRise 6 (December 30, 2020): 10–16. https://doi.org/10.21303/2313-8416.2020.001550.
Full textAbstract:
<strong>Object of research:</strong> basic architectures of deep learning neural networks. <strong>Investigated problem:</strong> insufficient accuracy of solving the classification problem based on the basic architectures of deep learning neural networks. An increase in accuracy requires a significant complication of the architecture, which, in turn, leads to an increase in the required computing resources, as well as the consumption of video memory and the cost of learning/output time. Therefore, the problem arises of determining such methods for modifying basic architectures that improve the classification accuracy and require insignificant additional computing resources. <strong>Main scientific results:</strong> based on the analysis of existing methods for improving the classification accuracy on the convolutional networks of basic architectures, it is determined what is most effective: scaling the ScanNet architecture, learning the ensemble of TreeNet models, integrating several CBNet backbone networks. For computational experiments, these modifications of the basic architectures are implemented, as well as their combinations: ScanNet + TreeNet, ScanNet + CBNet. The effectiveness of these methods in comparison with basic architectures has been proven when solving the problem of recognizing malignant tumors with diagnostic images – SIIM-ISIC Melanoma Classification, the train/test set of which is presented on the Kaggle platform. The accuracy value for the area under the ROC curve metric has increased from 0.94489 (basic architecture network) to 0.96317 (network with ScanNet + CBNet modifications). At the same time, the output compared to the basic architecture (EfficientNet-b5) increased from 440 to 490 seconds, and the consumption of video memory increased from 8 to 9.2 gigabytes, which is acceptable. <strong>Innovative technological product:</strong> methods for achieving high recognition accuracy from a diagnostic signal based on deep learning neural networks of basic architectures. <strong>Scope of application of the innovative technological product: </strong>automatic diagnostics systems in the following areas: medicine, seismology, astronomy (classification by images) onboard control systems and systems for monitoring transport and vehicle flows or visitors (recognition of scenes with camera frames).
4
Паршин, А. И., М. Н. Аралов, В. Ф. Барабанов, and Н. И. Гребенникова. "RANDOM MULTI-MODAL DEEP LEARNING IN THE PROBLEM OF IMAGE RECOGNITION." ВЕСТНИК ВОРОНЕЖСКОГО ГОСУДАРСТВЕННОГО ТЕХНИЧЕСКОГО УНИВЕРСИТЕТА, no. 4 (October 20, 2021): 21–26. http://dx.doi.org/10.36622/vstu.2021.17.4.003.
Full textAbstract:
Задача распознавания изображений - одна из самых сложных в машинном обучении, требующая от исследователя как глубоких знаний, так и больших временных и вычислительных ресурсов. В случае использования нелинейных и сложных данных применяются различные архитектуры глубоких нейронных сетей, но при этом сложным вопросом остается проблема выбора нейронной сети. Основными архитектурами, используемыми повсеместно, являются свёрточные нейронные сети (CNN), рекуррентные нейронные сети (RNN), глубокие нейронные сети (DNN). На основе рекуррентных нейронных сетей (RNN) были разработаны сети с долгой краткосрочной памятью (LSTM) и сети с управляемыми реккурентными блоками (GRU). Каждая архитектура нейронной сети имеет свою структуру, свои настраиваемые и обучаемые параметры, обладает своими достоинствами и недостатками. Комбинируя различные виды нейронных сетей, можно существенно улучшить качество предсказания в различных задачах машинного обучения. Учитывая, что выбор оптимальной архитектуры сети и ее параметров является крайне трудной задачей, рассматривается один из методов построения архитектуры нейронных сетей на основе комбинации свёрточных, рекуррентных и глубоких нейронных сетей. Показано, что такие архитектуры превосходят классические алгоритмы машинного обучения The image recognition task is one of the most difficult in machine learning, requiring both deep knowledge and large time and computational resources from the researcher. In the case of using nonlinear and complex data, various architectures of deep neural networks are used but the problem of choosing a neural network remains a difficult issue. The main architectures used everywhere are convolutional neural networks (CNN), recurrent neural networks (RNN), deep neural networks (DNN). Based on recurrent neural networks (RNNs), Long Short Term Memory Networks (LSTMs) and Controlled Recurrent Unit Networks (GRUs) were developed. Each neural network architecture has its own structure, customizable and trainable parameters, and advantages and disadvantages. By combining different types of neural networks, you can significantly improve the quality of prediction in various machine learning problems. Considering that the choice of the optimal network architecture and its parameters is an extremely difficult task, one of the methods for constructing the architecture of neural networks based on a combination of convolutional, recurrent and deep neural networks is considered. We showed that such architectures are superior to classical machine learning algorithms
5
Gallicchio, Claudio, and Alessio Micheli. "Fast and Deep Graph Neural Networks." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (2020): 3898–905. http://dx.doi.org/10.1609/aaai.v34i04.5803.
Full textAbstract:
We address the efficiency issue for the construction of a deep graph neural network (GNN). The approach exploits the idea of representing each input graph as a fixed point of a dynamical system (implemented through a recurrent neural network), and leverages a deep architectural organization of the recurrent units. Efficiency is gained by many aspects, including the use of small and very sparse networks, where the weights of the recurrent units are left untrained under the stability condition introduced in this work. This can be viewed as a way to study the intrinsic power of the architecture of a deep GNN, and also to provide insights for the set-up of more complex fully-trained models. Through experimental results, we show that even without training of the recurrent connections, the architecture of small deep GNN is surprisingly able to achieve or improve the state-of-the-art performance on a significant set of tasks in the field of graphs classification.
6
Christy, Ntambwe Kabamba, Mpuekela .N Lucie, Ntumba .B Simon, and Mbuyi .M Eugene. "Convolutional Neural Networks and Pattern Recognition: Application to Image Classification." International Journal of Computer Science Issues 16, no. 6 (2019): 10–18. https://doi.org/10.5281/zenodo.3987070.
Full textAbstract:
This research study focuses on pattern recognition using convolutional neural network. Deep neural network has been choosing as the best option for the training process because it produced a high percentage of accuracy. We designed different architectures of convolutional neural network in order to find the one with high accuracy of image classification and optimum bias. We used CIFAR-10 data set that contains 60000 Images to train our model on architectures. The best architecture was able to classify images with 95.55% of accuracy and an error of 0.32% using cross validation method. We note that, the numbers of epoch while running the model and the depth of the architecture are factors that contributed to get this performance.
7
Kniaz, V. V., V. S. Gorbatsevich, and V. A. Mizginov. "THERMALNET: A DEEP CONVOLUTIONAL NETWORK FOR SYNTHETIC THERMAL IMAGE GENERATION." ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLII-2/W4 (May 10, 2017): 41–45. http://dx.doi.org/10.5194/isprs-archives-xlii-2-w4-41-2017.
Full textAbstract:
Deep convolutional neural networks have dramatically changed the landscape of the modern computer vision. Nowadays methods based on deep neural networks show the best performance among image recognition and object detection algorithms. While polishing of network architectures received a lot of scholar attention, from the practical point of view the preparation of a large image dataset for a successful training of a neural network became one of major challenges. This challenge is particularly profound for image recognition in wavelengths lying outside the visible spectrum. For example no infrared or radar image datasets large enough for successful training of a deep neural network are available to date in public domain. Recent advances of deep neural networks prove that they are also capable to do arbitrary image transformations such as super-resolution image generation, grayscale image colorisation and imitation of style of a given artist. Thus a natural question arise: how could be deep neural networks used for augmentation of existing large image datasets? This paper is focused on the development of the Thermalnet deep convolutional neural network for augmentation of existing large visible image datasets with synthetic thermal images. The Thermalnet network architecture is inspired by colorisation deep neural networks.
8
Ghimire, Deepak, Dayoung Kil, and Seong-heum Kim. "A Survey on Efficient Convolutional Neural Networks and Hardware Acceleration." Electronics 11, no. 6 (2022): 945. http://dx.doi.org/10.3390/electronics11060945.
Full textAbstract:
Over the past decade, deep-learning-based representations have demonstrated remarkable performance in academia and industry. The learning capability of convolutional neural networks (CNNs) originates from a combination of various feature extraction layers that fully utilize a large amount of data. However, they often require substantial computation and memory resources while replacing traditional hand-engineered features in existing systems. In this review, to improve the efficiency of deep learning research, we focus on three aspects: quantized/binarized models, optimized architectures, and resource-constrained systems. Recent advances in light-weight deep learning models and network architecture search (NAS) algorithms are reviewed, starting with simplified layers and efficient convolution and including new architectural design and optimization. In addition, several practical applications of efficient CNNs have been investigated using various types of hardware architectures and platforms.
9
Wai Yong, Ching, Kareen Teo, Belinda Pingguan Murphy, Yan Chai Hum, and Khin Wee Lai. "CORSegNet: Deep Neural Network for Core Object Segmentation on Medical Images." Journal of Medical Imaging and Health Informatics 11, no. 5 (2021): 1364–71. http://dx.doi.org/10.1166/jmihi.2021.3380.
Full textAbstract:
In recent decades, convolutional neural networks (CNNs) have delivered promising results in vision-related tasks across different domains. Previous studies have introduced deeper network architectures to further improve the performances of object classification, localization, and segmentation. However, this induces the complexity in mapping network’s layer to the processing elements in the ventral visual pathway. Although CORnet models are not precisely biomimetic, they are closer approximations to the anatomy of ventral visual pathway compared with other deep neural networks. The uniqueness of this architecture inspires us to extend it into a core object segmentation network, CORSegnet-Z. This architecture utilizes CORnet-Z building blocks as the encoding elements. We train and evaluate the proposed model using two large datasets. Our proposed model shows significant improvements on the segmentation metrics in delineating cartilage tissues from knee magnetic resonance (MR) images and segmenting lesion boundary from dermoscopic images.
10
Feng, Wenfeng, Xin Zhang, Qiushuang Song, and Guoying Sun. "The Incoherence of Deep Isotropic Neural Networks Increases Their Performance in Image Classification." Electronics 11, no. 21 (2022): 3603. http://dx.doi.org/10.3390/electronics11213603.
Full textAbstract:
Although neural-network architectures are critical for their performance, how the structural characteristics of a neural network affect its performance has still not been fully explored. Here, we map architectures of neural networks to directed acyclic graphs (DAGs), and find that incoherence, a structural characteristic to measure the order of DAGs, is a good indicator for the performance of corresponding neural networks. Therefore, we propose a deep isotropic neural-network architecture by folding a chain of the same blocks and then connecting the blocks with skip connections at different distances. Our model, named FoldNet, has two distinguishing features compared with traditional residual neural networks. First, the distances between block pairs connected by skip connections increase from always equal to one to specially selected different values, which lead to more incoherent graphs and let the neural network explore larger receptive fields and, thus, enhance its multi-scale representation ability. Second, the number of direct paths increases from one to multiple, which leads to a larger proportion of shorter paths and, thus, improves the direct propagation of information throughout the entire network. Image-classification results on CIFAR-10 and Tiny ImageNet benchmarks suggested that our new network architecture performs better than traditional residual neural networks. FoldNet with 25.4M parameters can achieve 72.67% top-1 accuracy on the Tiny ImageNet after 100 epochs, which is competitive compared with the-state-of-art results on the Tiny ImageNet.
11
Guo, Xinwei, Yong Wu, Jingjing Miao, and Yang Chen. "LiteGaze: Neural architecture search for efficient gaze estimation." PLOS ONE 18, no. 5 (2023): e0284814. http://dx.doi.org/10.1371/journal.pone.0284814.
Full textAbstract:
Gaze estimation plays a critical role in human-centered vision applications such as human–computer interaction and virtual reality. Although significant progress has been made in automatic gaze estimation by deep convolutional neural networks, it is still difficult to directly deploy deep learning based gaze estimation models across different edge devices, due to the high computational cost and various resource constraints. This work proposes LiteGaze, a deep learning framework to learn architectures for efficient gaze estimation via neural architecture search (NAS). Inspired by the once-for-all model (Cai et al., 2020), this work decouples the model training and architecture search into two different stages. In particular, a supernet is trained to support diverse architectural settings. Then specialized sub-networks are selected from the obtained supernet, given different efficiency constraints. Extensive experiments are performed on two gaze estimation datasets and demonstrate the superiority of the proposed method over previous works, advancing the real-time gaze estimation on edge devices.
12
Mamun, Abdullah Al, Em Poh Ping, Jakir Hossen, Anik Tahabilder, and Busrat Jahan. "A Comprehensive Review on Lane Marking Detection Using Deep Neural Networks." Sensors 22, no. 19 (2022): 7682. http://dx.doi.org/10.3390/s22197682.
Full textAbstract:
Lane marking recognition is one of the most crucial features for automotive vehicles as it is one of the most fundamental requirements of all the autonomy features of Advanced Driver Assistance Systems (ADAS). Researchers have recently made promising improvements in the application of Lane Marking Detection (LMD). This research article has taken the initiative to review lane marking detection, mainly using deep learning techniques. This paper initially discusses the introduction of lane marking detection approaches using deep neural networks and conventional techniques. Lane marking detection frameworks can be categorized into single-stage and two-stage architectures. This paper elaborates on the network’s architecture and the loss function for improving the performance based on the categories. The network’s architecture is divided into object detection, classification, and segmentation, and each is discussed, including their contributions and limitations. There is also a brief indication of the simplification and optimization of the network for simplifying the architecture. Additionally, comparative performance results with a visualization of the final output of five existing techniques is elaborated. Finally, this review is concluded by pointing to particular challenges in lane marking detection, such as generalization problems and computational complexity. There is also a brief future direction for solving the issues, for instance, efficient neural network, Meta, and unsupervised learning.
13
Kalinina, M. O., and P. L. Nikolaev. "Book spine recognition with the use of deep neural networks." Computer Optics 44, no. 6 (2020): 968–77. http://dx.doi.org/10.18287/2412-6179-co-731.
Full textAbstract:
Nowadays deep neural networks play a significant part in various fields of human activity. Especially they benefit spheres dealing with large amounts of data and lengthy operations on obtaining and processing information from the visual environment. This article deals with the development of a convolutional neural network based on the YOLO architecture, intended for real-time book recognition. The creation of an original data set and the training of the deep neural network are described. The structure of the neural network obtained is presented and the most frequently used metrics for estimating the quality of the network performance are considered. A brief review of the existing types of neural network architectures is also made. YOLO architecture possesses a number of advantages that allow it to successfully compete with other models and make it the most suitable variant for creating an object detection network since it enables some of the common disadvantages of such networks to be significantly mitigated (such as recognition of similarly looking, same-color book coves or slanted books). The results obtained in the course of training the deep neural network allow us to use it as a basis for the development of the software for book spine recognition.
14
Baptista, Marcia, Helmut Prendinger, and Elsa Henriques. "Prognostics in Aeronautics with Deep Recurrent Neural Networks." PHM Society European Conference 5, no. 1 (2020): 11. http://dx.doi.org/10.36001/phme.2020.v5i1.1230.
Full textAbstract:
Recurrent neural networks (RNNs) such as LSTM and GRU are not new to the field of prognostics. However, the performance of neural networks strongly depends on their architectural structure. In this work, we investigate a hybrid network architecture that is a combination of recurrent and feed-forward (conditional) layers. Two networks, one recurrent and another feed-forward, are chained together, with inference and weight gradients being learned using the standard back-propagation learning procedure. To better tune the network, instead of using raw sensor data, we do some preprocessing on the data, using mostly simple but effective statistics (researched in previous work). This helps the feature extraction phase and eases the problem of finding a suitable network configuration among the immense set of possible ones. This is not the first proposal of a hybrid network in prognostics but our work is novel in the sense that it performs a more comprehensive comparison of this type of architecture for different RNN layers and number of layers. Also, we compare our work with other classical machine learning methods. Evaluation is performed on two real-world case studies from the aero-engine industry: one involving a critical valve subsystem of the jet engine and another the whole reliability of the jet engine. Our goal here is to compare two cases contrasting micro (valve) and macro (whole engine) prognostics. Our results indicate that the performance of the LSTM and GRU deep networks are significantly better than that of other models.
15
Varghese, Prathibha, and Arockia Selva Saroja. "Biologically inspired deep residual networks." IAES International Journal of Artificial Intelligence (IJ-AI) 12, no. 4 (2023): 1873. http://dx.doi.org/10.11591/ijai.v12.i4.pp1873-1882.
Full textAbstract:
<p>Many difficult computer vision issues have been effectively tackled by deep neural networks. Not only that but it was discovered that traditional residual neural networks (ResNet) captures features with high generalizability, rendering it a cutting-edge convolutional neural network (CNN). The images classified by the authors of this research introduce a deep residual neural network that is biologically inspired introduces hexagonal convolutions along the skip connection. With the competitive training techniques, the effectiveness of several ResNet variations using square and hexagonal convolution is assessed. Using the hex-convolution on skip connection, we designed a family of ResNet architecture,hexagonal residual neural network (HexResNet), which achieves the highest testing accuracy of 94.02%, and 55.71% on Canadian Institute For Advanced Research (CIFAR)-10 and TinyImageNet, respectively. We demonstrate that the suggested method improves vanilla ResNet architectures’ baseline image classification accuracy on the CIFAR-10 dataset, and a similar effect was seen on the TinyImageNet dataset. For Tiny- ImageNet and CIFAR-10, we saw an average increase in accuracy of 1.46% and 0.48% in the baseline Top-1 accuracy, respectively. The generalized performance of advancements was reported for the suggested bioinspired deep residual networks. This represents an area that might be explored more extensively in the future to enhance all the discriminative power of image classification systems.</p>
16
Varghese, Prathibha, and Arockia Selva Saroja. "Biologically inspired deep residual networks." IAES International Journal of Artificial Intelligence (IJ-AI) 12, no. 4 (2023): 1873–82. https://doi.org/10.11591/ijai.v12.i4.pp1873-1882.
Full textAbstract:
Many difficult computer vision issues have been effectively tackled by deep neural networks. Not only that but it was discovered that traditional residual neural networks (ResNet) captures features with high generalizability, rendering it a cutting-edge convolutional neural network (CNN). The images classified by the authors of this research introduce a deep residual neural network that is biologically inspired introduces hexagonal convolutions along the skip connection. With the competitive training techniques, the effectiveness of several ResNet variations using square and hexagonal convolution is assessed. Usingthe hex-convolution on skip connection, we designed a family of ResNet architecture, hexagonal residual neural network (HexResNet), which achieves the highest testing accuracy of 94.02%, and 55.71% on Canadian Institute For AdvancedResearch (CIFAR)-10 and TinyImageNet, respectively. We demonstrate that the suggested method improves vanilla ResNet architectures’ baseline image classification accuracy on the CIFAR-10 dataset, and a similar effect was seen on the TinyImageNet dataset. For Tiny- ImageNet and CIFAR-10, we saw an average increase in accuracy of 1.46% and 0.48% in the baseline Top-1 accuracy, respectively. The generalized performance of advancements was reported for the suggested bioinspired deep residual networks. This represents an area that might be explored more extensively in the future to enhance all the discriminative power of image classification systems.
17
İMİK ŞİMŞEK, Özlem, and Barış Baykant ALAGÖZ. "A THEORETICAL INVESTIGATION ON TRAINING OF PIPE-LIKE NEURAL NETWORK BENCHMARK ARCHITECTURES AND PERFORMANCE COMPARISONS OF POPULAR TRAINING ALGORITHMS." Mühendislik Bilimleri ve Tasarım Dergisi 10, no. 4 (2022): 1251–71. http://dx.doi.org/10.21923/jesd.1104772.
Full textAbstract:
Architectures of neural networks affect the training performance of artificial neural networks. For more consistent performance evaluation of training algorithms, hard-to-train benchmarking architectures should be used. This study introduces a benchmark neural network architecture, which is called pipe-like architecture, and presents training performance analyses for popular Neural Network Backpropagation Algorithms (NNBA) and well-known Metaheuristic Search Algorithms (MSA). The pipe-like neural architectures essentially resemble an elongated fraction of a deep neural network and form a narrowed long bottleneck for the learning process. Therefore, they can significantly complicate the training process by causing the gradient vanishing problems and large training delays in backward propagation of parameter updates throughout the elongated pipe-like network. The training difficulties of pipe-like architectures are theoretically demonstrated in this study by considering the upper bound of weight updates according to an aggregated one-neuron learning channels conjecture. These analyses also contribute to Baldi et al.'s learning channel theorem of neural networks in a practical aspect. The training experiments for popular NNBA and MSA algorithms were conducted on the pipe-like benchmark architecture by using a biological dataset. Moreover, a Normalized Overall Performance Scoring (NOPS) was performed for the criterion-based assessment of overall performance of training algorithms.
18
Bodyansky, E. V., and Т. Е. Antonenko. "Deep neo-fuzzy neural network and its learning." Bionics of Intelligence 1, no. 92 (2019): 3–8. http://dx.doi.org/10.30837/bi.2019.1(92).01.
Full textAbstract:
Optimizing the learning speedof deep neural networks is an extremely important issue. Modern approaches focus on the use of neural networksbased on the Rosenblatt perceptron. But the results obtained are not satisfactory for industrial and scientific needs inthe context of the speed of learning neural networks. Also, this approach stumbles upon the problems of a vanishingand exploding gradient. To solve the problem, the paper proposed using a neo-fuzzy neuron, whose properties arebased on the F-transform. The article discusses the use of neo-fuzzy neuron as the main component of the neuralnetwork. The architecture of a deep neo-fuzzy neural network is shown, as well as a backpropagation algorithmfor this architecture with a triangular membership function for neo-fuzzy neuron. The main advantages of usingneo-fuzzy neuron as the main component of the neural network are given. The article describes the properties of aneo-fuzzy neuron that addresses the issues of improving speed and vanishing or exploding gradient. The proposedneo-fuzzy deep neural network architecture is compared with standard deep networks based on the Rosenblattperceptron.
19
Caffaratti, Gabriel Dario, Martín Gastón Marchetta, and Raymundo Quilez Forradellas. "Stereo Matching through Squeeze Deep Neural Networks." Inteligencia Artificial 22, no. 63 (2019): 16–38. http://dx.doi.org/10.4114/intartif.vol22iss63pp16-38.
Full textAbstract:
Visual depth recognition through Stereo Matching is an active field of research due to the numerous applications in robotics, autonomous driving, user interfaces, etc. Multiple techniques have been developed in the last two decades to achieve accurate disparity maps in short time. With the arrival of Deep Leaning architectures, different fields of Artificial Vision, but mainly on image recognition, have achieved a great progress due to their easier training capabilities and reduction of parameters. This type of networks brought the attention of the Stereo Matching researchers who successfully applied the same concept to generate disparity maps. Even though multiple approaches have been taken towards the minimization of the execution time and errors in the results, most of the time the number of parameters of the networks is neither taken into consideration nor optimized. Inspired on the Squeeze-Nets developed for image recognition, we developed a Stereo Matching Squeeze neural network architecture capable of providing disparity maps with a highly reduced network size without a significant impact on quality and execution time compared with state of the art architectures. In addition, with the purpose of improving the quality of the solution and get solutions closer to real time, an extra refinement module is proposed and several tests are performed using different input size reductions.
20
Erdal, Mehmet, and Friedhelm Schwenker. "Learnability of the Boolean Innerproduct in Deep Neural Networks." Entropy 24, no. 8 (2022): 1117. http://dx.doi.org/10.3390/e24081117.
Full textAbstract:
In this paper, we study the learnability of the Boolean inner product by a systematic simulation study. The family of the Boolean inner product function is known to be representable by neural networks of threshold neurons of depth 3 with only 2n+1 units (n the input dimension)—whereas an exact representation by a depth 2 network cannot possibly be of polynomial size. This result can be seen as a strong argument for deep neural network architectures. In our study, we found that this depth 3 architecture of the Boolean inner product is difficult to train, much harder than the depth 2 network, at least for the small input size scenarios n≤16. Nonetheless, the accuracy of the deep architecture increased with the dimension of the input space to 94% on average, which means that multiple restarts are needed to find the compact depth 3 architecture. Replacing the fully connected first layer by a partially connected layer (a kind of convolutional layer sparsely connected with weight sharing) can significantly improve the learning performance up to 99% accuracy in simulations. Another way to improve the learnability of the compact depth 3 representation of the inner product could be achieved by adding just a few additional units into the first hidden layer.
21
Zheng, Wenqi, Yangyi Zhao, Yunfan Chen, Jinhong Park, and Hyunchul Shin. "Hardware Architecture Exploration for Deep Neural Networks." Arabian Journal for Science and Engineering 46, no. 10 (2021): 9703–12. http://dx.doi.org/10.1007/s13369-021-05455-4.
Full text
22
Gottapu, Ram Deepak, and Cihan H. Dagli. "Efficient Architecture Search for Deep Neural Networks." Procedia Computer Science 168 (2020): 19–25. http://dx.doi.org/10.1016/j.procs.2020.02.246.
Full text
23
Premanand, Ghadekar, Singh Gurdeep, Datta Joydeep, et al. "COVID-19 Face Mask Detection using Deep Convolutional Neural Networks & Computer Vision." Indian Journal of Science and Technology 14, no. 38 (2021): 2899–915. https://doi.org/10.17485/IJST/v14i38.996.
Full textAbstract:
<strong>Objectives:</strong> To propose a model which could classify in real-time if an individual is wearing a face mask or not wearing a face mask. A lightweight system that could be easily deployed and assist in surveillance. <strong>Methods/Statisticalanalysis:</strong> Analysis of the proposed model shows a limited number of research studies with regards to facial localizations. Several state-of-the-art methods were taken into considerations out of which the CNN architectural approach is analyzed in this study. Taking into consideration the use-case of deployments and structuring, a new Keras-based model is proposed that surpasses the achievement results of MobileNet-V2 and VGG-16 standard architectures. Effective facial localization is tackled with the MTCNN approach. <strong>Findings:</strong> The system has achieved a confidence score of 0.9914, an average weighted F1-score of 0.98, a precision value of 0.99. The proposed model has been compared with standard architectures of VGG16 and MobileNetV2 with regard to the accuracy, support values, precision, recall, and F1-score metrics. The proposed model performs better w.r.t traditional architectures. The average latency involved in prediction is 0.034 seconds making the average FPS 30 Frames per second. The compact architecture makes the model best for deployment in real-time scenarios. The system incorporates the concept of image localization with Multi-Task Cascaded Convolutional Neural Network (MTCNN) architecture. The analysis shows MTCNN is performing much better than Haar-Cascade in real-time facial prediction scenarios. <strong>Novelty/Applications:</strong> This compact architecture with minimal layers is easily deployable in edge devices. It can be used for mass screening at public places like railway stops, bus stops, streets, malls, entrances, schools, and many service-oriented business verticals requiring users to access the services as long as the mask has been worn correctly. <strong>Keywords:</strong> COVID19; Deep learning; Computer Vision; Mask Detection; Deep Convolutional Neural Network; Image Localization
24
Pelt, Daniël M., and James A. Sethian. "A mixed-scale dense convolutional neural network for image analysis." Proceedings of the National Academy of Sciences 115, no. 2 (2017): 254–59. http://dx.doi.org/10.1073/pnas.1715832114.
Full textAbstract:
Deep convolutional neural networks have been successfully applied to many image-processing problems in recent works. Popular network architectures often add additional operations and connections to the standard architecture to enable training deeper networks. To achieve accurate results in practice, a large number of trainable parameters are often required. Here, we introduce a network architecture based on using dilated convolutions to capture features at different image scales and densely connecting all feature maps with each other. The resulting architecture is able to achieve accurate results with relatively few parameters and consists of a single set of operations, making it easier to implement, train, and apply in practice, and automatically adapts to different problems. We compare results of the proposed network architecture with popular existing architectures for several segmentation problems, showing that the proposed architecture is able to achieve accurate results with fewer parameters, with a reduced risk of overfitting the training data.
25
Gupta, Rajat, and Rakesh Jindal. "Impact of Too Many Neural Network Layers on Overfitting." International Journal of Computer Science and Mobile Computing 14, no. 5 (2025): 1–14. https://doi.org/10.47760/ijcsmc.2025.v14i05.001.
Full textAbstract:
Deep neural networks have revolutionized artificial intelligence by enabling models to learn intricate data representations. However, when these networks become too deep, they risk overfitting—memorizing training data rather than learning patterns that generalize well to new inputs. Excessive complexity can lead models to capture irrelevant noise, and issues such as vanishing/exploding gradients, high computational costs, and the curse of dimensionality further complicate training deep architectures. This paper explores how neural network layers function in learning and the challenges that arise with increasing depth. It reviews regularization methods like L1/L2 penalties, dropout, and batch normalization, which help counteract overfitting and improve generalization. It also discusses training enhancements such as adaptive learning rate optimizers, gradient clipping, and early stopping for better efficiency and stability. Transfer learning is highlighted as a strategy to leverage pre-trained models while avoiding unnecessary depth. The paper also examines real-world cases where deep networks struggled to generalize and how techniques like neural architecture search (NAS), sparse networks, and meta-learning helped overcome these limitations. The future of deep learning lies in building efficient, flexible, and generalizable models that achieve high performance without excessive complexity. Through thoughtful architectural design and optimization, researchers can develop robust models that deliver accuracy without unnecessary depth.
26
Sewak, Mohit, Sanjay K. Sahay, and Hemant Rathore. "An Overview of Deep Learning Architecture of Deep Neural Networks and Autoencoders." Journal of Computational and Theoretical Nanoscience 17, no. 1 (2020): 182–88. http://dx.doi.org/10.1166/jctn.2020.8648.
Full textAbstract:
The recent wide applications of deep learning in multiple fields has shown a great progress, but to perform optimally, it requires the adjustment of various architectural features and hyper-parameters. Moreover, deep learning could be used with multiple varieties of architecture aimed at different objectives, e.g., autoencoders are popular for un-supervised learning applications for reducing the dimensionality of the dataset. Similarly, deep neural networks are popular for supervised learning applications viz., classification, regression, etc. Besides the type of deep learning architecture, some other decision criteria and parameter selection decisions are required for determining each layer size, number of layers, activation and loss functions for different layers, optimizer algorithm, regularization, etc. Thus, this paper aims to cover different choices available under each of these major and minor decision criteria for devising a neural network and to train it optimally for achieving the objectives effectively, e.g., malware detection, natural language processing, image recognition, etc.
27
Lawrence, Tom, Li Zhang, Kay Rogage, and Chee Peng Lim. "Evolving Deep Architecture Generation with Residual Connections for Image Classification Using Particle Swarm Optimization." Sensors 21, no. 23 (2021): 7936. http://dx.doi.org/10.3390/s21237936.
Full textAbstract:
Automated deep neural architecture generation has gained increasing attention. However, exiting studies either optimize important design choices, without taking advantage of modern strategies such as residual/dense connections, or they optimize residual/dense networks but reduce search space by eliminating fine-grained network setting choices. To address the aforementioned weaknesses, we propose a novel particle swarm optimization (PSO)-based deep architecture generation algorithm, to devise deep networks with residual connections, whilst performing a thorough search which optimizes important design choices. A PSO variant is proposed which incorporates a new encoding scheme and a new search mechanism guided by non-uniformly randomly selected neighboring and global promising solutions for the search of optimal architectures. Specifically, the proposed encoding scheme is able to describe convolutional neural network architecture configurations with residual connections. Evaluated using benchmark datasets, the proposed model outperforms existing state-of-the-art methods for architecture generation. Owing to the guidance of diverse non-uniformly selected neighboring promising solutions in combination with the swarm leader at fine-grained and global levels, the proposed model produces a rich assortment of residual architectures with great diversity. Our devised networks show better capabilities in tackling vanishing gradients with up to 4.34% improvement of mean accuracy in comparison with those of existing studies.
28
Bekhouche, Salah Eddine, Azeddine Benlamoudi, Fadi Dornaika, Hichem Telli, and Yazid Bounab. "Facial Age Estimation Using Multi-Stage Deep Neural Networks." Electronics 13, no. 16 (2024): 3259. http://dx.doi.org/10.3390/electronics13163259.
Full textAbstract:
Over the last decade, the world has witnessed many breakthroughs in artificial intelligence, largely due to advances in deep learning technology. Notably, computer vision solutions have significantly contributed to these achievements. Human face analysis, a core area of computer vision, has gained considerable attention due to its wide applicability in fields such as law enforcement, social media, and marketing. However, existing methods for facial age estimation often struggle with accuracy due to limited feature extraction capabilities and inefficiencies in learning hierarchical representations. This paper introduces a novel framework to address these issues by proposing a Multi-Stage Deep Neural Network (MSDNN) architecture. The MSDNN architecture divides each CNN backbone into multiple stages, enabling more comprehensive feature extraction, thereby improving the accuracy of age predictions from facial images. Our framework demonstrates a significant performance improvement over traditional solutions, with its effectiveness validated through comparisons with the EfficientNet and MobileNetV3 architectures. The proposed MSDNN architecture achieves a notable decrease in Mean Absolute Error (MAE) across three widely used public datasets (MORPH2, CACD, and AFAD) while maintaining a virtually identical parameter count compared to the initial backbone architectures. These results underscore the effectiveness and feasibility of our methodology in advancing the field of age estimation, showcasing it as a robust solution for enhancing the accuracy of age prediction algorithms.
29
Nossier, Soha A., Julie Wall, Mansour Moniri, Cornelius Glackin, and Nigel Cannings. "An Experimental Analysis of Deep Learning Architectures for Supervised Speech Enhancement." Electronics 10, no. 1 (2020): 17. http://dx.doi.org/10.3390/electronics10010017.
Full textAbstract:
Recent speech enhancement research has shown that deep learning techniques are very effective in removing background noise. Many deep neural networks are being proposed, showing promising results for improving overall speech perception. The Deep Multilayer Perceptron, Convolutional Neural Networks, and the Denoising Autoencoder are well-established architectures for speech enhancement; however, choosing between different deep learning models has been mainly empirical. Consequently, a comparative analysis is needed between these three architecture types in order to show the factors affecting their performance. In this paper, this analysis is presented by comparing seven deep learning models that belong to these three categories. The comparison includes evaluating the performance in terms of the overall quality of the output speech using five objective evaluation metrics and a subjective evaluation with 23 listeners; the ability to deal with challenging noise conditions; generalization ability; complexity; and, processing time. Further analysis is then provided while using two different approaches. The first approach investigates how the performance is affected by changing network hyperparameters and the structure of the data, including the Lombard effect. While the second approach interprets the results by visualizing the spectrogram of the output layer of all the investigated models, and the spectrograms of the hidden layers of the convolutional neural network architecture. Finally, a general evaluation is performed for supervised deep learning-based speech enhancement while using SWOC analysis, to discuss the technique’s Strengths, Weaknesses, Opportunities, and Challenges. The results of this paper contribute to the understanding of how different deep neural networks perform the speech enhancement task, highlight the strengths and weaknesses of each architecture, and provide recommendations for achieving better performance. This work facilitates the development of better deep neural networks for speech enhancement in the future.
30
Kosovets, Mykola, and Lilia Tovstenko. "Development of a Cluster with Cloud Computing Based on Neural Networks With Deep Learning for Modeling Multidimensional Fields." Cybernetics and Computer Technologies, no. 4 (December 30, 2021): 80–88. http://dx.doi.org/10.34229/2707-451x.21.4.8.
Full textAbstract:
Introduction. The modeling of multidimensional fields on multiprocessors, with a neural network architecture, which is rebuilt in the process of solving the problem by means of deep learning, is considered. This architecture of the calculator allows the device to be used to solve the problems of passive location, monitoring station, active LPI location station, base telecommunications station at the same time. Particular attention is paid to the use of bionic principles in the processing of multidimensional signals. A cluster computer with cloud computing is proposed for creating a modeling complex for processing multidimensional signals and debugging the target system. The cluster is made in the form of a multiprocessor based on neural network technology with deep learning. Biomimetic principles are used in the architecture of the modeling complex. The purpose of the work. Creation of a modeling complex as a cluster with cloud computing using neural networks with deep learning. The cluster is a neuromultiprocessor that is rebuilt in the process. Results. In the process, we managed to create a multiprocessor, which in the process of computing is rebuilt, to simulate a terahertz 3D Imager scanner using cloud computing. Conclusions. In the process of performing the work a complex for modeling multidimensional signals was created. As the basis of the computer used a cluster that is rebuilt in the process. The computing base consists of neural networks with cloud computing. Keywords: cognitive space, deep learning, convolutional neural network, neural network architectures, cluster.
31
Chen, Haojie, Hai Huang, Xingquan Zuo, and Xinchao Zhao. "Robustness Enhancement of Neural Networks via Architecture Search with Multi-Objective Evolutionary Optimization." Mathematics 10, no. 15 (2022): 2724. http://dx.doi.org/10.3390/math10152724.
Full textAbstract:
Along with the wide use of deep learning technology, its security issues have drawn much attention over the years. Adversarial examples expose the inherent vulnerability of deep learning models and make it a challenging task to improve their robustness. Model robustness is related not only to its parameters but also to its architecture. This paper proposes a novel robustness enhanced approach for neural networks based on a neural architecture search. First, we randomly sample multiple neural networks to construct the initial population. Second, we utilize the individual networks in the population to fit and update the surrogate models. Third, the population of neural networks is evolved through a multi-objective evolutionary algorithm, where the surrogate models accelerate the performance evaluation of networks. Finally, the second and third steps are performed alternately until a network architecture with high accuracy and robustness is achieved. Experimental results show that the proposed method outperforms some classical artificially designed neural networks and other architecture search algorithms in terms of robustness.
32
Greif, Kevin, and Kevin Lannon. "Physics Inspired Deep Neural Networks for Top Quark Reconstruction." EPJ Web of Conferences 245 (2020): 06029. http://dx.doi.org/10.1051/epjconf/202024506029.
Full textAbstract:
Deep neural networks (DNNs) have been applied to the fields of computer vision and natural language processing with great success in recent years. The success of these applications has hinged on the development of specialized DNN architectures that take advantage of specific characteristics of the problem to be solved, namely convolutional neural networks for computer vision and recurrent neural networks for natural language processing. This research explores whether a neural network architecture specific to the task of identifying t → Wb decays in particle collision data yields better performance than a generic, fully-connected DNN. Although applied here to resolved top quark decays, this approach is inspired by an DNN technique for tagging boosted top quarks, which consists of defining custom neural network layers known as the combination and Lorentz layers. These layers encode knowledge of relativistic kinematics applied to combinations of particles, and the output of these specialized layers can then be fed into a fully connected neural network to learn tasks such as classification. This research compares the performance of these physics inspired networks to that of a generic, fully-connected DNN, to see if there is any advantage in terms of classification performance, size of the network, or ease of training.
33
Herdt, Rudolf, Louisa Kinzel, Johann Georg Maaß, et al. "Enhancing the analysis of murine neonatal ultrasonic vocalizations: Development, evaluation, and application of different mathematical models." Journal of the Acoustical Society of America 156, no. 4 (2024): 2448–66. http://dx.doi.org/10.1121/10.0030473.
Full textAbstract:
Rodents employ a broad spectrum of ultrasonic vocalizations (USVs) for social communication. As these vocalizations offer valuable insights into affective states, social interactions, and developmental stages of animals, various deep learning approaches have aimed at automating both the quantitative (detection) and qualitative (classification) analysis of USVs. So far, no notable efforts have been made to determine the most suitable architecture. We present the first systematic evaluation of different types of neural networks for USV classification. We assessed various feedforward networks, including a custom-built, fully-connected network, a custom-built convolutional neural network, several residual neural networks, an EfficientNet, and a Vision Transformer. Our analysis concluded that convolutional networks with residual connections specifically adapted to USV data, are the most suitable architecture for analyzing USVs. Paired with a refined, entropy-based detection algorithm (achieving recall of 94.9 % and precision of 99.3 %), the best architecture (achieving 86.79 % accuracy) was integrated into a fully automated pipeline capable of analyzing extensive USV datasets with high reliability. In ongoing projects, our pipeline has proven to be a valuable tool in studying neonatal USVs. By comparing these distinct deep learning architectures side by side, we have established a solid foundation for future research.
34
Krishnan, Gokul, Sumit K. Mandal, Manvitha Pannala, et al. "SIAM: Chiplet-based Scalable In-Memory Acceleration with Mesh for Deep Neural Networks." ACM Transactions on Embedded Computing Systems 20, no. 5s (2021): 1–24. http://dx.doi.org/10.1145/3476999.
Full textAbstract:
In-memory computing (IMC) on a monolithic chip for deep learning faces dramatic challenges on area, yield, and on-chip interconnection cost due to the ever-increasing model sizes. 2.5D integration or chiplet-based architectures interconnect multiple small chips (i.e., chiplets) to form a large computing system, presenting a feasible solution beyond a monolithic IMC architecture to accelerate large deep learning models. This paper presents a new benchmarking simulator, SIAM, to evaluate the performance of chiplet-based IMC architectures and explore the potential of such a paradigm shift in IMC architecture design. SIAM integrates device, circuit, architecture, network-on-chip (NoC), network-on-package (NoP), and DRAM access models to realize an end-to-end system. SIAM is scalable in its support of a wide range of deep neural networks (DNNs), customizable to various network structures and configurations, and capable of efficient design space exploration. We demonstrate the flexibility, scalability, and simulation speed of SIAM by benchmarking different state-of-the-art DNNs with CIFAR-10, CIFAR-100, and ImageNet datasets. We further calibrate the simulation results with a published silicon result, SIMBA. The chiplet-based IMC architecture obtained through SIAM shows 130 and 72 improvement in energy-efficiency for ResNet-50 on the ImageNet dataset compared to Nvidia V100 and T4 GPUs.
35
Shapovalova, Svitlana, and Yurii Moskalenko. "METHODS FOR INCREASING THE CLASSIFICATION ACCURACY BASED ON MODIFICATIONS OF THE BASIC ARCHITECTURE OF CONVOLUTIONAL NEURAL NETWORKS." ScienceRise, no. 6 (December 30, 2020): 10–16. http://dx.doi.org/10.21303/2313-8416.2020.001550.
Full textAbstract:
Object of research: basic architectures of deep learning neural networks. Investigated problem: insufficient accuracy of solving the classification problem based on the basic architectures of deep learning neural networks. An increase in accuracy requires a significant complication of the architecture, which, in turn, leads to an increase in the required computing resources, as well as the consumption of video memory and the cost of learning/output time. Therefore, the problem arises of determining such methods for modifying basic architectures that improve the classification accuracy and require insignificant additional computing resources. Main scientific results: based on the analysis of existing methods for improving the classification accuracy on the convolutional networks of basic architectures, it is determined what is most effective: scaling the ScanNet architecture, learning the ensemble of TreeNet models, integrating several CBNet backbone networks. For computational experiments, these modifications of the basic architectures are implemented, as well as their combinations: ScanNet + TreeNet, ScanNet + CBNet. The effectiveness of these methods in comparison with basic architectures has been proven when solving the problem of recognizing malignant tumors with diagnostic images – SIIM-ISIC Melanoma Classification, the train/test set of which is presented on the Kaggle platform. The accuracy value for the area under the ROC curve metric has increased from 0.94489 (basic architecture network) to 0.96317 (network with ScanNet + CBNet modifications). At the same time, the output compared to the basic architecture (EfficientNet-b5) increased from 440 to 490 seconds, and the consumption of video memory increased from 8 to 9.2 gigabytes, which is acceptable. Innovative technological product: methods for achieving high recognition accuracy from a diagnostic signal based on deep learning neural networks of basic architectures. Scope of application of the innovative technological product: automatic diagnostics systems in the following areas: medicine, seismology, astronomy (classification by images) onboard control systems and systems for monitoring transport and vehicle flows or visitors (recognition of scenes with camera frames).
36
Larysa, Bogush. "FEATURES AND PROSPECTS OF THE ECONOMIC RENT FROM WORKFORCE AND SOCIAL CONDITIONS IN UKRAINE." ScienceRise 6 (December 30, 2020): 17–24. https://doi.org/10.21303/2313-8416.2020.001497.
Full textAbstract:
<strong>Object of research:</strong> basic architectures of deep learning neural networks. <strong>Investigated problem:</strong> insufficient accuracy of solving the classification problem based on the basic architectures of deep learning neural networks. An increase in accuracy requires a significant complication of the architecture, which, in turn, leads to an increase in the required computing resources, as well as the consumption of video memory and the cost of learning/output time. Therefore, the problem arises of determining such methods for modifying basic architectures that improve the classification accuracy and require insignificant additional computing resources. <strong>Main scientific results:</strong> based on the analysis of existing methods for improving the classification accuracy on the convolutional networks of basic architectures, it is determined what is most effective: scaling the ScanNet architecture, learning the ensemble of TreeNet models, integrating several CBNet backbone networks. For computational experiments, these modifications of the basic architectures are implemented, as well as their combinations: ScanNet + TreeNet, ScanNet + CBNet. The effectiveness of these methods in comparison with basic architectures has been proven when solving the problem of recognizing malignant tumors with diagnostic images – SIIM-ISIC Melanoma Classification, the train/test set of which is presented on the Kaggle platform. The accuracy value for the area under the ROC curve metric has increased from 0.94489 (basic architecture network) to 0.96317 (network with ScanNet + CBNet modifications). At the same time, the output compared to the basic architecture (EfficientNet-b5) increased from 440 to 490 seconds, and the consumption of video memory increased from 8 to 9.2 gigabytes, which is acceptable. <strong>Innovative technological product:</strong> methods for achieving high recognition accuracy from a diagnostic signal based on deep learning neural networks of basic architectures. <strong>Scope of application of the innovative technological product: </strong>automatic diagnostics systems in the following areas: medicine, seismology, astronomy (classification by images) onboard control systems and systems for monitoring transport and vehicle flows or visitors (recognition of scenes with camera frames).
37
Graziani, Salvatore, and Maria Gabriella Xibilia. "Innovative Topologies and Algorithms for Neural Networks." Future Internet 12, no. 7 (2020): 117. http://dx.doi.org/10.3390/fi12070117.
Full textAbstract:
The introduction of new topologies and training procedures to deep neural networks has solicited a renewed interest in the field of neural computation. The use of deep structures has significantly improved the state of the art in many applications, such as computer vision, speech and text processing, medical applications, and IoT (Internet of Things). The probability of a successful outcome from a neural network is linked to selection of an appropriate network architecture and training algorithm. Accordingly, much of the recent research on neural networks is devoted to the study and proposal of novel architectures, including solutions tailored to specific problems. The papers of this Special Issue make significant contributions to the above-mentioned fields by merging theoretical aspects and relevant applications. Twelve papers are collected in the issue, addressing many relevant aspects of the topic.
38
R, NETHRASHRUTHI. "AUTOMATED LUNG CANCER DETECTION USING NAS: A HIGH-PERFORMANCE DEEP LEARNING APPROACH." International Scientific Journal of Engineering and Management 04, no. 05 (2025): 1–7. https://doi.org/10.55041/isjem03443.
Full textAbstract:
Abstract – Lung cancer remains one of the leading causes of mortality worldwide, necessitating early and accurate detection for effective treatment. This study explores the application of deep learning techniques, specifically Convolutional Neural Networks (CNNs) and Neural Architecture Search (NAS), for automated lung cancer detection from CT scan images. CNNs, while effective, often require manual architecture tuning, leading to suboptimal performance. NAS, on the other hand, optimizes network architecture automatically, resulting in improved accuracy. Experimental results demonstrate that CNN achieves an accuracy of 84.38%, whereas NAS significantly outperforms it with an accuracy of 96.35%. The superior performance of NAS is attributed to its ability to discover the most efficient network structure tailored to lung cancer detection. These findings highlight the potential of automated deep learning approaches in medical image analysis, contributing to more reliable and precise diagnostic tools. Keywords: Lung Cancer Detection, Deep Learning, Neural Architecture Search (NAS), Convolutional Neural Networks (CNN).
39
Kong, Fancong, Xiaohua Wang, Kangran Pu, Jingqi Zhang, and Hua Dang. "A Practical Non-Profiled Deep-Learning-Based Power Analysis with Hybrid-Supervised Neural Networks." Electronics 12, no. 15 (2023): 3361. http://dx.doi.org/10.3390/electronics12153361.
Full textAbstract:
With the rapid advancement of deep learning, the neural network has become the primary approach for non-profiled side-channel attacks. Nevertheless, challenges arise in practical applications due to noise in collected power traces and the substantial amount of data required for training deep learning neural networks. Additionally, acquiring measuring equipment with exceptionally high sampling rates is difficult for average researchers, further obstructing the analysis process. To address these challenges, in this paper, we propose a novel architecture for non-profiled differential deep learning analysis, employing a hybrid-supervised neural network. The architecture incorporates a self-supervised autoencoder to enhance the features of power traces before they are utilized as training data for the supervised neural network. Experimental results demonstrate that the proposed architecture not only outperforms traditional differential deep learning networks by providing a more obvious distinction, but it also achieves key discrimination with reduced computational costs. Furthermore, the architecture is evaluated using small-scale and downsampled datasets, confirming its ability recover correct keys under such conditions. Moreover, the altered architecture designed for data resynchronization was proved to have the ability to distinguish the correct key from small-scale and desynchronized datasets.
40
Md Salim Chowdhury, Norun Nabi, Md Nasir Uddin Rana, et al. "Deep Learning Models for Stock Market Forecasting: A Comprehensive Comparative Analysis." Journal of Business and Management Studies 6, no. 2 (2024): 95–99. http://dx.doi.org/10.32996/jbms.2024.6.2.9.
Full textAbstract:
This study presents a comprehensive comparative analysis of deep learning models for stock market forecasting using data from two prominent stock exchanges, the National Stock Exchange (NSE) and the New York Stock Exchange (NYSE). Four deep neural network architectures—Multilayer Perceptron (MLP), Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), and Convolutional Neural Network (CNN)—were trained and tested on NSE data, focusing on Tata Motors in the automobile sector. The analysis included data from sectors such as Automobile, Banking, and IT for NSE and Financial and Petroleum sectors for NYSE. Results revealed that the deep neural network architectures consistently outperformed the traditional linear model, ARIMA, across both exchanges. The Mean Absolute Percentage Error (MAPE) values obtained for forecasting NSE values using ARIMA were notably higher compared to those derived from the neural networks, indicating the superior predictive capabilities of deep learning models. Notably, the CNN architecture demonstrated exceptional performance in capturing nonlinear trends, particularly in recognizing seasonal patterns within the data. Visualizations of predicted stock prices further supported the findings, showcasing the ability of deep learning models to adapt to dynamic market conditions and discern intricate patterns within financial time series data. Challenges encountered by different neural network architectures, such as difficulties in recognizing certain patterns within specific timeframes, were also analyzed, providing insights into the strengths and limitations of each model.
41
Koctúrová, Marianna, and Jozef Juhár. "Neural Network Architecture for EEG Based Speech Activity Detection." Acta Electrotechnica et Informatica 21, no. 4 (2021): 9–13. http://dx.doi.org/10.2478/aei-2021-0002.
Full textAbstract:
Abstract In this paper, research focused on speech activity detection using brain EEG signals is presented. In addition to speech stimulation of brain activity, an innovative approach based on the simultaneous stimulation of the brain by visual stimuli such as reading and color naming has been used. Designing the solution, classification using two types of artificial neural networks were proposed: shallow Feed-forward Neural Network and deep Convolutional Neural Network. Experimental results of classification demonstrated F1 score 79.50% speech detection using shallow neural network and 84.39% speech detection using deep neural network based on cross-evaluated classification models.
42
Hu, Jian, Xianlong Zhang, and Xiaohua Shi. "Simulating Neural Network Processors." Wireless Communications and Mobile Computing 2022 (February 23, 2022): 1–12. http://dx.doi.org/10.1155/2022/7500195.
Full textAbstract:
Deep learning has achieved competing results compared with human beings in many fields. Traditionally, deep learning networks are executed on CPUs and GPUs. In recent years, more and more neural network accelerators have been introduced in both academia and industry to improve the performance and energy efficiency for deep learning networks. In this paper, we introduce a flexible and configurable functional NN accelerator simulator, which could be configured to simulate u-architectures for different NN accelerators. The extensible and configurable simulator is helpful for system-level exploration of u-architecture, as well as operator optimization algorithm developments. The simulator is a functional simulator that simulates the latencies of calculation and memory access and the concurrent process between modules, and it gives the number of program execution cycles after the simulation is completed. We also integrated the simulator into the TVM compilation stack as an optional backend. Users can use TVM to write operators and execute them on the simulator.
43
Tripp, Bryan. "Approximating the Architecture of Visual Cortex in a Convolutional Network." Neural Computation 31, no. 8 (2019): 1551–91. http://dx.doi.org/10.1162/neco_a_01211.
Full textAbstract:
Deep convolutional neural networks (CNNs) have certain structural, mechanistic, representational, and functional parallels with primate visual cortex and also many differences. However, perhaps some of the differences can be reconciled. This study develops a cortex-like CNN architecture, via (1) a loss function that quantifies the consistency of a CNN architecture with neural data from tract tracing, cell reconstruction, and electrophysiology studies; (2) a hyperparameter-optimization approach for reducing this loss, and (3) heuristics for organizing units into convolutional-layer grids. The optimized hyperparameters are consistent with neural data. The cortex-like architecture differs from typical CNN architectures. In particular, it has longer skip connections, larger kernels and strides, and qualitatively different connection sparsity. Importantly, layers of the cortex-like network have one-to-one correspondences with cortical neuron populations. This should allow unambiguous comparison of model and brain representations in the future and, consequently, more precise measurement of progress toward more biologically realistic deep networks.
44
Jawad, Eman. "THE DEEP NEURAL NETWORK-A REVIEW." IJRDO -JOURNAL OF MATHEMATICS 9, no. 9 (2023): 1–5. http://dx.doi.org/10.53555/m.v9i9.5842.
Full textAbstract:
Deep neural networks are considered the backbone of artificial intelligence, we will present a review of an article about the importance of neural networks and their role in other sciences, their characteristic, networks architecture, types, mathematical definition of deep neural networks, as well as their applications.
45
Ling, Julia, Andrew Kurzawski, and Jeremy Templeton. "Reynolds averaged turbulence modelling using deep neural networks with embedded invariance." Journal of Fluid Mechanics 807 (October 18, 2016): 155–66. http://dx.doi.org/10.1017/jfm.2016.615.
Full textAbstract:
There exists significant demand for improved Reynolds-averaged Navier–Stokes (RANS) turbulence models that are informed by and can represent a richer set of turbulence physics. This paper presents a method of using deep neural networks to learn a model for the Reynolds stress anisotropy tensor from high-fidelity simulation data. A novel neural network architecture is proposed which uses a multiplicative layer with an invariant tensor basis to embed Galilean invariance into the predicted anisotropy tensor. It is demonstrated that this neural network architecture provides improved prediction accuracy compared with a generic neural network architecture that does not embed this invariance property. The Reynolds stress anisotropy predictions of this invariant neural network are propagated through to the velocity field for two test cases. For both test cases, significant improvement versus baseline RANS linear eddy viscosity and nonlinear eddy viscosity models is demonstrated.
46
Ahna, R., Ameena Nowshad, S. Fousiya, Marwa, Anisha Thomas, and G. S. Anju. "Deep Neural Architecture for Phishing Website Identification." International Journal of Recent Advances in Multidisciplinary Topics 5, no. 5 (2024): 63–66. https://doi.org/10.5281/zenodo.11192819.
Full textAbstract:
Phishing attacks remain a prevalent threat in the digital age, tricking users into surrendering sensitive information through fraudulent websites. Expand more Traditional machine learning approaches for phishing detection often rely on manually extracted features, which can be time-consuming and ineffective against evolving attack strategies. This paper proposes a novel deep learning framework for real-time phishing website detection utilizing Convolutional Neural Networks (CNNs) and Bidirectional Long Short-Term Memory (BiLSTM) networks. Expand more by leveraging the strengths of CNNs in feature extraction and BiLSTM networks in capturing sequential information, our framework aims to achieve superior accuracy and robustness in identifying phishing websites. Additionally, we present a web application built with the Python Django framework that allows users to submit website URLs for real-time analysis using the pre-trained deep learning models. This user-friendly application offers real-time phishing detection with informative probability scores, enhancing user security and awareness.
47
Benbatata, Sabrina, Bilal Saoud, Ibraheem Shayea, et al. "A novel deep neural network-based technique for network embedding." PeerJ Computer Science 10 (November 26, 2024): e2489. http://dx.doi.org/10.7717/peerj-cs.2489.
Full textAbstract:
In this paper, the graph segmentation (GSeg) method has been proposed. This solution is a novel graph neural network framework for network embedding that leverages the inherent characteristics of nodes and the underlying local network topology. The key innovation of GSeg lies in its encoder-decoder architecture, which is specifically designed to preserve the network’s structural properties. The key contributions of GSeg are: (1) a novel graph neural network architecture that effectively captures local and global network structures, and (2) a robust node representation learning approach that achieves superior performance in various network analysis tasks. The methodology employed in our study involves the utilization of a graph neural network framework for the acquisition of node representations. The design leverages the inherent characteristics of nodes and the underlying local network topology. To enhance the architectural framework of encoder- decoder networks, the GSeg model is specifically devised to exhibit a structural resemblance to the SegNet model. The obtained empirical results on multiple benchmark datasets demonstrate that the GSeg outperforms existing state-of-the-art methods in terms of network structure preservation and prediction accuracy for downstream tasks. The proposed technique has potential utility across a range of practical applications in the real world.
48
Pepe, Giovanni, Leonardo Gabrielli, Stefano Squartini, and Luca Cattani. "Designing Audio Equalization Filters by Deep Neural Networks." Applied Sciences 10, no. 7 (2020): 2483. http://dx.doi.org/10.3390/app10072483.
Full textAbstract:
Audio equalization is an active research topic aiming at improving the audio quality of a loudspeaker system by correcting the overall frequency response using linear filters. The estimation of their coefficients is not an easy task, especially in binaural and multipoint scenarios, due to the contribution of multiple impulse responses to each listening point. This paper presents a deep learning approach for tuning filter coefficients employing three different neural networks architectures—the Multilayer Perceptron, the Convolutional Neural Network, and the Convolutional Autoencoder. Suitable loss functions are proposed for each architecture, and are formulated in terms of spectral Euclidean distance. The experiments were conducted in the automotive scenario, considering several loudspeakers and microphones. The obtained results show that deep learning techniques give superior performance compared to baseline methods, achieving almost flat magnitude frequency response.
49
Anik, Shafayat Mowla, Kevyn Kelso, and Byeong Kil Lee. "Efficient Layer Optimizations for Deep Neural Networks." International Journal of Soft Computing and Engineering 14, no. 5 (2024): 20–29. http://dx.doi.org/10.35940/ijsce.e3650.14051124.
Full textAbstract:
Deep neural networks (DNNs) have technical issues such as long training time as the network size increases. Parameters require significant memory, which may cause migration issues for embedded devices. DNNs applied various pruning techniques to reduce the network size in deep neural networks, but many problems still exist when applying the pruning techniques. Among neural networks, several applications applied autoencoders for reconstruction and dimension reduction. However, network size is a disadvantage of autoencoders since the architecture of the autoencoders has a double workload due to the encoding and decoding processes. In this research, we chose autoencoders and two deep neural networks AlexNet and VGG16 to apply out of order layer pruning. We perform the sensitivity analysis to explore the performance variations for the network architecture and network complexity through an out of order layer pruning mechanism. As a result of applying the proposed layer pruning scheme to the autoencoder, we developed the accordion autoencoder (A2E) and applied credit card fraud detection and MNIST classification. Our results show 4.9 Percent and 13.6 Percent performance drops, respectively, but we observe a significant reduction in network complexity, 85.1 Percent and 94.5 Percent for each application. We extend the out of order layer pruning to deeper learning networks. In our approach, we propose a simple yet efficient scheme, accuracy aware structured filter pruning based on the characterization of each convolutional layer combined with the quantization of fully connected layers. We investigate the accuracy and compression rate of each layer using a fixed pruning ratio, and then the pruning priority is rearranged depending on the accuracy of each layer. Our analysis of layer characterization shows that the pruning order of the layers does affect the final accuracy of the deep neural network. Based on our experiments using the proposed pruning scheme, the parameter size in the AlexNet can be up to 47.28x smaller than the original model. We also obtained comparable results for VGG16, achieving a maximum compression rate of 35.21x.
50
Byeong, Kil Lee. "Efficient Layer Optimizations for Deep Neural Networks." International Journal of Soft Computing and Engineering (IJSCE) 14, no. 5 (2024): 20–29. https://doi.org/10.35940/ijsce.E3650.14051124.
Full textAbstract:
<strong>Abstract:</strong> Deep neural networks (DNNs) have technical issues such as long training time as the network size increases. Parameters require significant memory, which may cause migration issues for embedded devices. DNNs applied various pruning techniques to reduce the network size in deep neural networks, but many problems still exist when applying the pruning techniques. Among neural networks, several applications applied autoencoders for reconstruction and dimension reduction. However, network size is a disadvantage of autoencoders since the architecture of the autoencoders has a double workload due to the encoding and decoding processes. In this research, we chose autoencoders and two deep neural networks – AlexNet and VGG16 to apply out-of-order layer pruning. We perform the sensitivity analysis to explore the performance variations for the network architecture and network complexity through an out-of-order layer pruning mechanism. As a result of applying the proposed layer pruning scheme to the autoencoder, we developed the accordion autoencoder (A2E) and applied credit card fraud detection and MNIST classification. Our results show 4.9% and 13.6% performance drops, respectively, but we observe a significant reduction in network complexity, 85.1% and 94.5% for each application. We extend the out-of-order layer pruning to deeper learning networks. In our approach, we propose a simple yet efficient scheme, accuracy-aware structured filter pruning based on the characterization of each convolutional layer combined with the quantization of fully connected layers. We investigate the accuracy and compression rate of each layer using a fixed pruning ratio, and then the pruning priority is rearranged depending on the accuracy of each layer. Our analysis of layer characterization shows that the pruning order of the layers does affect the final accuracy of the deep neural network. Based on our experiments using the proposed pruning scheme, the parameter size in the AlexNet can be up to 47.28x smaller than the original model. We also obtained comparable results for VGG16, achieving a maximum compression rate of 35.21x.