Journal articles: 'Sparse deep neural networks'

1

Scardapane, Simone, Danilo Comminiello, Amir Hussain, and Aurelio Uncini. "Group sparse regularization for deep neural networks." Neurocomputing 241 (June 2017): 81–89. http://dx.doi.org/10.1016/j.neucom.2017.02.029.

Full text

APA, Harvard, Vancouver, ISO, and other styles

2

Zang, Ke, Wenqi Wu, and Wei Luo. "Deep Sparse Learning for Automatic Modulation Classification Using Recurrent Neural Networks." Sensors 21, no. 19 (September 25, 2021): 6410. http://dx.doi.org/10.3390/s21196410.

Full text

Abstract:

Deep learning models, especially recurrent neural networks (RNNs), have been successfully applied to automatic modulation classification (AMC) problems recently. However, deep neural networks are usually overparameterized, i.e., most of the connections between neurons are redundant. The large model size hinders the deployment of deep neural networks in applications such as Internet-of-Things (IoT) networks. Therefore, reducing parameters without compromising the network performance via sparse learning is often desirable since it can alleviates the computational and storage burdens of deep learning models. In this paper, we propose a sparse learning algorithm that can directly train a sparsely connected neural network based on the statistics of weight magnitude and gradient momentum. We first used the MNIST and CIFAR10 datasets to demonstrate the effectiveness of this method. Subsequently, we applied it to RNNs with different pruning strategies on recurrent and non-recurrent connections for AMC problems. Experimental results demonstrated that the proposed method can effectively reduce the parameters of the neural networks while maintaining model performance. Moreover, we show that appropriate sparsity can further improve network generalization ability.

APA, Harvard, Vancouver, ISO, and other styles

3

Wu, Kailun, Yiwen Guo, and Changshui Zhang. "Compressing Deep Neural Networks With Sparse Matrix Factorization." IEEE Transactions on Neural Networks and Learning Systems 31, no. 10 (October 2020): 3828–38. http://dx.doi.org/10.1109/tnnls.2019.2946636.

Full text

APA, Harvard, Vancouver, ISO, and other styles

4

Gangopadhyay, Briti, Pallab Dasgupta, and Soumyajit Dey. "Safety Aware Neural Pruning for Deep Reinforcement Learning (Student Abstract)." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 13 (June 26, 2023): 16212–13. http://dx.doi.org/10.1609/aaai.v37i13.26966.

Full text

Abstract:

Neural network pruning is a technique of network compression by removing weights of lower importance from an optimized neural network. Often, pruned networks are compared in terms of accuracy, which is realized in terms of rewards for Deep Reinforcement Learning (DRL) networks. However, networks that estimate control actions for safety-critical tasks, must also adhere to safety requirements along with obtaining rewards. We propose a methodology to iteratively refine the weights of a pruned neural network such that we get a sparse high-performance network without significant side effects on safety.

APA, Harvard, Vancouver, ISO, and other styles

5

Petschenig, Horst, and Robert Legenstein. "Quantized rewiring: hardware-aware training of sparse deep neural networks." Neuromorphic Computing and Engineering 3, no. 2 (May 26, 2023): 024006. http://dx.doi.org/10.1088/2634-4386/accd8f.

Full text

Abstract:

Abstract Mixed-signal and fully digital neuromorphic systems have been of significant interest for deploying spiking neural networks in an energy-efficient manner. However, many of these systems impose constraints in terms of fan-in, memory, or synaptic weight precision that have to be considered during network design and training. In this paper, we present quantized rewiring (Q-rewiring), an algorithm that can train both spiking and non-spiking neural networks while meeting hardware constraints during the entire training process. To demonstrate our approach, we train both feedforward and recurrent neural networks with a combined fan-in/weight precision limit, a constraint that is, for example, present in the DYNAP-SE mixed-signal analog/digital neuromorphic processor. Q-rewiring simultaneously performs quantization and rewiring of synapses and synaptic weights through gradient descent updates and projecting the trainable parameters to a constraint-compliant region. Using our algorithm, we find trade-offs between the number of incoming connections to neurons and network performance for a number of common benchmark datasets.

APA, Harvard, Vancouver, ISO, and other styles

6

Belay, Kaleab. "Gradient and Mangitude Based Pruning for Sparse Deep Neural Networks." Proceedings of the AAAI Conference on Artificial Intelligence 36, no. 11 (June 28, 2022): 13126–27. http://dx.doi.org/10.1609/aaai.v36i11.21699.

Full text

Abstract:

Deep Neural Networks have memory and computational demands that often render them difficult to use in low-resource environments. Also, highly dense networks are over-parameterized and thus prone to overfitting. To address these problems, we introduce a novel algorithm that prunes (sparsifies) weights from the network by taking into account their magnitudes and gradients taken against a validation dataset. Unlike existing pruning methods, our method does not require the network model to be retrained once initial training is completed. On the CIFAR-10 dataset, our method reduced the number of paramters of MobileNet by a factor of 9X, from 14 million to 1.5 million, with just a 3.8% drop in accuracy.

APA, Harvard, Vancouver, ISO, and other styles

7

Kaur, Mandeep, and Pradip Kumar Yadava. "A Review on Classification of Images with Convolutional Neural Networks." International Journal for Research in Applied Science and Engineering Technology 11, no. 7 (July 31, 2023): 658–63. http://dx.doi.org/10.22214/ijraset.2023.54704.

Full text

Abstract:

Abstract: Deep learning has recently been applied to scene labelling, object tracking, pose estimation, text detection and recognition, visual saliency detection, and image categorization. Deep learning typically uses models like Auto Encoder, Sparse Coding, Restricted Boltzmann Machine, Deep Belief Networks, and Convolutional Neural Networks. Convolutional neural networks have exhibited good performance in picture categorization when compared to other types of models. A straightforward Convolutional neural network for image categorization was built in this paper. The image classification was finished by this straightforward Convolutional neural network. On the foundation of the Convolutional neural network, we also examined several learning rate setting techniques and different optimisation algorithms for determining the ideal parameters that have the greatest influence on image categorization

APA, Harvard, Vancouver, ISO, and other styles

8

Bi, Jia, and Steve R. Gunn. "Sparse Deep Neural Network Optimization for Embedded Intelligence." International Journal on Artificial Intelligence Tools 29, no. 03n04 (June 2020): 2060002. http://dx.doi.org/10.1142/s0218213020600027.

Full text

Abstract:

Deep neural networks become more popular as its ability to solve very complex pattern recognition problems. However, deep neural networks often need massive computational and memory resources, which is main reason resulting them to be difficult efficiently and entirely running on embedded platforms. This work addresses this problem by saving the computational and memory requirements of deep neural networks by proposing a variance reduced (VR)-based optimization with regularization techniques to compress the requirements of memory of models within fast training process. It is shown theoretically and experimentally that sparsity-inducing regularization can be effectively worked with the VR-based optimization whereby in the optimizer the behaviors of the stochastic element is controlled by a hyper-parameter to solve non-convex problems.

APA, Harvard, Vancouver, ISO, and other styles

9

Gallicchio, Claudio, and Alessio Micheli. "Fast and Deep Graph Neural Networks." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 3898–905. http://dx.doi.org/10.1609/aaai.v34i04.5803.

Full text

Abstract:

We address the efficiency issue for the construction of a deep graph neural network (GNN). The approach exploits the idea of representing each input graph as a fixed point of a dynamical system (implemented through a recurrent neural network), and leverages a deep architectural organization of the recurrent units. Efficiency is gained by many aspects, including the use of small and very sparse networks, where the weights of the recurrent units are left untrained under the stability condition introduced in this work. This can be viewed as a way to study the intrinsic power of the architecture of a deep GNN, and also to provide insights for the set-up of more complex fully-trained models. Through experimental results, we show that even without training of the recurrent connections, the architecture of small deep GNN is surprisingly able to achieve or improve the state-of-the-art performance on a significant set of tasks in the field of graphs classification.

APA, Harvard, Vancouver, ISO, and other styles

10

Tartaglione, Enzo, Andrea Bragagnolo, Attilio Fiandrotti, and Marco Grangetto. "LOss-Based SensiTivity rEgulaRization: Towards deep sparse neural networks." Neural Networks 146 (February 2022): 230–37. http://dx.doi.org/10.1016/j.neunet.2021.11.029.

Full text

APA, Harvard, Vancouver, ISO, and other styles

11

Ma, Rongrong, Jianyu Miao, Lingfeng Niu, and Peng Zhang. "Transformed ℓ1 regularization for learning sparse deep neural networks." Neural Networks 119 (November 2019): 286–98. http://dx.doi.org/10.1016/j.neunet.2019.08.015.

Full text

APA, Harvard, Vancouver, ISO, and other styles

12

Zhao, Jin, and Licheng Jiao. "Fast Sparse Deep Neural Networks: Theory and Performance Analysis." IEEE Access 7 (2019): 74040–55. http://dx.doi.org/10.1109/access.2019.2920688.

Full text

APA, Harvard, Vancouver, ISO, and other styles

13

Karim, Ahmad M., Mehmet S. Güzel, Mehmet R. Tolun, Hilal Kaya, and Fatih V. Çelebi. "A New Generalized Deep Learning Framework Combining Sparse Autoencoder and Taguchi Method for Novel Data Classification and Processing." Mathematical Problems in Engineering 2018 (June 7, 2018): 1–13. http://dx.doi.org/10.1155/2018/3145947.

Full text

Abstract:

Deep autoencoder neural networks have been widely used in several image classification and recognition problems, including hand-writing recognition, medical imaging, and face recognition. The overall performance of deep autoencoder neural networks mainly depends on the number of parameters used, structure of neural networks, and the compatibility of the transfer functions. However, an inappropriate structure design can cause a reduction in the performance of deep autoencoder neural networks. A novel framework, which primarily integrates the Taguchi Method to a deep autoencoder based system without considering to modify the overall structure of the network, is presented. Several experiments are performed using various data sets from different fields, i.e., network security and medicine. The results show that the proposed method is more robust than some of the well-known methods in the literature as most of the time our method performed better. Therefore, the results are quite encouraging and verified the overall performance of the proposed framework.

APA, Harvard, Vancouver, ISO, and other styles

14

Li, Yihang. "Sparse-Aware Deep Learning Accelerator." Highlights in Science, Engineering and Technology 39 (April 1, 2023): 305–10. http://dx.doi.org/10.54097/hset.v39i.6544.

Full text

Abstract:

In view of the difficulty of hardware implementation of convolutional neural network computing, most of the previous convolutional neural network accelerator designs focused on solving the bottleneck of computational performance and bandwidth, ignoring the importance of convolutional neural network scarcity for accelerator design. In recent years, there are a few convolutional neural network accelerators that can take advantage of the scarcity, but they are usually difficult to consider in terms of computational flexibility, parallel efficiency and resource overhead. In view of the problem that the application of convolutional neural network (CNN) on the embedded side is limited by real-time, and there is a large degree of sparsity in CNN convolution calculation. This paper summarizes the methods of sparsification from the algorithm level and based on FPGA level. The different methods of sparsification and the research and analysis of different application layers are introduced. The advantages and development trend of sparsification are analyzed and summarized.

APA, Harvard, Vancouver, ISO, and other styles

15

Ohn, Ilsang, and Yongdai Kim. "Nonconvex Sparse Regularization for Deep Neural Networks and Its Optimality." Neural Computation 34, no. 2 (January 14, 2022): 476–517. http://dx.doi.org/10.1162/neco_a_01457.

Full text

Abstract:

Abstract Recent theoretical studies proved that deep neural network (DNN) estimators obtained by minimizing empirical risk with a certain sparsity constraint can attain optimal convergence rates for regression and classification problems. However, the sparsity constraint requires knowing certain properties of the true model, which are not available in practice. Moreover, computation is difficult due to the discrete nature of the sparsity constraint. In this letter, we propose a novel penalized estimation method for sparse DNNs that resolves the problems existing in the sparsity constraint. We establish an oracle inequality for the excess risk of the proposed sparse-penalized DNN estimator and derive convergence rates for several learning tasks. In particular, we prove that the sparse-penalized estimator can adaptively attain minimax convergence rates for various nonparametric regression problems. For computation, we develop an efficient gradient-based optimization algorithm that guarantees the monotonic reduction of the objective function.

APA, Harvard, Vancouver, ISO, and other styles

16

Avgerinos, Christos, Nicholas Vretos, and Petros Daras. "Less Is More: Adaptive Trainable Gradient Dropout for Deep Neural Networks." Sensors 23, no. 3 (January 24, 2023): 1325. http://dx.doi.org/10.3390/s23031325.

Full text

Abstract:

The undeniable computational power of artificial neural networks has granted the scientific community the ability to exploit the available data in ways previously inconceivable. However, deep neural networks require an overwhelming quantity of data in order to interpret the underlying connections between them, and therefore, be able to complete the specific task that they have been assigned to. Feeding a deep neural network with vast amounts of data usually ensures efficiency, but may, however, harm the network’s ability to generalize. To tackle this, numerous regularization techniques have been proposed, with dropout being one of the most dominant. This paper proposes a selective gradient dropout method, which, instead of relying on dropping random weights, learns to freeze the training process of specific connections, thereby increasing the overall network’s sparsity in an adaptive manner, by driving it to utilize more salient weights. The experimental results show that the produced sparse network outperforms the baseline on numerous image classification datasets, and additionally, the yielded results occurred after significantly less training epochs.

APA, Harvard, Vancouver, ISO, and other styles

17

Hao, Yutong, Yunpeng Liu, Jinmiao Zhao, and Chuang Yu. "Dual-Domain Prior-Driven Deep Network for Infrared Small-Target Detection." Remote Sensing 15, no. 15 (July 31, 2023): 3827. http://dx.doi.org/10.3390/rs15153827.

Full text

Abstract:

In recent years, data-driven deep networks have demonstrated remarkable detection performance for infrared small targets. However, continuously increasing the depth of neural networks to enhance performance has proven impractical. Consequently, the integration of prior physical knowledge related to infrared small targets within deep neural networks has become crucial. It aims to improve the models’ awareness of inherent physical characteristics. In this paper, we propose a novel dual-domain prior-driven deep network (DPDNet) for infrared small-target detection. Our method integrates the advantages of both data-driven and model-driven methods by leveraging the prior physical characteristics as the driving force. Initially, we utilize the sparse characteristics of infrared small targets to boost their saliency at the input level of the network. Subsequently, a high-frequency feature extraction module, seamlessly integrated into the network’s backbone, is employed to excavate feature information. DPDNet simultaneously emphasizes the prior sparse characteristics of infrared small targets in the spatial domain and their prior high-frequency characteristics in the frequency domain. Compared with previous CNN-based methods, our method achieves superior performance while utilizing fewer convolutional layers. It has a performance of 78.64% IoU, 95.56 Pd, and 2.15 × 10−6 Fa on the SIRST dataset.

APA, Harvard, Vancouver, ISO, and other styles

18

Lee, Sangkyun, and Jeonghyun Lee. "Compressed Learning of Deep Neural Networks for OpenCL-Capable Embedded Systems." Applied Sciences 9, no. 8 (April 23, 2019): 1669. http://dx.doi.org/10.3390/app9081669.

Full text

Abstract:

Deep neural networks (DNNs) have been quite successful in solving many complex learning problems. However, DNNs tend to have a large number of learning parameters, leading to a large memory and computation requirement. In this paper, we propose a model compression framework for efficient training and inference of deep neural networks on embedded systems. Our framework provides data structures and kernels for OpenCL-based parallel forward and backward computation in a compressed form. In particular, our method learns sparse representations of parameters using ℓ 1 -based sparse coding while training, storing them in compressed sparse matrices. Unlike the previous works, our method does not require a pre-trained model as an input and therefore can be more versatile for different application environments. Even though the use of ℓ 1 -based sparse coding for model compression is not new, we show that it can be far more effective than previously reported when we use proximal point algorithms and the technique of debiasing. Our experiments show that our method can produce minimal learning models suitable for small embedded devices.

APA, Harvard, Vancouver, ISO, and other styles

19

Mousavi, Hamid, Mohammad Loni, Mina Alibeigi, and Masoud Daneshtalab. "DASS: Differentiable Architecture Search for Sparse Neural Networks." ACM Transactions on Embedded Computing Systems 22, no. 5s (September 9, 2023): 1–21. http://dx.doi.org/10.1145/3609385.

Full text

Abstract:

The deployment of Deep Neural Networks (DNNs) on edge devices is hindered by the substantial gap between performance requirements and available computational power. While recent research has made significant strides in developing pruning methods to build a sparse network for reducing the computing overhead of DNNs, there remains considerable accuracy loss, especially at high pruning ratios. We find that the architectures designed for dense networks by differentiable architecture search methods are ineffective when pruning mechanisms are applied to them. The main reason is that the current methods do not support sparse architectures in their search space and use a search objective that is made for dense networks and does not focus on sparsity. This paper proposes a new method to search for sparsity-friendly neural architectures. It is done by adding two new sparse operations to the search space and modifying the search objective. We propose two novel parametric SparseConv and SparseLinear operations in order to expand the search space to include sparse operations. In particular, these operations make a flexible search space due to using sparse parametric versions of linear and convolution operations. The proposed search objective lets us train the architecture based on the sparsity of the search space operations. Quantitative analyses demonstrate that architectures found through DASS outperform those used in the state-of-the-art sparse networks on the CIFAR-10 and ImageNet datasets. In terms of performance and hardware effectiveness, DASS increases the accuracy of the sparse version of MobileNet-v2 from 73.44% to 81.35% (+7.91% improvement) with a 3.87× faster inference time.

APA, Harvard, Vancouver, ISO, and other styles

20

Ao, Ren, Zhang Tao, Wang Yuhao, Lin Sheng, Dong Peiyan, Chen Yen-kuang, Xie Yuan, and Wang Yanzhi. "DARB: A Density-Adaptive Regular-Block Pruning for Deep Neural Networks." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 5495–502. http://dx.doi.org/10.1609/aaai.v34i04.6000.

Full text

Abstract:

The rapidly growing parameter volume of deep neural networks (DNNs) hinders the artificial intelligence applications on resource constrained devices, such as mobile and wearable devices. Neural network pruning, as one of the mainstream model compression techniques, is under extensive study to reduce the model size and thus the amount of computation. And thereby, the state-of-the-art DNNs are able to be deployed on those devices with high runtime energy efficiency. In contrast to irregular pruning that incurs high index storage and decoding overhead, structured pruning techniques have been proposed as the promising solutions. However, prior studies on structured pruning tackle the problem mainly from the perspective of facilitating hardware implementation, without diving into the deep to analyze the characteristics of sparse neural networks. The neglect on the study of sparse neural networks causes inefficient trade-off between regularity and pruning ratio. Consequently, the potential of structurally pruning neural networks is not sufficiently mined.In this work, we examine the structural characteristics of the irregularly pruned weight matrices, such as the diverse redundancy of different rows, the sensitivity of different rows to pruning, and the position characteristics of retained weights. By leveraging the gained insights as a guidance, we first propose the novel block-max weight masking (BMWM) method, which can effectively retain the salient weights while imposing high regularity to the weight matrix. As a further optimization, we propose a density-adaptive regular-block (DARB) pruning that can effectively take advantage of the intrinsic characteristics of neural networks, and thereby outperform prior structured pruning work with high pruning ratio and decoding efficiency. Our experimental results show that DARB can achieve 13× to 25× pruning ratio, which are 2.8× to 4.3× improvements than the state-of-the-art counterparts on multiple neural network models and tasks. Moreover, DARB can achieve 14.3× decoding efficiency than block pruning with higher pruning ratio.

APA, Harvard, Vancouver, ISO, and other styles

21

Östling, Robert. "Part of Speech Tagging: Shallow or Deep Learning?" Northern European Journal of Language Technology 5 (June 19, 2018): 1–15. http://dx.doi.org/10.3384/nejlt.2000-1533.1851.

Full text

Abstract:

Deep neural networks have advanced the state of the art in numerous fields, but they generally suffer from low computational efficiency and the level of improvement compared to more efficient machine learning models is not always significant. We perform a thorough PoS tagging evaluation on the Universal Dependencies treebanks, pitting a state-of-the-art neural network approach against UDPipe and our sparse structured perceptron-based tagger, efselab. In terms of computational efficiency, efselab is three orders of magnitude faster than the neural network model, while being more accurate than either of the other systems on 47 of 65 treebanks.

APA, Harvard, Vancouver, ISO, and other styles

22

Gong, Maoguo, Jia Liu, Hao Li, Qing Cai, and Linzhi Su. "A Multiobjective Sparse Feature Learning Model for Deep Neural Networks." IEEE Transactions on Neural Networks and Learning Systems 26, no. 12 (December 2015): 3263–77. http://dx.doi.org/10.1109/tnnls.2015.2469673.

Full text

APA, Harvard, Vancouver, ISO, and other styles

23

Boo, Yoonho, and Wonyong Sung. "Compression of Deep Neural Networks with Structured Sparse Ternary Coding." Journal of Signal Processing Systems 91, no. 9 (November 6, 2018): 1009–19. http://dx.doi.org/10.1007/s11265-018-1418-z.

Full text

APA, Harvard, Vancouver, ISO, and other styles

24

Zhao, Yao, Qingsong Liu, He Tian, Bingo Wing-Kuen Ling, and Zhe Zhang. "DeepRED Based Sparse SAR Imaging." Remote Sensing 16, no. 2 (January 5, 2024): 212. http://dx.doi.org/10.3390/rs16020212.

Full text

Abstract:

The integration of deep neural networks into sparse synthetic aperture radar (SAR) imaging is explored to enhance SAR imaging performance and reduce the system’s sampling rate. However, the scarcity of training samples and mismatches between the training data and the SAR system pose significant challenges to the method’s further development. In this paper, we propose a novel SAR imaging approach based on deep image prior powered by RED (DeepRED), enabling unsupervised SAR imaging without the need for additional training data. Initially, DeepRED is introduced as the regularization technique within the sparse SAR imaging model. Subsequently, variable splitting and the alternating direction method of multipliers (ADMM) are employed to solve the imaging model, alternately updating the magnitude and phase of the SAR image. Additionally, the SAR echo simulation operator is utilized as an observation model to enhance computational efficiency. Through simulations and real data experiments, we demonstrate that our method maintains imaging quality and system downsampling rate on par with deep-neural-network-based sparse SAR imaging but without the requirement for training data.

APA, Harvard, Vancouver, ISO, and other styles

25

Wan, Xinyue, Bofeng Zhang, Guobing Zou, and Furong Chang. "Sparse Data Recommendation by Fusing Continuous Imputation Denoising Autoencoder and Neural Matrix Factorization." Applied Sciences 9, no. 1 (December 24, 2018): 54. http://dx.doi.org/10.3390/app9010054.

Full text

Abstract:

In recent years, although deep neural networks have yielded immense success in solving various recognition and classification problems, the exploration of deep neural networks in recommender systems has received relatively less attention. Meanwhile, the inherent sparsity of data is still a challenging problem for deep neural networks. In this paper, firstly, we propose a new CIDAE (Continuous Imputation Denoising Autoencoder) model based on the Denoising Autoencoder to alleviate the problem of data sparsity. CIDAE performs regular continuous imputation on the missing parts of the original data and trains the imputed data as the desired output. Then, we optimize the existing advanced NeuMF (Neural Matrix Factorization) model, which combines matrix factorization and a multi-layer perceptron. By optimizing the training process of NeuMF, we improve the accuracy and robustness of NeuMF. Finally, this paper fuses CIDAE and optimized NeuMF with reference to the idea of ensemble learning. We name the fused model the I-NMF (Imputation-Neural Matrix Factorization) model. I-NMF can not only alleviate the problem of data sparsity, but also fully exploit the ability of deep neural networks to learn potential features. Our experimental results prove that I-NMF performs better than the state-of-the-art methods for the public MovieLens datasets.

APA, Harvard, Vancouver, ISO, and other styles

26

El-Yabroudi, Mohammad Z., Ikhlas Abdel-Qader, Bradley J. Bazuin, Osama Abudayyeh, and Rakan C. Chabaan. "Guided Depth Completion with Instance Segmentation Fusion in Autonomous Driving Applications." Sensors 22, no. 24 (December 7, 2022): 9578. http://dx.doi.org/10.3390/s22249578.

Full text

Abstract:

Pixel-level depth information is crucial to many applications, such as autonomous driving, robotics navigation, 3D scene reconstruction, and augmented reality. However, depth information, which is usually acquired by sensors such as LiDAR, is sparse. Depth completion is a process that predicts missing pixels’ depth information from a set of sparse depth measurements. Most of the ongoing research applies deep neural networks on the entire sparse depth map and camera scene without utilizing any information about the available objects, which results in more complex and resource-demanding networks. In this work, we propose to use image instance segmentation to detect objects of interest with pixel-level locations, along with sparse depth data, to support depth completion. The framework utilizes a two-branch encoder–decoder deep neural network. It fuses information about scene available objects, such as objects’ type and pixel-level location, LiDAR, and RGB camera, to predict dense accurate depth maps. Experimental results on the KITTI dataset showed faster training and improved prediction accuracy. The proposed method reaches a convergence state faster and surpasses the baseline model in all evaluation metrics.

APA, Harvard, Vancouver, ISO, and other styles

27

Qiao, Chen, Yan Shi, Yu-Xian Diao, Vince D. Calhoun, and Yu-Ping Wang. "Log-sum enhanced sparse deep neural network." Neurocomputing 407 (September 2020): 206–20. http://dx.doi.org/10.1016/j.neucom.2020.04.118.

Full text

APA, Harvard, Vancouver, ISO, and other styles

28

Morotti, Elena, Davide Evangelista, and Elena Loli Piccolomini. "A Green Prospective for Learned Post-Processing in Sparse-View Tomographic Reconstruction." Journal of Imaging 7, no. 8 (August 7, 2021): 139. http://dx.doi.org/10.3390/jimaging7080139.

Full text

Abstract:

Deep Learning is developing interesting tools that are of great interest for inverse imaging applications. In this work, we consider a medical imaging reconstruction task from subsampled measurements, which is an active research field where Convolutional Neural Networks have already revealed their great potential. However, the commonly used architectures are very deep and, hence, prone to overfitting and unfeasible for clinical usages. Inspired by the ideas of the green AI literature, we propose a shallow neural network to perform efficient Learned Post-Processing on images roughly reconstructed by the filtered backprojection algorithm. The results show that the proposed inexpensive network computes images of comparable (or even higher) quality in about one-fourth of time and is more robust than the widely used and very deep ResUNet for tomographic reconstructions from sparse-view protocols.

APA, Harvard, Vancouver, ISO, and other styles

29

Wan, Lulu, Tao Chen, Antonio Plaza, and Haojie Cai. "Hyperspectral Unmixing Based on Spectral and Sparse Deep Convolutional Neural Networks." IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 14 (2021): 11669–82. http://dx.doi.org/10.1109/jstars.2021.3126755.

Full text

APA, Harvard, Vancouver, ISO, and other styles

30

Khattak, Muhammad Irfan, Nasir Saleem, Jiechao Gao, Elena Verdu, and Javier Parra Fuente. "Regularized sparse features for noisy speech enhancement using deep neural networks." Computers and Electrical Engineering 100 (May 2022): 107887. http://dx.doi.org/10.1016/j.compeleceng.2022.107887.

Full text

APA, Harvard, Vancouver, ISO, and other styles

31

Xie, Zhihua, Yi Li, Jieyi Niu, Ling Shi, Zhipeng Wang, and Guoyu Lu. "Hyperspectral face recognition based on sparse spectral attention deep neural networks." Optics Express 28, no. 24 (November 16, 2020): 36286. http://dx.doi.org/10.1364/oe.404793.

Full text

APA, Harvard, Vancouver, ISO, and other styles

32

Liu, Wei, Yue Yang, and Longsheng Wei. "Weather Recognition of Street Scene Based on Sparse Deep Neural Networks." Journal of Advanced Computational Intelligence and Intelligent Informatics 21, no. 3 (May 19, 2017): 403–8. http://dx.doi.org/10.20965/jaciii.2017.p0403.

Full text

Abstract:

Recognizing different weather conditions is a core component of many different applications of outdoor video analysis and computer vision. Street analysis performance, including detecting street objects, detecting road lines, recognizing street sign and etc., varies greatly with weather, so modeling based on weather recognition is the key resolution in this field. Features derived from intrinsic properties of different weather conditions contribute to successful classification. We first propose using deep learning features from convolutional neural networks (CNN) for fine recognition. In order to reduce the parameter redundancy in CNN, we used sparse decomposition to dramatically cut down the computation. Recognition results for databases show superior performance and indicate the effectiveness of extracted features.

APA, Harvard, Vancouver, ISO, and other styles

33

Schwab, Johannes, Stephan Antholzer, and Markus Haltmeier. "Big in Japan: Regularizing Networks for Solving Inverse Problems." Journal of Mathematical Imaging and Vision 62, no. 3 (October 3, 2019): 445–55. http://dx.doi.org/10.1007/s10851-019-00911-1.

Full text

Abstract:

Abstract Deep learning and (deep) neural networks are emerging tools to address inverse problems and image reconstruction tasks. Despite outstanding performance, the mathematical analysis for solving inverse problems by neural networks is mostly missing. In this paper, we introduce and rigorously analyze families of deep regularizing neural networks (RegNets) of the form $$\mathbf {B}_\alpha + \mathbf {N}_{\theta (\alpha )} \mathbf {B}_\alpha $$Bα+Nθ(α)Bα, where $$\mathbf {B}_\alpha $$Bα is a classical regularization and the network $$\mathbf {N}_{\theta (\alpha )} \mathbf {B}_\alpha $$Nθ(α)Bα is trained to recover the missing part $${\text {Id}}_X - \mathbf {B}_\alpha $$IdX-Bα not found by the classical regularization. We show that these regularizing networks yield a convergent regularization method for solving inverse problems. Additionally, we derive convergence rates (quantitative error estimates) assuming a sufficient decay of the associated distance function. We demonstrate that our results recover existing convergence and convergence rates results for filter-based regularization methods as well as the recently introduced null space network as special cases. Numerical results are presented for a tomographic sparse data problem, which clearly demonstrate that the proposed RegNets improve classical regularization as well as the null space network.

APA, Harvard, Vancouver, ISO, and other styles

34

.., Vani, and Piyush Kumar Pareek. "Deep Multiple Instance Learning Approach for Classification in Clinical Decision Support Systems." American Journal of Business and Operations Research 10, no. 2 (2023): 52–60. http://dx.doi.org/10.54216/ajbor.100206.

Full text

Abstract:

To get around the drawbacks of conventional classification algorithms that required manual feature extraction and the high computational cost of neural networks, this paper introduces a deep convolutional neural network with multiple instance learning approaches, namely dynamic max pooling and sparse representation. For the categorization of tuberculosis lung illness, this model combines deep convolutional neural networks and multiple instance learning. The design was composed of four phases: pre-processing, instance production, feature extraction, and classification. To perform feature extraction, a model based on a customized version of the VGG16 architecture was trained from scratch. Multiple instance learning techniques such as Diverse Density (DD) and the Maximum pattern bag formulation of the Support Vector Machine were used to evaluate how well the proposed classification algorithm performed in comparison (SVM).The numerical findings demonstrated that the new method offered a higher level of accuracy than the methods that had been used in the past. When evaluating the efficacy of the current method, accuracy, specificity, sensitivity, and error rate were all taken into consideration. The accuracy of the max-pooling based framework and the sparse representation framework was found to be greater than that of the other multiple instance strategies, coming in at 91.51% and 89.84%, respectively, when compared to that of the other methods. The improved accuracy of the present system that makes use of deep neural networks is mostly attributable to the contributions made by features such as transfer learning and automatic feature extraction.

APA, Harvard, Vancouver, ISO, and other styles

35

He, Haoyuan, Lingxuan Huang, Zisen Huang, and Tiantian Yang. "The Compression Techniques Applied on Deep Learning Model." Highlights in Science, Engineering and Technology 4 (July 26, 2022): 325–31. http://dx.doi.org/10.54097/hset.v4i.920.

Full text

Abstract:

In recent years, the penetration rate of smartphones has gradually completed, artificial intelligence is the cutting-edge technology that can trigger disruptive changes. Deep learning neural networks are also starting to appear on mobile devices. In order to obtain better performance, more complex networks need to be designed, and the corresponding models, computation and storage space are increasing, however, the challenges of resource allocation and energy consumption still exist in mobile. The techniques for compressing deep learning models are quite important, and this paper studies a series of related literatures. This paper reviews deep learning-based deep neural network compression techniques and introduces the key operational points of knowledge extraction and network model on the learning performance of Resolution-Aware Knowledge Distillation. In this paper, a low-rank decomposition algorithm is evaluated based on sparse parameters and rank using the extended BIC for tuning parameter selection. This paper discusses the reduction of redundancy in the fully connected and constitutive layers of the training network model by pruning strategies.Moreover, this paper presents the quantization techniques and a neural network that quantifies weights and activations by applying differentiable nonlinear functions.

APA, Harvard, Vancouver, ISO, and other styles

36

Almulla Khalaf, Maysa Ibrahem, and John Q. Gan. "A three-stage learning algorithm for deep multilayer perceptron with effective weight initialisation based on sparse auto-encoder." Artificial Intelligence Research 8, no. 1 (April 2, 2019): 41. http://dx.doi.org/10.5430/air.v8n1p41.

Full text

Abstract:

A three-stage learning algorithm for deep multilayer perceptron (DMLP) with effective weight initialisation based on sparse auto-encoder is proposed in this paper, which aims to overcome difficulties in training deep neural networks with limited training data in high-dimensional feature space. At the first stage, unsupervised learning is adopted using sparse auto-encoder to obtain the initial weights of the feature extraction layers of the DMLP. At the second stage, error back-propagation is used to train the DMLP by fixing the weights obtained at the first stage for its feature extraction layers. At the third stage, all the weights of the DMLP obtained at the second stage are refined by error back-propagation. Network structures and values of learning parameters are determined through cross-validation, and test datasets unseen in the cross-validation are used to evaluate the performance of the DMLP trained using the three-stage learning algorithm. Experimental results show that the proposed method is effective in combating overfitting in training deep neural networks.

APA, Harvard, Vancouver, ISO, and other styles

37

Zahn, Olivia, Jorge Bustamante, Callin Switzer, Thomas L. Daniel, and J. Nathan Kutz. "Pruning deep neural networks generates a sparse, bio-inspired nonlinear controller for insect flight." PLOS Computational Biology 18, no. 9 (September 27, 2022): e1010512. http://dx.doi.org/10.1371/journal.pcbi.1010512.

Full text

Abstract:

Insect flight is a strongly nonlinear and actuated dynamical system. As such, strategies for understanding its control have typically relied on either model-based methods or linearizations thereof. Here we develop a framework that combines model predictive control on an established flight dynamics model and deep neural networks (DNN) to create an efficient method for solving the inverse problem of flight control. We turn to natural systems for inspiration since they inherently demonstrate network pruning with the consequence of yielding more efficient networks for a specific set of tasks. This bio-inspired approach allows us to leverage network pruning to optimally sparsify a DNN architecture in order to perform flight tasks with as few neural connections as possible, however, there are limits to sparsification. Specifically, as the number of connections falls below a critical threshold, flight performance drops considerably. We develop sparsification paradigms and explore their limits for control tasks. Monte Carlo simulations also quantify the statistical distribution of network weights during pruning given initial random weights of the DNNs. We demonstrate that on average, the network can be pruned to retain a small amount of original network weights and still perform comparably to its fully-connected counterpart. The relative number of remaining weights, however, is highly dependent on the initial architecture and size of the network. Overall, this work shows that sparsely connected DNNs are capable of predicting the forces required to follow flight trajectories. Additionally, sparsification has sharp performance limits.

APA, Harvard, Vancouver, ISO, and other styles

38

Liu, Xiao, Wenbin Li, Jing Huo, Lili Yao, and Yang Gao. "Layerwise Sparse Coding for Pruned Deep Neural Networks with Extreme Compression Ratio." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 4900–4907. http://dx.doi.org/10.1609/aaai.v34i04.5927.

Full text

Abstract:

Deep neural network compression is important and increasingly developed especially in resource-constrained environments, such as autonomous drones and wearable devices. Basically, we can easily and largely reduce the number of weights of a trained deep model by adopting a widely used model compression technique, e.g., pruning. In this way, two kinds of data are usually preserved for this compressed model, i.e., non-zero weights and meta-data, where meta-data is employed to help encode and decode these non-zero weights. Although we can obtain an ideally small number of non-zero weights through pruning, existing sparse matrix coding methods still need a much larger amount of meta-data (may several times larger than non-zero weights), which will be a severe bottleneck of the deploying of very deep models. To tackle this issue, we propose a layerwise sparse coding (LSC) method to maximize the compression ratio by extremely reducing the amount of meta-data. We first divide a sparse matrix into multiple small blocks and remove zero blocks, and then propose a novel signed relative index (SRI) algorithm to encode the remaining non-zero blocks (with much less meta-data). In addition, the proposed LSC performs parallel matrix multiplication without full decoding, while traditional methods cannot. Through extensive experiments, we demonstrate that LSC achieves substantial gains in pruned DNN compression (e.g., 51.03x compression ratio on ADMM-Lenet) and inference computation (i.e., time reduction and extremely less memory bandwidth), over state-of-the-art baselines.

APA, Harvard, Vancouver, ISO, and other styles

39

Yao, Zhongtian, Kejie Huang, Haibin Shen, and Zhaoyan Ming. "Deep Neural Network Acceleration With Sparse Prediction Layers." IEEE Access 8 (2020): 6839–48. http://dx.doi.org/10.1109/access.2020.2963941.

Full text

APA, Harvard, Vancouver, ISO, and other styles

40

Lee, Gwo-Chuan, Jyun-Hong Li, and Zi-Yang Li. "A Wasserstein Generative Adversarial Network–Gradient Penalty-Based Model with Imbalanced Data Enhancement for Network Intrusion Detection." Applied Sciences 13, no. 14 (July 12, 2023): 8132. http://dx.doi.org/10.3390/app13148132.

Full text

Abstract:

In today’s network intrusion detection systems (NIDS), certain types of network attack packets are sparse compared to regular network packets, making them challenging to collect, and resulting in significant data imbalances in public NIDS datasets. With respect to attack types with rare data, it is difficult to classify them, even by using various algorithms such as machine learning and deep learning. To address this issue, this study proposes a data augmentation technique based on the WGAN-GP model to enhance the recognition accuracy of sparse attacks in network intrusion detection. The enhanced performance of the WGAN-GP model on sparse attack classes is validated by evaluating three sparse data generation methods, namely Gaussian noise, WGAN-GP, and SMOTE, using the NSL-KDD dataset. Additionally, machine learning algorithms, including KNN, SVM, random forest, and XGBoost, as well as neural network models such as multilayer perceptual neural networks (MLP) and convolutional neural networks (CNN), are applied to classify the enhanced NSL-KDD dataset. Experimental results revealed that the WGAN-GP generation model is the most effective for detecting sparse data probes. Furthermore, a two-stage fine-tuning algorithm based on the WGAN-GP model is developed, fine-tuning the classification algorithms and model parameters to optimize the recognition accuracy of the sparse data probes. The final experimental results demonstrate that the MLP classifier significantly increases the accuracy rate from 74% to 80% after fine tuning, surpassing all other classifiers. The proposed method exhibits a 10%, 7%, and 13% improvement over untuned Gaussian noise enhancement, untuned SMOTE enhancement, and no enhancement.

APA, Harvard, Vancouver, ISO, and other styles

41

Phan, Huy, Miao Yin, Yang Sui, Bo Yuan, and Saman Zonouz. "CSTAR: Towards Compact and Structured Deep Neural Networks with Adversarial Robustness." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 2 (June 26, 2023): 2065–73. http://dx.doi.org/10.1609/aaai.v37i2.25299.

Full text

Abstract:

Model compression and model defense for deep neural networks (DNNs) have been extensively and individually studied. Considering the co-importance of model compactness and robustness in practical applications, several prior works have explored to improve the adversarial robustness of the sparse neural networks. However, the structured sparse models obtained by the existing works suffer severe performance degradation for both benign and robust accuracy, thereby causing a challenging dilemma between robustness and structuredness of compact DNNs. To address this problem, in this paper, we propose CSTAR, an efficient solution that simultaneously impose Compactness, high STructuredness and high Adversarial Robustness on the target DNN models. By formulating the structuredness and robustness requirement within the same framework, the compressed DNNs can simultaneously achieve high compression performance and strong adversarial robustness. Evaluations for various DNN models on different datasets demonstrate the effectiveness of CSTAR. Compared with the state-of-the-art robust structured pruning, CSTAR shows consistently better performance. For instance, when compressing ResNet-18 on CIFAR-10, CSTAR achieves up to 20.07% and 11.91% improvement for benign accuracy and robust accuracy, respectively. For compressing ResNet-18 with 16x compression ratio on Imagenet, CSTAR obtains 8.58% benign accuracy gain and 4.27% robust accuracy gain compared to the existing robust structured pruning.

APA, Harvard, Vancouver, ISO, and other styles

42

Zhang, Hongwei, Jiacheng Ni, Kaiming Li, Ying Luo, and Qun Zhang. "Nonsparse SAR Scene Imaging Network Based on Sparse Representation and Approximate Observations." Remote Sensing 15, no. 17 (August 22, 2023): 4126. http://dx.doi.org/10.3390/rs15174126.

Full text

Abstract:

Sparse-representation-based synthetic aperture radar (SAR) imaging technology has shown superior potential in the reconstruction of nonsparse scenes. However, many existing compressed sensing (CS) methods with sparse representation cannot obtain an optimal sparse basis and only apply to the sensing matrix obtained by exact observation, resulting in a low image quality occupying more storage space. To reduce the computational cost and improve the imaging performance of nonsparse scenes, we formulate a deep learning SAR imaging method based on sparse representation and approximated observation deduced from the chirp-scaling algorithm (CSA). First, we incorporate the CSA-derived approximated observation model and a nonlinear transform function within a sparse reconstruction framework. Second, an iterative shrinkage threshold algorithm is adopted to solve this framework, and the solving process is unfolded as a deep SAR imaging network. Third, a dual-path convolutional neural network (CNN) block is designed in the network to achieve the nonlinear transform, dramatically improving the sparse representation capability over conventional transform-domain-based CS methods. Last, we improve the CNN block to develop an enhanced version of the deep SAR imaging network, in which all the parameters are layer-varied and trained by supervised learning. The experiments demonstrate that our proposed two imaging networks outperform conventional CS-driven and deep-learning-based methods in terms of computing efficiency and reconstruction performance of nonsparse scenes.

APA, Harvard, Vancouver, ISO, and other styles

43

Gong, Zhenghui, Xiaolong Su, Panhe Hu, Shuowei Liu, and Zhen Liu. "Deep Unfolding Sparse Bayesian Learning Network for Off-Grid DOA Estimation with Nested Array." Remote Sensing 15, no. 22 (November 10, 2023): 5320. http://dx.doi.org/10.3390/rs15225320.

Full text

Abstract:

Recently, deep unfolding networks have been widely used in direction of arrival (DOA) estimation because of their improved estimation accuracy and reduced computational cost. However, few have considered the existence of a nested array (NA) with off-grid DOA estimation. In this study, we present a deep sparse Bayesian learning (DSBL) network to solve this problem. We first establish the signal model for off-grid DOA with NA. Then, we transform the array output into a real domain for neural networks. Finally, we construct and train the DSBL network to determine the on-grid spatial spectrum and off-grid value, where the loss function is calculated using reconstruction error and the sparsity of network output, and the layers correspond to the steps of the sparse Bayesian learning algorithm. We demonstrate that the DSBL network can achieve better generalization ability without training labels and large-scale training data. The simulation results validate the effectiveness of the DSBL network when compared with those of existing methods.

APA, Harvard, Vancouver, ISO, and other styles

44

Chen, Yuanyuan, and Zhang Yi. "Adaptive sparse dropout: Learning the certainty and uncertainty in deep neural networks." Neurocomputing 450 (August 2021): 354–61. http://dx.doi.org/10.1016/j.neucom.2021.04.047.

Full text

APA, Harvard, Vancouver, ISO, and other styles

45

Chen, Jiayu, Xiang Li, Vince D. Calhoun, Jessica A. Turner, Theo G. M. Erp, Lei Wang, Ole A. Andreassen, et al. "Sparse deep neural networks on imaging genetics for schizophrenia case–control classification." Human Brain Mapping 42, no. 8 (March 16, 2021): 2556–68. http://dx.doi.org/10.1002/hbm.25387.

Full text

APA, Harvard, Vancouver, ISO, and other styles

46

Kovacs, Mate, and Victor V. Kryssanov. "Expanding the Feature Space of Deep Neural Networks for Sentiment Classification." International Journal of Machine Learning and Computing 10, no. 2 (February 2020): 271–76. http://dx.doi.org/10.18178/ijmlc.2020.10.2.931.

Full text

APA, Harvard, Vancouver, ISO, and other styles

47

Lui, Hugo F. S., and William R. Wolf. "Construction of reduced-order models for fluid flows using deep feedforward neural networks." Journal of Fluid Mechanics 872 (June 14, 2019): 963–94. http://dx.doi.org/10.1017/jfm.2019.358.

Full text

Abstract:

We present a numerical methodology for construction of reduced-order models (ROMs) of fluid flows through the combination of flow modal decomposition and regression analysis. Spectral proper orthogonal decomposition is applied to reduce the dimensionality of the model and, at the same time, filter the proper orthogonal decomposition temporal modes. The regression step is performed by a deep feedforward neural network (DNN), and the current framework is implemented in a context similar to the sparse identification of nonlinear dynamics algorithm. A discussion on the optimization of the DNN hyperparameters is provided for obtaining the best ROMs and an assessment of these models is presented for a canonical nonlinear oscillator and the compressible flow past a cylinder. Then the method is tested on the reconstruction of a turbulent flow computed by a large eddy simulation of a plunging airfoil under dynamic stall. The reduced-order model is able to capture the dynamics of the leading edge stall vortex and the subsequent trailing edge vortex. For the cases analysed, the numerical framework allows the prediction of the flow field beyond the training window using larger time increments than those employed by the full-order model. We also demonstrate the robustness of the current ROMs constructed via DNNs through a comparison with sparse regression. The DNN approach is able to learn transient features of the flow and presents more accurate and stable long-term predictions compared to sparse regression.

APA, Harvard, Vancouver, ISO, and other styles

48

Chen, Qipeng, Qiaoqiao Xiong, Haisong Huang, Saihong Tang, and Zhenghong Liu. "Research on the Construction of an Efficient and Lightweight Online Detection Method for Tiny Surface Defects through Model Compression and Knowledge Distillation." Electronics 13, no. 2 (January 5, 2024): 253. http://dx.doi.org/10.3390/electronics13020253.

Full text

Abstract:

In response to the current issues of poor real-time performance, high computational costs, and excessive memory usage of object detection algorithms based on deep convolutional neural networks in embedded devices, a method for improving deep convolutional neural networks based on model compression and knowledge distillation is proposed. Firstly, data augmentation is employed in the preprocessing stage to increase the diversity of training samples, thereby improving the model’s robustness and generalization capability. The K-means++ clustering algorithm generates candidate bounding boxes, adapting to defects of different sizes and selecting finer features earlier. Secondly, the cross stage partial (CSP) Darknet53 network and spatial pyramid pooling (SPP) module extract features from the input raw images, enhancing the accuracy of defect location detection and recognition in YOLO. Finally, the concept of model compression is integrated, utilizing scaling factors in the batch normalization (BN) layer, and introducing sparse factors to perform sparse training on the network. Channel pruning and layer pruning are applied to the sparse model, and post-processing methods using knowledge distillation are used to effectively reduce the model size and forward inference time while maintaining model accuracy. The improved model size decreases from 244 M to 4.19 M, the detection speed increases from 32.8 f/s to 68 f/s, and mAP reaches 97.41. Experimental results demonstrate that this method is conducive to deploying network models on embedded devices with limited GPU computing and storage resources. It can be applied in distributed service architectures for edge computing, providing new technological references for deploying deep learning models in the industrial sector.

APA, Harvard, Vancouver, ISO, and other styles

49

Zhao, Yao, Chengwen Ou, He Tian, Bingo Wing-Kuen Ling, Ye Tian, and Zhe Zhang. "Sparse SAR Imaging Algorithm in Marine Environments Based on Memory-Augmented Deep Unfolding Network." Remote Sensing 16, no. 7 (April 5, 2024): 1289. http://dx.doi.org/10.3390/rs16071289.

Full text

Abstract:

Oceanic targets, including ripples, islands, vessels, and coastlines, display distinct sparse characteristics, rendering the ocean a significant arena for sparse Synthetic Aperture Radar (SAR) imaging rooted in sparse signal processing. Deep neural networks (DNNs), a current research emphasis, have, when integrated with sparse SAR, attracted notable attention for their exceptional imaging capabilities and high computational efficiency. Yet, the efficiency of traditional unfolding techniques is impeded by their architecturally inefficient design, which curtails their information transmission capacity and consequently detracts from the quality of reconstruction. This paper unveils a novel Memory-Augmented Deep Unfolding Network (MADUN) for SAR imaging in marine environments. Our methodology harnesses the synergies between deep learning and algorithmic unfolding, enhanced with a memory component, to elevate SAR imaging’s computational precision. At the heart of our investigation is the incorporation of High-Throughput Short-Term Memory (HSM) and Cross-Stage Long-Term Memory (CLM) within the MADUN framework, ensuring robust information flow across unfolding stages and solidifying the foundation for deep, long-term informational correlations. Our experimental results demonstrate that our strategy significantly surpasses existing methods in enhancing the reconstruction of sparse marine scenes.

APA, Harvard, Vancouver, ISO, and other styles

50

Kohjima, Masahiro. "Shuffled Deep Regression." Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 12 (March 24, 2024): 13238–45. http://dx.doi.org/10.1609/aaai.v38i12.29224.

Full text

Abstract:

Shuffled regression is the problem of learning regression models from shuffled data that consists of a set of input features and a set of target outputs where the correspondence between the input and output is unknown. This study proposes a new deep learning method for shuffled regression called Shuffled Deep Regression (SDR). We derive the sparse and stochastic variant of the Expectation-Maximization algorithm for SDR that iteratively updates discrete latent variables and the parameters of neural networks. The effectiveness of the proposal is confirmed by benchmark data experiments.

APA, Harvard, Vancouver, ISO, and other styles

Journal articles on the topic 'Sparse deep neural networks'

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles