To see the other types of publications on this topic, follow the link: Autoencoders.
Journal articles on the topic 'Autoencoders'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Autoencoders.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Alfayez, Sarah, Ouiem Bchir, and Mohamed Maher Ben Ismail. "Dynamic Depth Learning in Stacked AutoEncoders." Applied Sciences 13, no. 19 (2023): 10994. http://dx.doi.org/10.3390/app131910994.
Full textAbstract:
The effectiveness of deep learning models depends on their architecture and topology. Thus, it is essential to determine the optimal depth of the network. In this paper, we propose a novel approach to learn the optimal depth of a stacked AutoEncoder, called Dynamic Depth for Stacked AutoEncoders (DDSAE). DDSAE learns in an unsupervised manner the depth of a stacked AutoEncoder while training the network model. Specifically, we propose a novel objective function, aside from the AutoEncoder’s loss function to optimize the network depth: The optimization of the objective function determines the layers’ relevance weights. Additionally, we propose an algorithm that iteratively prunes the irrelevant layers based on the learned relevance weights. The performance of DDSAE was assessed using benchmark and real datasets.
2
Sreeteish, M. "Image De-Noising Using Convolutional Variational Autoencoders." International Journal for Research in Applied Science and Engineering Technology 10, no. 6 (2022): 4002–9. http://dx.doi.org/10.22214/ijraset.2022.44826.
Full textAbstract:
Abstract: Typically, image noise is random colour information in picture pixels that serves as an unfavourable by-product of the image, obscuring the intended information. In most cases, noise is injected into photographs during the transfer or reception of the image, or during the capture of an image when an object is moving quickly. To improve the noisy picture predictions, autoencoders that denoise the input images are employed. Autoencoders are a sort of unsupervised machine learning that compresses the input and reconstructs an output that is very similar to the original input. The autoencoder tries to figure out non-linear correlations between data points. An encoder, a latent space, and a decoder all exist in autoencoders. The encoder reduces the dimensionality of an original picture to its latent space representation, which is then used by the decoder to reconstruct the reduced dimensional image back to its original image. Basic Autoencoder, Variational Autoencoder, and Convolutional Autoencoder are the three approaches that were employed to denoise the picture. In the basic and convolutional autoencoders, there is only one loss parameter, however in the variational autoencoder, there are two losses: generative loss and latent loss. TensorFlow as the frontend and Keras as the backend are used to implement autoencoders in this project. The noisy pictures are trained on every convolutional variational autoencoder techniques to produce a decent prediction of noisy test data.
3
Jin, Weihua, Bo Sun, Zhidong Li, Shijie Zhang, and Zhonggui Chen. "Detecting Anomalies of Satellite Power Subsystem via Stage-Training Denoising Autoencoders." Sensors 19, no. 14 (2019): 3216. http://dx.doi.org/10.3390/s19143216.
Full textAbstract:
Satellite telemetry data contains satellite status information, and ground-monitoring personnel need to promptly detect satellite anomalies from these data. This paper takes the satellite power subsystem as an example and presents a reliable anomaly detection method. Due to the lack of abnormal data, the autoencoder is a powerful method for unsupervised anomaly detection. This study proposes a novel stage-training denoising autoencoder (ST-DAE) that trains the features, in stages. This novel method has better reconstruction capabilities in comparison to common autoencoders, sparse autoencoders, and denoising autoencoders. Meanwhile, a cluster-based anomaly threshold determination method is proposed. In this study, specific methods were designed to evaluate the autoencoder performance in three perspectives. Experiments were carried out on real satellite telemetry data, and the results showed that the proposed ST-DAE generally outperformed the autoencoders, in comparison.
4
Shevchenko, Dmytro, Mykhaylo Ugryumov, and Sergii Artiukh. "MONITORING DATA AGGREGATION OF DYNAMIC SYSTEMS USING INFORMATION TECHNOLOGIES." Innovative Technologies and Scientific Solutions for Industries, no. 1 (23) (April 20, 2023): 123–31. http://dx.doi.org/10.30837/itssi.2023.23.123.
Full textAbstract:
The subject matter of the article is models, methods and information technologies of monitoring data aggregation. The goal of the article is to determine the best deep learning model for reducing the dimensionality of dynamic systems monitoring data. The following tasks were solved: analysis of existing dimensionality reduction approaches, description of the general architecture of vanilla and variational autoencoders, development of their architecture, development of software for training and testing of autoencoders, conducting research on the performance quality of autoencoders for the problem of dimensionality reduction. The following models and methods were used: data processing and preparation, data dimensionality reduction. The software was developed using the Python language. Scikit-learn, Pandas, PyTorch, NumPy, argparse and others were used as auxiliary libraries. Obtained results: the work presents a classification of models and methods for dimensionality reduction, general reviews of vanilla and variational autoencoders, which include a description of the models, their properties, loss functions and their application to the problem of dimensionality reduction. Custom autoencoder architectures were also created, including visual representations of the autoencoder architecture and descriptions of each component. The software for training and testing autoencoders was developed, the dynamic system monitoring data set, and the steps for pre-training the data set were described. The metric for evaluating the quality of models is also described; the configuration of autoencoders and their training are considered. Conclusions: The vanilla autoencoder recovers the data much better than the variational one. Looking at the fact that the architectures of the autoencoders are the same, except for the peculiarities of the autoencoders, it can be noted that a vanilla autoencoder compresses data better by keeping more useful variables for later recovery from the bottleneck. Additionally, by training on different bottleneck sizes, you can determine the size at which the data is recovered best, which means that the most important variables are preserved. Looking at the results in general, the autoencoders work effectively for the dimensionality reduction task and the data recovery quality metric shows that they recover the data well with an error of 3–4 digits after 0. In conclusion, the vanilla autoencoder is the best deep learning model for aggregating monitoring data of dynamic systems.
5
Shin, Seung Yeop, and Han-joon Kim. "Extended Autoencoder for Novelty Detection with Reconstruction along Projection Pathway." Applied Sciences 10, no. 13 (2020): 4497. http://dx.doi.org/10.3390/app10134497.
Full textAbstract:
Recently, novelty detection with reconstruction along projection pathway (RaPP) has made progress toward leveraging hidden activation values. RaPP compares the input and its autoencoder reconstruction in hidden spaces to detect novelty samples. Nevertheless, traditional autoencoders have not yet begun to fully exploit this method. In this paper, we propose a new model, the Extended Autoencoder Model, that adds an adversarial component to the autoencoder to take full advantage of RaPP. The adversarial component matches the latent variables of the reconstructed input to the latent variables of the original input to detect novelty samples with high hidden reconstruction errors. The proposed model can be combined with variants of the autoencoder, such as a variational autoencoder or adversarial autoencoder. The effectiveness of the proposed model was evaluated across various novelty detection datasets. Our results demonstrated that extended autoencoders are capable of outperforming conventional autoencoders in detecting novelties using the RaPP method.
6
Song, Youngrok, Sangwon Hyun, and Yun-Gyung Cheong. "Analysis of Autoencoders for Network Intrusion Detection." Sensors 21, no. 13 (2021): 4294. http://dx.doi.org/10.3390/s21134294.
Full textAbstract:
As network attacks are constantly and dramatically evolving, demonstrating new patterns, intelligent Network Intrusion Detection Systems (NIDS), using deep-learning techniques, have been actively studied to tackle these problems. Recently, various autoencoders have been used for NIDS in order to accurately and promptly detect unknown types of attacks (i.e., zero-day attacks) and also alleviate the burden of the laborious labeling task. Although the autoencoders are effective in detecting unknown types of attacks, it takes tremendous time and effort to find the optimal model architecture and hyperparameter settings of the autoencoders that result in the best detection performance. This can be an obstacle that hinders practical applications of autoencoder-based NIDS. To address this challenge, we rigorously study autoencoders using the benchmark datasets, NSL-KDD, IoTID20, and N-BaIoT. We evaluate multiple combinations of different model structures and latent sizes, using a simple autoencoder model. The results indicate that the latent size of an autoencoder model can have a significant impact on the IDS performance.
7
Ghafar, Abdul, and Usman Sattar. "Convolutional Autoencoder for Image Denoising." UMT Artificial Intelligence Review 1, no. 2 (2021): 1–11. http://dx.doi.org/10.32350/air.0102.01.
Full textAbstract:
Image denoising is a process used to remove noise from the image to create a sharp and clear image. It is mainly used in medical imaging, where due to the malfunctioning of machines or due to the precautions taken to protect patients from radiation, medical imaging machines create a lot of noise in the final image. Several techniques can be used in order to avoid such distortions in the image before their final printing. Autoencoders are the most notable software used to denoise images before their final printing. These software are not intelligent so the resultant image is not of good quality. In this paper, we introduced a modified autoencoder having a deep convolutional neural network. It creates better quality images as compared to traditional autoencoders. After training with a test dataset on the tensor board, the modified autoencoder is tested on a different dataset having various shapes. The results were satisfactory but not desirable due to several reasons. Nevertheless, our proposed system still performed better than traditional autoencoders.
 KEYWORDS: image denoising, deep learning, convolutional neural network, image autoencoder, image convolutional autoencoder
8
Liu, Junhong. "Review of variational autoencoders model." Applied and Computational Engineering 4, no. 1 (2023): 588–96. http://dx.doi.org/10.54254/2755-2721/4/2023328.
Full textAbstract:
Variational autoencoder is one of the deep latent space generation models, which has become increasingly popular in image generation and anomaly detection in recent years. In this paper, we first review the development and research status of traditional variational autoencoders and their variants, and summarize and compare the performance of all variational autoencoders. then give a possible development direction of VAE.
9
Lin, Yen-Kuang, Chen-Yin Lee, and Chen-Yueh Chen. "Robustness of autoencoders for establishing psychometric properties based on small sample sizes: results from a Monte Carlo simulation study and a sports fan curiosity study." PeerJ Computer Science 8 (February 9, 2022): e782. http://dx.doi.org/10.7717/peerj-cs.782.
Full textAbstract:
Background The principal component analysis (PCA) is known as a multivariate statistical model for reducing dimensions into a representation of principal components. Thus, the PCA is commonly adopted for establishing psychometric properties, i.e., the construct validity. Autoencoder is a neural network model, which has also been shown to perform well in dimensionality reduction. Although there are several ways the PCA and autoencoders could be compared for their differences, most of the recent literature focused on differences in image reconstruction, which are often sufficient for training data. In the current study, we looked at details of each autoencoder classifier and how they may provide neural network superiority that can better generalize non-normally distributed small datasets. Methodology A Monte Carlo simulation was conducted, varying the levels of non-normality, sample sizes, and levels of communality. The performances of autoencoders and a PCA were compared using the mean square error, mean absolute value, and Euclidian distance. The feasibility of autoencoders with small sample sizes was examined. Conclusions With extreme flexibility in decoding representation using linear and non-linear mapping, this study demonstrated that the autoencoder can robustly reduce dimensions, and hence was effective in building the construct validity with a sample size as small as 100. The autoencoders could obtain a smaller mean square error and small Euclidian distance between original dataset and predictions for a small non-normal dataset. Hence, when behavioral scientists attempt to explore the construct validity of a newly designed questionnaire, an autoencoder could also be considered an alternative to a PCA.
10
Alam, Fardina Fathmiul, Taseef Rahman, and Amarda Shehu. "Evaluating Autoencoder-Based Featurization and Supervised Learning for Protein Decoy Selection." Molecules 25, no. 5 (2020): 1146. http://dx.doi.org/10.3390/molecules25051146.
Full textAbstract:
Rapid growth in molecular structure data is renewing interest in featurizing structure. Featurizations that retain information on biological activity are particularly sought for protein molecules, where decades of research have shown that indeed structure encodes function. Research on featurization of protein structure is active, but here we assess the promise of autoencoders. Motivated by rapid progress in neural network research, we investigate and evaluate autoencoders on yielding linear and nonlinear featurizations of protein tertiary structures. An additional reason we focus on autoencoders as the engine to obtain featurizations is the versatility of their architectures and the ease with which changes to architecture yield linear versus nonlinear features. While open-source neural network libraries, such as Keras, which we employ here, greatly facilitate constructing, training, and evaluating autoencoder architectures and conducting model search, autoencoders have not yet gained popularity in the structure biology community. Here we demonstrate their utility in a practical context. Employing autoencoder-based featurizations, we address the classic problem of decoy selection in protein structure prediction. Utilizing off-the-shelf supervised learning methods, we demonstrate that the featurizations are indeed meaningful and allow detecting active tertiary structures, thus opening the way for further avenues of research.
11
Provan, Gregory. "Toward Explainable AutoEncoder-Based Diagnosis of Dynamical Systems." Algorithms 16, no. 4 (2023): 178. http://dx.doi.org/10.3390/a16040178.
Full textAbstract:
Autoencoders have been used widely for diagnosing devices, for example, faults in rotating machinery. However, autoencoder-based approaches lack explainability for their results and can be hard to tune. In this article, we propose an explainable method for applying autoencoders for diagnosis, where we use a metric that maximizes the diagnostics accuracy. Since an autoencoder projects the input into a reduced subspace (the code), we define a theoretically well-understood approach, the subspace principal angle, to define a metric over the possible fault labels. We show how this approach can be used for both single-device diagnostics (e.g., faults in rotating machinery) and complex (multi-device) dynamical systems. We empirically validate the theoretical claims using multiple autoencoder architectures.
12
Wu, Hanwei, and Markus Flierl. "Vector Quantization-Based Regularization for Autoencoders." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (2020): 6380–87. http://dx.doi.org/10.1609/aaai.v34i04.6108.
Full textAbstract:
Autoencoders and their variations provide unsupervised models for learning low-dimensional representations for downstream tasks. Without proper regularization, autoencoder models are susceptible to the overfitting problem and the so-called posterior collapse phenomenon. In this paper, we introduce a quantization-based regularizer in the bottleneck stage of autoencoder models to learn meaningful latent representations. We combine both perspectives of Vector Quantized-Variational AutoEncoders (VQ-VAE) and classical denoising regularization methods of neural networks. We interpret quantizers as regularizers that constrain latent representations while fostering a similarity-preserving mapping at the encoder. Before quantization, we impose noise on the latent codes and use a Bayesian estimator to optimize the quantizer-based representation. The introduced bottleneck Bayesian estimator outputs the posterior mean of the centroids to the decoder, and thus, is performing soft quantization of the noisy latent codes. We show that our proposed regularization method results in improved latent representations for both supervised learning and clustering downstream tasks when compared to autoencoders using other bottleneck structures.
13
Abdullayeva, Fargana J. "Cloud Computing Virtual Machine Workload Prediction Method Based on Variational Autoencoder." International Journal of Systems and Software Security and Protection 12, no. 2 (2021): 33–45. http://dx.doi.org/10.4018/ijsssp.2021070103.
Full textAbstract:
The paper proposes a method for predicting the workload of virtual machines in the cloud infrastructure. Reconstruction probabilities of variational autoencoders were used to provide the prediction. Reconstruction probability is a probability criterion that considers the variability in the distribution of variables. In the proposed approach, the values of the reconstruction probabilities of the variational autoencoder show the workload level of the virtual machines. The results of the experiments showed that variational autoencoders gave better results in predicting the workload of virtual machines compared to simple deep neural networks. The generative characteristics of the variational autoencoders determine the workload level by the data reconstruction.
14
Zeng, Mengjie, Shunming Li, Ranran Li, et al. "A Hierarchical Sparse Discriminant Autoencoder for Bearing Fault Diagnosis." Applied Sciences 12, no. 2 (2022): 818. http://dx.doi.org/10.3390/app12020818.
Full textAbstract:
Although some traditional autoencoders and their extensions have been widely used in the research of intelligent fault diagnosis of rotating parts, their feature extraction capabilities are limited without label information. In response to this problem, this research proposes a hierarchical sparse discriminant autoencoder (HSDAE) method for fault diagnosis of rotating components, which is a new semi-supervised autoencoder structure. By considering the sparsity of autoencoders, a hierarchical sparsity strategy was proposed to improve the stacked sparsity autoencoders, and the particle swarm optimization algorithm was used to obtain the optimal sparsity parameters to improve network performance. In order to enhance the classification of the autoencoder, a class aggregation and class separability strategy was used, which is an additional discriminative distance that was added as a penalty term in the loss function to enhance the feature extraction ability of the network. Finally, the reliability of the proposed method was verified on the bearing data set of Case Western Reserve University and the bearing data set of the laboratory test platform. The results of comparison with other methods show that the HSDAE method can enhance the feature extraction ability of the network and has reliability and stability for different data sets.
15
Srinivasa Rao Kandula, Et al. "Performance Evaluation of Deep Learning Autoencoder in Single and Multi-Carrier Systems." International Journal on Recent and Innovation Trends in Computing and Communication 11, no. 9 (2023): 3083–93. http://dx.doi.org/10.17762/ijritcc.v11i9.9448.
Full textAbstract:
An entire system of Single Carrier Communication and Orthogonal Frequency Division Multiplexing is modelled using an autoencoder. The model employs Deep Neural Networks (DNNs) as the transmitter and receiver, responsible for tasks such as encoding, modulation, demodulation, and decoding. The effectiveness of this approach is demonstrated by its ability to outperform traditional communication systems in real-world scenarios that involve channel and interference effects, as measured by the Block Error Rate. AI-enabled wireless systems can overcome limitations of traditional communication systems by learning from wireless spectrum data and optimizing performance for new wireless applications. The aim of this paper is to examine how autoencoder-based deep learning can enhance the performance of a communication system that employs Single Carrier and OFDM. The architecture effectively addresses channel impairments and improves overall performance. The simulation results suggest that even when the autoencoder's channel layer is affected by impairments, autoencoders still outperform traditional communication systems in terms of BLER performance.
16
Walbech, Julie Sparholt, Savvas Kinalis, Ole Winther, Finn Cilius Nielsen, and Frederik Otzen Bagger. "Interpretable Autoencoders Trained on Single Cell Sequencing Data Can Transfer Directly to Data from Unseen Tissues." Cells 11, no. 1 (2021): 85. http://dx.doi.org/10.3390/cells11010085.
Full textAbstract:
Autoencoders have been used to model single-cell mRNA-sequencing data with the purpose of denoising, visualization, data simulation, and dimensionality reduction. We, and others, have shown that autoencoders can be explainable models and interpreted in terms of biology. Here, we show that such autoencoders can generalize to the extent that they can transfer directly without additional training. In practice, we can extract biological modules, denoise, and classify data correctly from an autoencoder that was trained on a different dataset and with different cells (a foreign model). We deconvoluted the biological signal encoded in the bottleneck layer of scRNA-models using saliency maps and mapped salient features to biological pathways. Biological concepts could be associated with specific nodes and interpreted in relation to biological pathways. Even in this unsupervised framework, with no prior information about cell types or labels, the specific biological pathways deduced from the model were in line with findings in previous research. It was hypothesized that autoencoders could learn and represent meaningful biology; here, we show with a systematic experiment that this is true and even transcends the training data. This means that carefully trained autoencoders can be used to assist the interpretation of new unseen data.
17
Marchi, Erik, Fabio Vesperini, Stefano Squartini, and Björn Schuller. "Deep Recurrent Neural Network-Based Autoencoders for Acoustic Novelty Detection." Computational Intelligence and Neuroscience 2017 (2017): 1–14. http://dx.doi.org/10.1155/2017/4694860.
Full textAbstract:
In the emerging field of acoustic novelty detection, most research efforts are devoted to probabilistic approaches such as mixture models or state-space models. Only recent studies introduced (pseudo-)generative models for acoustic novelty detection with recurrent neural networks in the form of an autoencoder. In these approaches, auditory spectral features of the next short term frame are predicted from the previous frames by means of Long-Short Term Memory recurrent denoising autoencoders. The reconstruction error between the input and the output of the autoencoder is used as activation signal to detect novel events. There is no evidence of studies focused on comparing previous efforts to automatically recognize novel events from audio signals and giving a broad and in depth evaluation of recurrent neural network-based autoencoders. The present contribution aims to consistently evaluate our recent novel approaches to fill this white spot in the literature and provide insight by extensive evaluations carried out on three databases: A3Novelty, PASCAL CHiME, and PROMETHEUS. Besides providing an extensive analysis of novel and state-of-the-art methods, the article shows how RNN-based autoencoders outperform statistical approaches up to an absolute improvement of 16.4% averageF-measure over the three databases.
18
Lee, Rich C., and Ing-Yi Chen. "A Deep Dive of Autoencoder Models on Low-Contrast Aquatic Images." Sensors 21, no. 15 (2021): 4966. http://dx.doi.org/10.3390/s21154966.
Full textAbstract:
Public aquariums and similar institutions often use video as a method to monitor the behavior, health, and status of aquatic organisms in their environments. These video footages take up a sizeable amount of space and require the use of autoencoders to reduce their file size for efficient storage. The autoencoder neural network is an emerging technique which uses the extracted latent space from an input source to reduce the image size for storage, and then reconstructs the source within an acceptable loss range for use. To meet an aquarium’s practical needs, the autoencoder must have easily maintainable codes, low power consumption, be easily adoptable, and not require a substantial amount of memory use or processing power. Conventional configurations of autoencoders often provide results that perform beyond an aquarium’s needs at the cost of being too complex for their architecture to handle, while few take low-contrast sources into consideration. Thus, in this instance, “keeping it simple” would be the ideal approach to the autoencoder’s model design. This paper proposes a practical approach catered to an aquarium’s specific needs through the configuration of autoencoder parameters. It first explores the differences between the two of the most widely applied autoencoder approaches, Multilayer Perceptron (MLP) and Convolution Neural Networks (CNN), to identify the most appropriate approach. The paper concludes that while both approaches (with proper configurations and image preprocessing) can reduce the dimensionality and reduce visual noise of the low-contrast images gathered from aquatic video footage, the CNN approach is more suitable for an aquarium’s architecture. As an unexpected finding of the experiments conducted, the paper also discovered that by manipulating the formula for the MLP approach, the autoencoder could generate a denoised differential image that contains sharper and more desirable visual information to an aquarium’s operation. Lastly, the paper has found that proper image preprocessing prior to the application of the autoencoder led to better model convergence and prediction results, as demonstrated both visually and numerically in the experiment. The paper concludes that by combining the denoising effect of MLP, CNN’s ability to manage memory consumption, and proper image preprocessing, the specific practical needs of an aquarium can be adeptly fulfilled.
19
Li, Yanjun, and Yongquan Yan. "Training Autoencoders Using Relative Entropy Constraints." Applied Sciences 13, no. 1 (2022): 287. http://dx.doi.org/10.3390/app13010287.
Full textAbstract:
Autoencoders are widely used for dimensionality reduction and feature extraction. The backpropagation algorithm for training the parameters of the autoencoder model suffers from problems such as slow convergence. Therefore, researchers propose forward propagation algorithms. However, the existing forward propagation algorithms do not consider the characteristics of the data itself. This paper proposes an autoencoder forward training algorithm based on relative entropy constraints, called relative entropy autoencoder (REAE). When solving the feature map parameters, REAE imposes different constraints on the average activation value of the hidden layer outputs obtained by the feature map for different data sets. In the experimental section, different forward propagation algorithms are compared by applying the features extracted from the autoencoder to an image classification task. The experimental results on three image classification datasets show that the classification performance of the classification model constructed by REAE is better than that of the classification model constructed by other forward propagation algorithms.
20
Kristian, Yosi, Natanael Simogiarto, Mahendra Tri Arif Sampurna, and Elizeus Hanindito. "Ensemble of multimodal deep learning autoencoder for infant cry and pain detection." F1000Research 11 (March 28, 2022): 359. http://dx.doi.org/10.12688/f1000research.73108.1.
Full textAbstract:
Background: Babies cannot communicate their pain properly. Several pain scores are developed, but they are subjective and have high variability inter-observer agreement. The aim of this study was to construct models that use both facial expression and infant voice in classifying pain levels and cry detection. Methods: The study included a total of 23 infants below 12-months who were treated at Dr Soetomo General Hospital. The the Face Leg Activity Cry and Consolability (FLACC) pain scale and recordings of the baby's cries were taken in the video format. A machine-learning-based system was created to detect infant cries and pain levels. Spectrograms with the Short-Time Fourier Transform were used to convert the audio data into a time-frequency representation. Facial features combined with voice features extracted by using the Deep Learning Autoencoders was used for the classification of infant pain levels. Two types of autoencoders: Convolutional Autoencoder and Variational Autoencoder were used for both faces and voices. Result: The goal of the autoencoder was to produce a latent-vector with much smaller dimensions that was still able to recreate the data with minor losses. From the latent-vectors, a multimodal data representation for Convolutional Neural Network (CNN) was used for producing a relatively high F1 score, higher than single data modal such as the voice or facial expressions alone. Two major parts of the experiment were: 1. Building the three autoencoder models, which were autoencoder for the infant’s face, amplitude spectrogram, and dB-scaled spectrogram of infant’s voices. 2. Utilising the latent-vector result from the autoencoders to build the cry detection and pain classification models. Conclusion: In this paper, four pain classifier models with a relatively good F1 score was developed. These models were combined by using ensemble methods to improve performance, which resulted in a better F1 score.
21
Kristian, Yosi, Natanael Simogiarto, Mahendra Tri Arif Sampurna, Elizeus Hanindito, and Visuddho Visuddho. "Ensemble of multimodal deep learning autoencoder for infant cry and pain detection." F1000Research 11 (January 30, 2023): 359. http://dx.doi.org/10.12688/f1000research.73108.2.
Full textAbstract:
Background: Babies cannot communicate their pain properly. Several pain scores are developed, but they are subjective and have high variability inter-observer agreement. The aim of this study was to construct models that use both facial expression and infant voice in classifying pain levels and cry detection. Methods: The study included a total of 23 infants below 12-months who were treated at Dr Soetomo General Hospital. The the Face Leg Activity Cry and Consolability (FLACC) pain scale and recordings of the baby's cries were taken in the video format. A machine-learning-based system was created to detect infant cries and pain levels. Spectrograms with the Short-Time Fourier Transform were used to convert the audio data into a time-frequency representation. Facial features combined with voice features extracted by using the Deep Learning Autoencoders was used for the classification of infant pain levels. Two types of autoencoders: Convolutional Autoencoder and Variational Autoencoder were used for both faces and voices. Result: The goal of the autoencoder was to produce a latent-vector with much smaller dimensions that was still able to recreate the data with minor losses. From the latent-vectors, a multimodal data representation for Convolutional Neural Network (CNN) was used for producing a relatively high F1 score, higher than single data modal such as the voice or facial expressions alone. Two major parts of the experiment were: 1. Building the three autoencoder models, which were autoencoder for the infant’s face, amplitude spectrogram, and dB-scaled spectrogram of infant’s voices. 2. Utilising the latent-vector result from the autoencoders to build the cry detection and pain classification models. Conclusion: In this paper, four pain classifier models with a relatively good F1 score was developed. These models were combined by using ensemble methods to improve performance, which resulted in a better F1 score.
22
Al Machot, Fadi, Mohib Ullah, and Habib Ullah. "HFM: A Hybrid Feature Model Based on Conditional Auto Encoders for Zero-Shot Learning." Journal of Imaging 8, no. 6 (2022): 171. http://dx.doi.org/10.3390/jimaging8060171.
Full textAbstract:
Zero-Shot Learning (ZSL) is related to training machine learning models capable of classifying or predicting classes (labels) that are not involved in the training set (unseen classes). A well-known problem in Deep Learning (DL) is the requirement for large amount of training data. Zero-Shot learning is a straightforward approach that can be applied to overcome this problem. We propose a Hybrid Feature Model (HFM) based on conditional autoencoders for training a classical machine learning model on pseudo training data generated by two conditional autoencoders (given the semantic space as a condition): (a) the first autoencoder is trained with the visual space concatenated with the semantic space and (b) the second autoencoder is trained with the visual space as an input. Then, the decoders of both autoencoders are fed by the test data of the unseen classes to generate pseudo training data. To classify the unseen classes, the pseudo training data are combined to train a support vector machine. Tests on four different benchmark datasets show that the proposed method shows promising results compared to the current state-of-the-art when it comes to settings for both standard Zero-Shot Learning (ZSL) and Generalized Zero-Shot Learning (GZSL).
23
Vincent, Pascal. "A Connection Between Score Matching and Denoising Autoencoders." Neural Computation 23, no. 7 (2011): 1661–74. http://dx.doi.org/10.1162/neco_a_00142.
Full textAbstract:
Denoising autoencoders have been previously shown to be competitive alternatives to restricted Boltzmann machines for unsupervised pretraining of each layer of a deep architecture. We show that a simple denoising autoencoder training criterion is equivalent to matching the score (with respect to the data) of a specific energy-based model to that of a nonparametric Parzen density estimator of the data. This yields several useful insights. It defines a proper probabilistic model for the denoising autoencoder technique, which makes it in principle possible to sample from them or rank examples by their energy. It suggests a different way to apply score matching that is related to learning to denoise and does not require computing second derivatives. It justifies the use of tied weights between the encoder and decoder and suggests ways to extend the success of denoising autoencoders to a larger family of energy-based models.
24
Mujkic, Esma, Mark P. Philipsen, Thomas B. Moeslund, Martin P. Christiansen, and Ole Ravn. "Anomaly Detection for Agricultural Vehicles Using Autoencoders." Sensors 22, no. 10 (2022): 3608. http://dx.doi.org/10.3390/s22103608.
Full textAbstract:
The safe in-field operation of autonomous agricultural vehicles requires detecting all objects that pose a risk of collision. Current vision-based algorithms for object detection and classification are unable to detect unknown classes of objects. In this paper, the problem is posed as anomaly detection instead, where convolutional autoencoders are applied to identify any objects deviating from the normal pattern. Training an autoencoder network to reconstruct normal patterns in agricultural fields makes it possible to detect unknown objects by high reconstruction error. Basic autoencoder (AE), vector-quantized variational autoencoder (VQ-VAE), denoising autoencoder (DAE) and semisupervised autoencoder (SSAE) with a max-margin-inspired loss function are investigated and compared with a baseline object detector based on YOLOv5. Results indicate that SSAE with an area under the curve for precision/recall (PR AUC) of 0.9353 outperforms other autoencoder models and is comparable to an object detector with a PR AUC of 0.9794. Qualitative results show that SSAE is capable of detecting unknown objects, whereas the object detector is unable to do so and fails to identify known classes of objects in specific cases.
25
Kerner, Hannah R., Danika F. Wellington, Kiri L. Wagstaff, James F. Bell, Chiman Kwan, and Heni Ben Amor. "Novelty Detection for Multispectral Images with Application to Planetary Exploration." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 9484–91. http://dx.doi.org/10.1609/aaai.v33i01.33019484.
Full textAbstract:
In this work, we present a system based on convolutional autoencoders for detecting novel features in multispectral images. We introduce SAMMIE: Selections based on Autoencoder Modeling of Multispectral Image Expectations. Previous work using autoencoders employed the scalar reconstruction error to classify new images as novel or typical. We show that a spatial-spectral error map can enable both accurate classification of novelty in multispectral images as well as human-comprehensible explanations of the detection. We apply our methodology to the detection of novel geologic features in multispectral images of the Martian surface collected by the Mastcam imaging system on the Mars Science Laboratory Curiosity rover.
26
БОДЯНСЬКИЙ, Є. В., О. А. ВИНОКУРОВА, Д. Д. ПЕЛЕШКО, and Ю. М. РАШКЕВИЧ. "ON-LINE NEO-PHASE AUTOENKODER FOR SYSTEMS WITH DEEP LEARNING ON THE BASE OF THE KOLMOGOROV’S NEURO-PHASE NETWORK." Transport development, no. 1(1) (September 27, 2017): 60–67. http://dx.doi.org/10.33082/td.2017.1-1.06.
Full textAbstract:
One of the important problem, which is connected with big high dimensional data processing, is the task of their compression without significant loss of information that is contained in this data. The systems, which solve this problem and are called autoencoders, are the inherent part of deep neural networks. The main disadvantage of well-known autoencoders is low speed of learning process, which is implemented in the batch mode. In the paper the two-layered autoencoder is proposed. This system is the modification of Kolmogorov’s neuro-fuzzy system. Thus, in the paper the hybrid neo-fuzzy syste- mencoder is proposed that has essentially advantages comparatively with conventional neurocompressors-encoders.
27
Peng, Ching-Tung, Yung-Kuan Chan, and Shyr-Shen Yu. "Data Imbalance Immunity Bone Age Assessment System Using Independent Autoencoders." Applied Sciences 12, no. 16 (2022): 7974. http://dx.doi.org/10.3390/app12167974.
Full textAbstract:
Bone age assessment (BAA) is an important indicator of child maturity. Generally, a person is evaluated for bone age mostly during puberty stage; compared to toddlers and post-puberty stages, the data of bone age at puberty stage are much easier to obtain. As a result, the amount of bone age data collected at the toddler and post-puberty stages are often much fewer than the amount of bone age data collected at the puberty stage. This so-called data imbalance problem affects the prediction accuracy. To deal with this problem, in this paper, a data imbalance immunity bone age assessment (DIIBAA) system is proposed. It consists of two branches, the first branch consists of a CNN-based autoencoder and a CNN-based scoring network. This branch builds three autoencoders for the bone age data of toddlers, puberty, and post-puberty stages, respectively. Since the three types of autoencoders do not interfere with each other, there is no data imbalance problem in the first branch. After that, the outputs of the three autoencoders are input into the scoring network, and the autoencoder which produces the image with the highest score is regarded as the final prediction result. In the experiments, imbalanced training data with a positive and negative sample ratio of 1:2 are used, which has been alleviated compared to the original highly imbalanced data. In addition, since the scoring network converts the classification problem into an image quality scoring problem, it does not use the classification features of the image. Therefore, in the second branch, we also add the classification features to the DIIBAA system. At this time, DIIBAA considers both image quality features and classification features. Finally, the DenseNet169-based autoencoders are employed in the experiments, and the obtained evaluation accuracies are improved compared to the baseline network.
28
Liu, Zixiang. "Autoencoders and their application in removing masks." Theoretical and Natural Science 18, no. 1 (2023): 110–17. http://dx.doi.org/10.54254/2753-8818/18/20230352.
Full textAbstract:
Images are frequently distorted by noises that have a negative impact on the quality of image data. In this study, the author focuses on coping with a specific type of noise that has arisen regularly in recent years as a result of the pandemic: masks covering portions of the photographs of human faces. The paper employs the autoencoder model, which offers unsupervised learning. It compresses or encodes original data input into a smaller latent vector, then decodes it back to its original size, learning and extracting relevant features from the data in the process. In a further phase, the author employs a combination of convolutional autoencoders and denoising autoencoders, treating masks as corruptions in order to get more accurate predictions regarding the image of a human face without any covering. After training on 2,500 image pairs with and without masks and validating on 200 such image pairs, the model presented in this research achieves an overall accuracy of 93%. The research demonstrates that the combination of convolutional and denoising autoencoders is an excellent method for removing masks from facial images, and the author believes it can also be used to effectively remove other types of noise. However, the study also reveals that the picture data generated in this manner are always inferior to the original, and that the autoencoder can only process data of the same or comparable type on which it has been trained. In the future, improved models will exist to address these shortcomings and be applied to more real-life situations.
29
Tian, Ruiqi, Santiago Gomez-Rosero, and Miriam A. M. Capretz. "Health Prognostics Classification with Autoencoders for Predictive Maintenance of HVAC Systems." Energies 16, no. 20 (2023): 7094. http://dx.doi.org/10.3390/en16207094.
Full textAbstract:
Buildings’ heating, ventilation, and air-conditioning (HVAC) systems account for significant global energy use. Proper maintenance can minimize their environmental footprint and enhance the quality of the indoor environment. The adoption of Internet of Things (IoT) sensors integrated into HVAC systems has paved the way for predictive maintenance (PdM) grounded in real-time operational metrics. However, HVAC systems without such sensors cannot leverage the advantages of current data-driven PdM techniques. This work introduces a novel data-driven framework, the health prognostics classification with autoencoders (HPC-AE), designed specifically for PdM. It utilizes solely HVAC power consumption and outside temperature readings for its operations, both of which are readily obtainable. The primary objective of the HPC-AE framework is to facilitate PdM through a health prognostic approach. The HPC-AE framework utilizes an autoencoder for feature enrichment and then applies an artificial neural network to classify the daily health condition of an HVAC system. A multi-objective evaluation metric is employed to ensure optimal performance of the autoencoder within this framework. This metric evaluates the autoencoder’s proficiency in reducing reconstruction discrepancies in standard data conditions and its capability to differentiate between standard and degraded data scenarios. The HPC-AE framework is validated in two HVAC fault scenarios, including a clogged air filter and air duct leakage. The experimental results show that compared to methods used in similar studies, HPC-AE exhibits a 5.7% and 2.1% increase in the F1 score for the clogged air filter and duct leakage scenarios.
30
Cardoso Pereira, Ricardo, Miriam Seoane Santos, Pedro Pereira Rodrigues, and Pedro Henriques Abreu. "Reviewing Autoencoders for Missing Data Imputation: Technical Trends, Applications and Outcomes." Journal of Artificial Intelligence Research 69 (December 14, 2020): 1255–85. http://dx.doi.org/10.1613/jair.1.12312.
Full textAbstract:
Missing data is a problem often found in real-world datasets and it can degrade the performance of most machine learning models. Several deep learning techniques have been used to address this issue, and one of them is the Autoencoder and its Denoising and Variational variants. These models are able to learn a representation of the data with missing values and generate plausible new ones to replace them. This study surveys the use of Autoencoders for the imputation of tabular data and considers 26 works published between 2014 and 2020. The analysis is mainly focused on discussing patterns and recommendations for the architecture, hyperparameters and training settings of the network, while providing a detailed discussion of the results obtained by Autoencoders when compared to other state-of-the-art methods, and of the data contexts where they have been applied. The conclusions include a set of recommendations for the technical settings of the network, and show that Denoising Autoencoders outperform their competitors, particularly the often used statistical methods.
31
Schreiber, Jens, and Bernhard Sick. "Multi-Task Autoencoders and Transfer Learning for Day-Ahead Wind and Photovoltaic Power Forecasts." Energies 15, no. 21 (2022): 8062. http://dx.doi.org/10.3390/en15218062.
Full textAbstract:
Integrating new renewable energy resources requires robust and reliable forecasts to ensure a stable electrical grid and avoid blackouts. Sophisticated representation learning techniques, such as autoencoders, play an essential role, as they allow for the extraction of latent features to forecast the expected generated wind and photovoltaic power for the next seconds up to days. Thereby, autoencoders reduce the required training time and the time spent in manual feature engineering and often improve the forecast error. However, most current renewable energy forecasting research on autoencoders focuses on smaller forecast horizons for the following seconds and hours based on meteorological measurements. At the same time, larger forecast horizons, such as day-ahead power forecasts based on numerical weather predictions, are crucial for planning loads and demands within the electrical grid to prevent power failures. There is little evidence on the ability of autoencoders and their respective forecasting models to improve through multi-task learning and time series autoencoders for day-ahead power forecasts. We can close these gaps by proposing a multi-task learning autoencoder based on the recently introduced temporal convolution network. This approach reduces the number of trainable parameters by 38 for photovoltaic data and 202 for wind data while having the best reconstruction error compared to nine other representation learning techniques. At the same time, this model decreases the day-ahead forecast error up to 18.3% for photovoltaic parks and 1.5% for wind parks. We round off these results by analyzing the influences of the latent size and the number of layers to fine-tune the encoder for wind and photovoltaic power forecasts.
32
Verma, Sonu Kumar, Purushotam Soudagar, Pankaj Kunekar, et al. "Noise Reduction in Images Using Autoencoders." International Journal for Research in Applied Science and Engineering Technology 10, no. 12 (2022): 1732–36. http://dx.doi.org/10.22214/ijraset.2022.48306.
Full textAbstract:
Abstract: Ideally, the signals which are pure can exist only on paper. As there are some techniques for denoising the provided signal up to some degree, so procedure during that time it is important that such techniques must be reconcilable with the most of the devices. This article describes a for denoising with the help of an autoencoder using image processing technique and algorithms which are based on deep learning. With the aid of autoencoders, noise reduction is not accomplished using a conventional method in which the output signal is essentially the same signal that was used as an input previously. Here the main focus remains originality as the autoencoder follows a back propagation process It is one of the approaches that focuses on the techniques described in this article are interchangeable. i.e., Working for any signal and having, reliability, efficient Ness and compatibility with more devices.
33
Soydaner, Derya. "Hyper Autoencoders." Neural Processing Letters 52, no. 2 (2020): 1395–413. http://dx.doi.org/10.1007/s11063-020-10310-y.
Full text
34
Odunga, Nelson Ochieng, Ronald Waweru Mwangi, and Ismail Ateya Lukandu. "Reducing Generalization Error Using Autoencoders for The Detection of Computer Worms." Computer Engineering and Applications Journal 9, no. 3 (2020): 175–82. http://dx.doi.org/10.18495/comengapp.v9i3.348.
Full textAbstract:
This paper discusses computer worm detection using machine learning. More specifically, the generalization capability of autoencoders is investigated and improved using regularization and deep autoencoders. Models are constructed first without autoencoders and thereafter with autoencoders. The models with autoencoders are further improved using regularization and deep autoencoders. Results show an improved in the capability of models to generalize well to new examples.
35
Akkari, Nissrine, Fabien Casenave, Elie Hachem, and David Ryckelynck. "A Bayesian Nonlinear Reduced Order Modeling Using Variational AutoEncoders." Fluids 7, no. 10 (2022): 334. http://dx.doi.org/10.3390/fluids7100334.
Full textAbstract:
This paper presents a new nonlinear projection based model reduction using convolutional Variational AutoEncoders (VAEs). This framework is applied on transient incompressible flows. The accuracy is obtained thanks to the expression of the velocity and pressure fields in a nonlinear manifold maximising the likelihood on pre-computed data in the offline stage. A confidence interval is obtained for each time instant thanks to the definition of the reduced dynamic coefficients as independent random variables for which the posterior probability given the offline data is known. The parameters of the nonlinear manifold are optimized as the ones of the decoder layers of an autoencoder. The parameters of the conditional posterior probability of the reduced coefficients are the ones of the encoder layers of the same autoencoder. The optimization of both sets of the encoder and the decoder parameters is obtained thanks to the application of a variational Bayesian method, leading to variational autoencoders. This Reduced Order Model (ROM) is not a regression model over the offline pre-computed data. The numerical resolution of the ROM is based on the Chorin projection method. We apply this new nonlinear projection-based Reduced Order Modeling (ROM) for a 2D Karman Vortex street flow and a 3D incompressible and unsteady flow in an aeronautical injection system.
36
Alves de Oliveira, Vinicius, Marie Chabert, Thomas Oberlin, et al. "Reduced-Complexity End-to-End Variational Autoencoder for on Board Satellite Image Compression." Remote Sensing 13, no. 3 (2021): 447. http://dx.doi.org/10.3390/rs13030447.
Full textAbstract:
Recently, convolutional neural networks have been successfully applied to lossy image compression. End-to-end optimized autoencoders, possibly variational, are able to dramatically outperform traditional transform coding schemes in terms of rate-distortion trade-off; however, this is at the cost of a higher computational complexity. An intensive training step on huge databases allows autoencoders to learn jointly the image representation and its probability distribution, possibly using a non-parametric density model or a hyperprior auxiliary autoencoder to eliminate the need for prior knowledge. However, in the context of on board satellite compression, time and memory complexities are submitted to strong constraints. The aim of this paper is to design a complexity-reduced variational autoencoder in order to meet these constraints while maintaining the performance. Apart from a network dimension reduction that systematically targets each parameter of the analysis and synthesis transforms, we propose a simplified entropy model that preserves the adaptability to the input image. Indeed, a statistical analysis performed on satellite images shows that the Laplacian distribution fits most features of their representation. A complex non parametric distribution fitting or a cumbersome hyperprior auxiliary autoencoder can thus be replaced by a simple parametric estimation. The proposed complexity-reduced autoencoder outperforms the Consultative Committee for Space Data Systems standard (CCSDS 122.0-B) while maintaining a competitive performance, in terms of rate-distortion trade-off, in comparison with the state-of-the-art learned image compression schemes.
37
Lin, Haicai, Ruixia Liu, and Zhaoyang Liu. "ECG Signal Denoising Method Based on Disentangled Autoencoder." Electronics 12, no. 7 (2023): 1606. http://dx.doi.org/10.3390/electronics12071606.
Full textAbstract:
The electrocardiogram (ECG) is widely used in medicine because it can provide basic information about different types of heart disease. However, ECG data are usually disturbed by various types of noise, which can lead to errors in diagnosis by doctors. To address this problem, this study proposes a method for denoising ECG based on disentangled autoencoders. A disentangled autoencoder is an improved autoencoder suitable for denoising ECG data. In our proposed method, we use a disentangled autoencoder model based on a fully convolutional neural network to effectively separate the clean ECG data from the noise. Unlike conventional autoencoders, we disentangle the features of the coding hidden layer to separate the signal-coding features from the noise-coding features. We performed simulation experiments on the MIT-BIH Arrhythmia Database and found that the algorithm had better noise reduction results when dealing with four different types of noise. In particular, using our method, the average improved signal-to-noise ratios for the three noises in the MIT-BIH Noise Stress Test Database were 27.45 db for baseline wander, 25.72 db for muscle artefacts, and 29.91 db for electrode motion artefacts. Compared to a denoising autoencoder based on a fully convolutional neural network (FCN), the signal-to-noise ratio was improved by an average of 12.57%. We can conclude that the model has scientific validity. At the same time, our noise reduction method can effectively remove noise while preserving the important information conveyed by the original signal.
38
Haga, Takeshi, Hiroshi Kera, and Kazuhiko Kawamoto. "Sequential Variational Autoencoder with Adversarial Classifier for Video Disentanglement." Sensors 23, no. 5 (2023): 2515. http://dx.doi.org/10.3390/s23052515.
Full textAbstract:
In this paper, we propose a sequential variational autoencoder for video disentanglement, which is a representation learning method that can be used to separately extract static and dynamic features from videos. Building sequential variational autoencoders with a two-stream architecture induces inductive bias for video disentanglement. However, our preliminary experiment demonstrated that the two-stream architecture is insufficient for video disentanglement because static features frequently contain dynamic features. Additionally, we found that dynamic features are not discriminative in the latent space. To address these problems, we introduced an adversarial classifier using supervised learning into the two-stream architecture. The strong inductive bias through supervision separates dynamic features from static features and yields discriminative representations of the dynamic features. Through a comparison with other sequential variational autoencoders, we qualitatively and quantitatively demonstrate the effectiveness of the proposed method on the Sprites and MUG datasets.
39
Hlihor, Petru, Riccardo Volpi, and Luigi Malagò. "Evaluating the Robustness of Defense Mechanisms based on AutoEncoder Reconstructions against Carlini-Wagner Adversarial Attacks." Proceedings of the Northern Lights Deep Learning Workshop 1 (February 6, 2020): 6. http://dx.doi.org/10.7557/18.5173.
Full textAbstract:
Adversarial Examples represent a serious problem affecting the security of machine learning systems. In this paper we focus on a defense mechanism based on reconstructing images before classification using an autoencoder. We experiment on several types of autoencoders and evaluate the impact of strategies such as injecting noise in the input during training and in the latent space at inference time.We tested the models on adversarial examples generated with the Carlini-Wagner attack, in a white-box scenario and on the stacked system composed by the autoencoder and the classifier.
40
Havrylovych, Mariia, and Valeriy Danylov. "Research on hybrid transformer-based autoencoders for user biometric verification." System research and information technologies, no. 3 (September 29, 2023): 42–53. http://dx.doi.org/10.20535/srit.2308-8893.2023.3.03.
Full textAbstract:
Our current study extends previous work on motion-based biometric verification using sensory data by exploring new architectures and more complex input from various sensors. Biometric verification offers advantages like uniqueness and protection against fraud. The state-of-the-art transformer architecture in AI is known for its attention block and applications in various fields, including NLP and CV. We investigated its potential value for applications involving sensory data. The research proposes a hybrid architecture, integrating transformer attention blocks with different autoencoders, to evaluate its efficacy for biometric verification and user authentication. Various configurations were compared, including LSTM autoencoder, transformer autoencoder, LSTM VAE, and transformer VAE. Results showed that combining transformer blocks with an undercomplete deterministic autoencoder yields the best performance, but model performance is significantly influenced by data preprocessing and configuration parameters. The application of transformers for biometric verification and sensory data appears promising, performing on par with or surpassing LSTM-based models but with lower inference and training time.
41
Refinetti, Maria, and Sebastian Goldt. "The dynamics of representation learning in shallow, non-linear autoencoders *." Journal of Statistical Mechanics: Theory and Experiment 2023, no. 11 (2023): 114010. http://dx.doi.org/10.1088/1742-5468/ad01af.
Full textAbstract:
Abstract Autoencoders are the simplest neural network for unsupervised learning, and thus an ideal framework for studying feature learning. While a detailed understanding of the dynamics of linear autoencoders has recently been obtained, the study of non-linear autoencoders has been hindered by the technical difficulty of handling training data with non-trivial correlations—a fundamental prerequisite for feature extraction. Here, we study the dynamics of feature learning in non-linear, shallow autoencoders. We derive a set of asymptotically exact equations that describe the generalisation dynamics of autoencoders trained with stochastic gradient descent (SGD) in the limit of high-dimensional inputs. These equations reveal that autoencoders learn the leading principal components of their inputs sequentially. An analysis of the long-time dynamics explains the failure of sigmoidal autoencoders to learn with tied weights, and highlights the importance of training the bias in ReLU autoencoders. Building on previous results for linear networks, we analyse a modification of the vanilla SGD algorithm, which allows learning of the exact principal components. Finally, we show that our equations accurately describe the generalisation dynamics of non-linear autoencoders trained on realistic datasets such as CIFAR10, thus establishing shallow autoencoders as an instance of the recently observed Gaussian universality.
42
Guo, Tianyu, Jianxin Liu, and Zhenwei Guo. "Compression and Reconstruction of Magnetotelluric Data Based on Convolutional Neural Network." Journal of Physics: Conference Series 2651, no. 1 (2023): 012122. http://dx.doi.org/10.1088/1742-6596/2651/1/012122.
Full textAbstract:
Abstract Magnetotelluric method is one of the important geophysical methods, and its signal acquisition requires more stacking times and longer stacking time. With the development of instruments, the acquisition time becomes longer and the amount of data becomes larger, which brings new challenges to data storage and transmission. Aiming at the above problems, a compression and reconstruction technology of magnetotelluric time series based on convolutional neural network is proposed. This paper introduces two convolutional autoencoders based on convolutional neural networks, which can effectively compress data space, improve transmission efficiency, and have high data reconstruction accuracy. Using the measured magnetotelluric time series, two autoencoder models based on convolutional networks are verified in this paper, which proves the feasibility of convolutional autoencoders in magnetotelluric data compression; the results show that model 2 can better reproduce It constructs the magnetotelluric time series, and has high training efficiency and good generalization ability.
43
Zhang, Lisa, Pouria Fewzee, and Charbel Feghali. "AI education matters." AI Matters 7, no. 3 (2021): 18–20. http://dx.doi.org/10.1145/3511322.3511327.
Full textAbstract:
We introduce a Model AI Assignment (Neller et al., 2021) where students combine various techniques from a deep learning course to build a denoising autoencoder (Shen, Mueller, Barzilay, & Jaakkola, 2020) for news headlines. Students then use this denoising autoencoder to query similar headlines, and interpolate between headlines. Building this denoising autoencoder requires students to apply many course concepts, including data augmentation, word and sentence embeddings, autoencoders, recurrent neural networks, sequence-to-sequence networks, and temperature. As such, this assignment can be ideal as a final assessment that synthesizes many topics. This assignment is written in PyTorch, uses the torchtext package, and is intended to be completed on the Google Colab platform.
44
Tan HP, Nguyen, Bang Le Thanh, Thanh-Nha To, et al. "Performance Evaluation of Single-Carrier and Orthogonal Frequency Divison Multiplexing-Based Autoencoders in Comparison with Low-Density Parity-Check Encoder." Electronics 12, no. 18 (2023): 3945. http://dx.doi.org/10.3390/electronics12183945.
Full textAbstract:
Recently, the growing demands for ultra-high speed applications require more advanced and optimal data transmission techniques. Wireless autoencoders have gained significant attention since they provide global optimization of the transceiver structure. This article explores the application of autoencoders to enhance the performance of wireless communication systems. It provides the performance evaluation of the systems using single-carrier and OFDM-based autoencoders under the conditions of AWGN and fading channels. Then, in terms of the BLER metric, the wireless systems with autoencoders are compared with conventional systems using LDPC coding and quadrature amplitude modulation for various configurations. Simulation results indicate that for high-modulation orders (QAM-64 or QAM-256), communication systems employing autoencoders provide superior performance compared to systems using LDPC channel encoding in regions with a low signal-to-noise (SNR) ratio. Specifically, a gain of 1–2 dB in signal power is obtained for single-carrier autoencoders and 0.3–2 dB is obtained for OFDM-based autoencoders. Therefore, wireless communication systems utilizing autoencoders can be considered as a promising candidate for future wireless communication systems.
45
Torti, Emanuele, Alessandro Fontanella, Antonio Plaza, Javier Plaza, and Francesco Leporati. "Hyperspectral Image Classification Using Parallel Autoencoding Diabolo Networks on Multi-Core and Many-Core Architectures." Electronics 7, no. 12 (2018): 411. http://dx.doi.org/10.3390/electronics7120411.
Full textAbstract:
One of the most important tasks in hyperspectral imaging is the classification of the pixels in the scene in order to produce thematic maps. This problem can be typically solved through machine learning techniques. In particular, deep learning algorithms have emerged in recent years as a suitable methodology to classify hyperspectral data. Moreover, the high dimensionality of hyperspectral data, together with the increasing availability of unlabeled samples, makes deep learning an appealing approach to process and interpret those data. However, the limited number of labeled samples often complicates the exploitation of supervised techniques. Indeed, in order to guarantee a suitable precision, a large number of labeled samples is normally required. This hurdle can be overcome by resorting to unsupervised classification algorithms. In particular, autoencoders can be used to analyze a hyperspectral image using only unlabeled data. However, the high data dimensionality leads to prohibitive training times. In this regard, it is important to realize that the operations involved in autoencoders training are intrinsically parallel. Therefore, in this paper we present an approach that exploits multi-core and many-core devices in order to achieve efficient autoencoders training in hyperspectral imaging applications. Specifically, in this paper, we present new OpenMP and CUDA frameworks for autoencoder training. The obtained results show that the CUDA framework provides a speed-up of about two orders of magnitudes as compared to an optimized serial processing chain.
46
Wu, Lihao, and Jiahui Liang. "Anomaly detection based on temporal convolution Autoencoders." Journal of Physics: Conference Series 2366, no. 1 (2022): 012041. http://dx.doi.org/10.1088/1742-6596/2366/1/012041.
Full textAbstract:
Abstract Due to the rapid growth of the number of sensors in modern industrial society, detecting outliers in time series has become very important. And unsupervised detection of outliers in time series is a very challenging task. For most detection tasks of time series outliers, the autoencoder is one of the main choices. And the recursive network is usually used in the self-coding network structure. However, recent studies have shown that the network structure using dilated causal convolution performs better than the recursive network in all kinds of sequence modeling. In this paper, the encoder and decoder are network structures based on extended causal convolution The time series are reconstructed and then compared with the original data to calculate the distance between them, so as to identify the outliers. We use the Temporal convolution autoencoders to evaluate anomaly data sets in multiple time series. Our results show that the Temporal convolution autoencoders has better anomaly detection ability. In addition, we learned that the combination of Feature Engineering and super parameters will also have a great impact on the results, so ablation experiments need to be carried out carefully.
47
Nieto-Mora, Daniel Alexis, Maria Cristina Ferreira de Oliveira, Camilo Sanchez-Giraldo, Leonardo Duque-Muñoz, Claudia Isaza-Narváez, and Juan David Martínez-Vargas. "Soundscape Characterization Using Autoencoders and Unsupervised Learning." Sensors 24, no. 8 (2024): 2597. http://dx.doi.org/10.3390/s24082597.
Full textAbstract:
Passive acoustic monitoring (PAM) through acoustic recorder units (ARUs) shows promise in detecting early landscape changes linked to functional and structural patterns, including species richness, acoustic diversity, community interactions, and human-induced threats. However, current approaches primarily rely on supervised methods, which require prior knowledge of collected datasets. This reliance poses challenges due to the large volumes of ARU data. In this work, we propose a non-supervised framework using autoencoders to extract soundscape features. We applied this framework to a dataset from Colombian landscapes captured by 31 audiomoth recorders. Our method generates clusters based on autoencoder features and represents cluster information with prototype spectrograms using centroid features and the decoder part of the neural network. Our analysis provides valuable insights into the distribution and temporal patterns of various sound compositions within the study area. By utilizing autoencoders, we identify significant soundscape patterns characterized by recurring and intense sound types across multiple frequency ranges. This comprehensive understanding of the study area’s soundscape allows us to pinpoint crucial sound sources and gain deeper insights into its acoustic environment. Our results encourage further exploration of unsupervised algorithms in soundscape analysis as a promising alternative path for understanding and monitoring environmental changes.
48
Kanász, Róbert, Peter Gnip, Martin Zoričák, and Peter Drotár. "Bankruptcy prediction using ensemble of autoencoders optimized by genetic algorithm." PeerJ Computer Science 9 (June 8, 2023): e1257. http://dx.doi.org/10.7717/peerj-cs.1257.
Full textAbstract:
The prediction of imminent bankruptcy for a company is important to banks, government agencies, business owners, and different business stakeholders. Bankruptcy is influenced by many global and local aspects, so it can hardly be anticipated without deeper analysis and economic modeling knowledge. To make this problem even more challenging, the available bankruptcy datasets are usually imbalanced since even in times of financial crisis, bankrupt companies constitute only a fraction of all operating businesses. In this article, we propose a novel bankruptcy prediction approach based on a shallow autoencoder ensemble that is optimized by a genetic algorithm. The goal of the autoencoders is to learn the distribution of the majority class: going concern businesses. Then, the bankrupt companies are represented by higher autoencoder reconstruction errors. The choice of the optimal threshold value for the reconstruction error, which is used to differentiate between bankrupt and nonbankrupt companies, is crucial and determines the final classification decision. In our approach, the threshold for each autoencoder is determined by a genetic algorithm. We evaluate the proposed method on four different datasets containing small and medium-sized enterprises. The results show that the autoencoder ensemble is able to identify bankrupt companies with geometric mean scores ranging from 71% to 93.7%, (depending on the industry and evaluation year).
49
Serradilla, Oscar, Ekhi Zugasti, Julian Ramirez de Okariz, Jon Rodriguez, and Urko Zurutuza. "Adaptable and Explainable Predictive Maintenance: Semi-Supervised Deep Learning for Anomaly Detection and Diagnosis in Press Machine Data." Applied Sciences 11, no. 16 (2021): 7376. http://dx.doi.org/10.3390/app11167376.
Full textAbstract:
Predictive maintenance (PdM) has the potential to reduce industrial costs by anticipating failures and extending the work life of components. Nowadays, factories are monitoring their assets and most collected data belong to correct working conditions. Thereby, semi-supervised data-driven models are relevant to enable PdM application by learning from assets’ data. However, their main challenges for application in industry are achieving high accuracy on anomaly detection, diagnosis of novel failures, and adaptability to changing environmental and operational conditions (EOC). This article aims to tackle these challenges, experimenting with algorithms in press machine data of a production line. Initially, state-of-the-art and classic data-driven anomaly detection model performance is compared, including 2D autoencoder, null-space, principal component analysis (PCA), one-class support vector machines (OC-SVM), and extreme learning machine (ELM) algorithms. Then, diagnosis tools are developed supported on autoencoder’s latent space feature vector, including clustering and projection algorithms to cluster data of synthetic failure types semi-supervised. In addition, explainable artificial intelligence techniques have enabled to track the autoencoder’s loss with input data to detect anomalous signals. Finally, transfer learning is applied to adapt autoencoders to changing EOC data of the same process. The data-driven techniques used in this work can be adapted to address other industrial use cases, helping stakeholders gain trust and thus promote the adoption of data-driven PdM systems in smart factories.
50
Yu, Xinran. "Autoencoder combined with the multilayer perceptron for Alzheimers disease classification." Applied and Computational Engineering 19, no. 1 (2023): 139–45. http://dx.doi.org/10.54254/2755-2721/19/20231022.
Full textAbstract:
Alzheimer's disease (AD) is a prevalent neurodegenerative disorder that poses significant challenges for accurate diagnosis and treatment. The classification of AD Neurofibrillary Changes (ADNC) levels is crucial for understanding disease progression and developing effective interventions. In this paper, a method was proposed for classifying ADNC levels based on single-cell RNA sequencing (scRNA-seq) data obtained from the SEA-AD dataset. An autoencoder was employed to reduce the dimensionality of the scRNA-seq data, followed by a Multilayer Perceptron (MLP) for classification based on the autoencoder's embedding. The autoencoder effectively reduces the dimension of the scRNA-seq data from 4344 to 30 features. However, the embedding does not exhibit clear boundaries between different ADNC levels. The MLP model achieves a classification accuracy of 39% on the ADNC levels, indicating the complexity of the task and the need for more advanced classification methods. Additionally, the overfitting in both models was observed, and dropout regularization is applied to mitigate this issue. While the results indicate the potential of feature extraction and dimensionality reduction using autoencoders, the accuracy of ADNC level classification remains limited. Combining multiple approaches and aspects in AD diagnosis is necessary, as RNA-seq data alone may not be sufficient for accurate prediction. Future work could explore more sophisticated classification algorithms to improve the accuracy of ADNC level classification and consider integrating other data modalities to enhance disease diagnosis and understanding.