Artigos de revistas sobre o tema "Hyperparameter search"
Siga este link para ver outros tipos de publicações sobre o tema: Hyperparameter search.
Crie uma referência precisa em APA, MLA, Chicago, Harvard, e outros estilos
Veja os 50 melhores artigos de revistas para estudos sobre o assunto "Hyperparameter search".
Ao lado de cada fonte na lista de referências, há um botão "Adicionar à bibliografia". Clique e geraremos automaticamente a citação bibliográfica do trabalho escolhido no estilo de citação de que você precisa: APA, MLA, Harvard, Chicago, Vancouver, etc.
Você também pode baixar o texto completo da publicação científica em formato .pdf e ler o resumo do trabalho online se estiver presente nos metadados.
Veja os artigos de revistas das mais diversas áreas científicas e compile uma bibliografia correta.
1
Florea, Adrian-Catalin, e Razvan Andonie. "Weighted Random Search for Hyperparameter Optimization". International Journal of Computers Communications & Control 14, n.º 2 (14 de abril de 2019): 154–69. http://dx.doi.org/10.15837/ijccc.2019.2.3514.
Resumo:
We introduce an improved version of Random Search (RS), used here for hyperparameter optimization of machine learning algorithms. Unlike the standard RS, which generates for each trial new values for all hyperparameters, we generate new values for each hyperparameter with a probability of change. The intuition behind our approach is that a value that already triggered a good result is a good candidate for the next step, and should be tested in new combinations of hyperparameter values. Within the same computational budget, our method yields better results than the standard RS. Our theoretical results prove this statement. We test our method on a variation of one of the most commonly used objective function for this class of problems (the Grievank function) and for the hyperparameter optimization of a deep learning CNN architecture. Our results can be generalized to any optimization problem dened on a discrete domain.
2
Ghawi, Raji, e Jürgen Pfeffer. "Efficient Hyperparameter Tuning with Grid Search for Text Categorization using kNN Approach with BM25 Similarity". Open Computer Science 9, n.º 1 (8 de agosto de 2019): 160–80. http://dx.doi.org/10.1515/comp-2019-0011.
Resumo:
AbstractIn machine learning, hyperparameter tuning is the problem of choosing a set of optimal hyperparameters for a learning algorithm. Several approaches have been widely adopted for hyperparameter tuning, which is typically a time consuming process. We propose an efficient technique to speed up the process of hyperparameter tuning with Grid Search. We applied this technique on text categorization using kNN algorithm with BM25 similarity, where three hyperparameters need to be tuned. Our experiments show that our proposed technique is at least an order of magnitude faster than conventional tuning.
3
Yang, Eun-Suk, Jong Dae Kim, Chan-Young Park, Hye-Jeong Song e Yu-Seop Kim. "Hyperparameter tuning for hidden unit conditional random fields". Engineering Computations 34, n.º 6 (7 de agosto de 2017): 2054–62. http://dx.doi.org/10.1108/ec-11-2015-0350.
Resumo:
Purpose In this paper, the problem of a nonlinear model – specifically the hidden unit conditional random fields (HUCRFs) model, which has binary stochastic hidden units between the data and the labels – exhibiting unstable performance depending on the hyperparameter under consideration. Design/methodology/approach There are three main optimization search methods for hyperparameter tuning: manual search, grid search and random search. This study shows that HUCRFs’ unstable performance depends on the hyperparameter values used and its performance is based on tuning that draws on grid and random searches. All experiments conducted used the n-gram features – specifically, unigram, bigram, and trigram. Findings Naturally, selecting a list of hyperparameter values based on a researchers’ experience to find a set in which the best performance is exhibited is better than finding it from a probability distribution. Realistically, however, it is impossible to calculate using the parameters in all combinations. The present research indicates that the random search method has a better performance compared with the grid search method while requiring shorter computation time and a reduced cost. Originality/value In this paper, the issues affecting the performance of HUCRF, a nonlinear model with performance that varies depending on the hyperparameters, but performs better than CRF, has been examined.
4
Wen, Long, Xingchen Ye e Liang Gao. "A new automatic machine learning based hyperparameter optimization for workpiece quality prediction". Measurement and Control 53, n.º 7-8 (21 de julho de 2020): 1088–98. http://dx.doi.org/10.1177/0020294020932347.
Resumo:
Workpiece quality prediction is very important in modern manufacturing industry. However, traditional machine learning methods are very sensitive to their hyperparameters, making the tuning of the machine learning methods essential to improve the prediction performance. Hyperparameter optimization (HPO) approaches are applied attempting to tune hyperparameters, such as grid search and random search. However, the hyperparameters space for workpiece quality prediction model is high dimension and it consists with continuous, combinational and conditional types of hyperparameters, which is difficult to be tuned. In this article, a new automatic machine learning based HPO, named adaptive Tree Pazen Estimator (ATPE), is proposed for workpiece quality prediction in high dimension. In the proposed method, it can iteratively search the best combination of hyperparameters in the automatic way. During the warm-up process for ATPE, it can adaptively adjust the hyperparameter interval to guide the search. The proposed ATPE is tested on sparse stack autoencoder based MNIST and XGBoost based WorkpieceQuality dataset, and the results show that ATPE provides the state-of-the-art performances in high-dimensional space and can search the hyperparameters in reasonable range by comparing with Tree Pazen Estimator, annealing, and random search, showing its potential in the field of workpiece quality prediction.
5
Hinz, Tobias, Nicolás Navarro-Guerrero, Sven Magg e Stefan Wermter. "Speeding up the Hyperparameter Optimization of Deep Convolutional Neural Networks". International Journal of Computational Intelligence and Applications 17, n.º 02 (junho de 2018): 1850008. http://dx.doi.org/10.1142/s1469026818500086.
Resumo:
Most learning algorithms require the practitioner to manually set the values of many hyperparameters before the learning process can begin. However, with modern algorithms, the evaluation of a given hyperparameter setting can take a considerable amount of time and the search space is often very high-dimensional. We suggest using a lower-dimensional representation of the original data to quickly identify promising areas in the hyperparameter space. This information can then be used to initialize the optimization algorithm for the original, higher-dimensional data. We compare this approach with the standard procedure of optimizing the hyperparameters only on the original input. We perform experiments with various state-of-the-art hyperparameter optimization algorithms such as random search, the tree of parzen estimators (TPEs), sequential model-based algorithm configuration (SMAC), and a genetic algorithm (GA). Our experiments indicate that it is possible to speed up the optimization process by using lower-dimensional data representations at the beginning, while increasing the dimensionality of the input later in the optimization process. This is independent of the underlying optimization procedure, making the approach promising for many existing hyperparameter optimization algorithms.
6
Yang, Zeshi, e Zhiqi Yin. "Efficient Hyperparameter Optimization for Physics-based Character Animation". Proceedings of the ACM on Computer Graphics and Interactive Techniques 4, n.º 1 (26 de abril de 2021): 1–19. http://dx.doi.org/10.1145/3451254.
Resumo:
Physics-based character animation has seen significant advances in recent years with the adoption of Deep Reinforcement Learning (DRL). However, DRL-based learning methods are usually computationally expensive and their performance crucially depends on the choice of hyperparameters. Tuning hyperparameters for these methods often requires repetitive training of control policies, which is even more computationally prohibitive. In this work, we propose a novel Curriculum-based Multi-Fidelity Bayesian Optimization framework (CMFBO) for efficient hyperparameter optimization of DRL-based character control systems. Using curriculum-based task difficulty as fidelity criterion, our method improves searching efficiency by gradually pruning search space through evaluation on easier motor skill tasks. We evaluate our method on two physics-based character control tasks: character morphology optimization and hyperparameter tuning of DeepMimic. Our algorithm significantly outperforms state-of-the-art hyperparameter optimization methods applicable for physics-based character animation. In particular, we show that hyperparameters optimized through our algorithm result in at least 5x efficiency gain comparing to author-released settings in DeepMimic.
7
Han, Junjie, Cedric Gondro e Juan Steibel. "98 Using differential evolution to improve predictive accuracy of deep learning models applied to pig production data". Journal of Animal Science 98, Supplement_3 (2 de novembro de 2020): 27. http://dx.doi.org/10.1093/jas/skaa054.048.
Resumo:
Abstract Deep learning (DL) is being used for prediction in precision livestock farming and in genomic prediction. However, optimizing hyperparameters in DL models is critical for their predictive performance. Grid search is the traditional approach to select hyperparameters in DL, but it requires exhaustive search over the parameter space. We propose hyperparameter selection using differential evolution (DE), which is a heuristic algorithm that does not require exhaustive search. The goal of this study was to design and apply DE to optimize hyperparameters of DL models for genomic prediction and image analysis in pig production systems. One dataset consisted of 910 pigs genotyped with 28,916 SNP markers to predict their post-mortem meat pH. Another dataset consisted of 1,334 images of pigs eating inside a single-spaced feeder classified as: “single pig” or “multiple pigs.” The accuracy of genomic prediction was defined as the correlation between the predicted pH and the observed pH. The image classification prediction accuracy was the proportion of correctly classified images. For genomic prediction, a multilayer perceptron (MLP) was optimized. For image classification, MLP and convolutional neural networks (CNN) were optimized. For genomic prediction, the initial hyperparameter set resulted in an accuracy of 0.032 and for image classification, the initial accuracy was between 0.72 and 0.76. After optimization using DE, the genomic prediction accuracy was 0.3688 compared to 0.334 using GBLUP. The top selected models included one layer, 60 neurons, sigmoid activation and L2 penalty = 0.3. The accuracy of image classification after optimization was between 0.89 and 0.92. Selected models included three layers, adamax optimizer and relu or elu activation for the MLP, and one layer, 64 filters and 5×5 filter size for the CNN. DE can adapt the hyperparameter selection to each problem, dataset and model, and it significantly increased prediction accuracy with minimal user input.
8
Tsai, Chun-Wei, e Zhi-Yan Fang. "An Effective Hyperparameter Optimization Algorithm for DNN to Predict Passengers at a Metro Station". ACM Transactions on Internet Technology 21, n.º 2 (30 de março de 2021): 1–24. http://dx.doi.org/10.1145/3410156.
Resumo:
As one of the public transportation systems, metro is certainly an indispensable part in urban areas of a metropolis today. Several successful results have shown that deep learning technologies might provide an effective way to predict the number of passengers at a metro station. However, most information systems based on deep learning technologies are usually designed and tuned manually by using domain knowledge and trial-and-error; thus, how to find out a set of suitable hyperparameters for a deep neural network (DNN) has become a critical research issue. To deal with the problem of hyperparameter setting for a DNN in solving the prediction of passengers at a metro station, a novel metaheuristic algorithm called search economics for hyperparameter optimization is presented to improve the accuracy rate of such a prediction system. The basic idea of the proposed algorithm is to divide the solution space into a set of subspaces first and then assign a different number of search agents to each subspace based on the “potential of each subspace.” The potential is estimated based on the objective values of the searched solutions, the objective values of the probe solutions, and the computation time invested in each subspace. The proposed method is compared with Bayesian, random forest, support vector regression, DNN, and DNN with different hyperparameter search algorithms, namely, grid search, simulated annealing, and particle swarm optimization. The simulation results using the data provided by the government of Taipei city, Taiwan, indicate that the proposed method outperforms all the other forecasting methods compared in this article in terms of the mean absolute percentage error.
9
Contreras, Pablo, Johanna Orellana-Alvear, Paul Muñoz, Jörg Bendix e Rolando Célleri. "Influence of Random Forest Hyperparameterization on Short-Term Runoff Forecasting in an Andean Mountain Catchment". Atmosphere 12, n.º 2 (10 de fevereiro de 2021): 238. http://dx.doi.org/10.3390/atmos12020238.
Resumo:
The Random Forest (RF) algorithm, a decision-tree-based technique, has become a promising approach for applications addressing runoff forecasting in remote areas. This machine learning approach can overcome the limitations of scarce spatio-temporal data and physical parameters needed for process-based hydrological models. However, the influence of RF hyperparameters is still uncertain and needs to be explored. Therefore, the aim of this study is to analyze the sensitivity of RF runoff forecasting models of varying lead time to the hyperparameters of the algorithm. For this, models were trained by using (a) default and (b) extensive hyperparameter combinations through a grid-search approach that allow reaching the optimal set. Model performances were assessed based on the R2, %Bias, and RMSE metrics. We found that: (i) The most influencing hyperparameter is the number of trees in the forest, however the combination of the depth of the tree and the number of features hyperparameters produced the highest variability-instability on the models. (ii) Hyperparameter optimization significantly improved model performance for higher lead times (12- and 24-h). For instance, the performance of the 12-h forecasting model under default RF hyperparameters improved to R2 = 0.41 after optimization (gain of 0.17). However, for short lead times (4-h) there was no significant model improvement (0.69 < R2 < 0.70). (iii) There is a range of values for each hyperparameter in which the performance of the model is not significantly affected but remains close to the optimal. Thus, a compromise between hyperparameter interactions (i.e., their values) can produce similar high model performances. Model improvements after optimization can be explained from a hydrological point of view, the generalization ability for lead times larger than the concentration time of the catchment tend to rely more on hyperparameterization than in what they can learn from the input data. This insight can help in the development of operational early warning systems.
10
Jervis, Michael, Mingliang Liu e Robert Smith. "Deep learning network optimization and hyperparameter tuning for seismic lithofacies classification". Leading Edge 40, n.º 7 (julho de 2021): 514–23. http://dx.doi.org/10.1190/tle40070514.1.
Resumo:
Deep learning is increasingly being applied in many aspects of seismic processing and interpretation. Here, we look at a deep convolutional neural network approach to multiclass seismic lithofacies characterization using well logs and seismic data. In particular, we focus on network performance and hyperparameter tuning. Several hyperparameter tuning approaches are compared, including true and directed random search methods such as very fast simulated annealing and Bayesian hyperparameter optimization. The results show that improvements in predictive capability are possible by using automatic optimization compared with manual parameter selection. In addition to evaluating the prediction accuracy's sensitivity to hyperparameters, we test various types of data representations. The choice of input seismic data can significantly impact the overall accuracy and computation speed of the optimized networks for the classification challenge under consideration. This is validated on a 3D synthetic seismic lithofacies example with acoustic and lithologic properties based on real well data and structure from an onshore oil field.
11
Grakova, Ekaterina, Martin Golasowski, Roberto Montemanni, Kateřina Slaninová, Jan Martinovič, Jafar Jamal, Kateřina Janurová e Matteo Salani. "Hyperparameter search in periodic vehicle routing problem". MATEC Web of Conferences 259 (2019): 01003. http://dx.doi.org/10.1051/matecconf/201925901003.
Resumo:
The large number of real-world applications have shown that the use of computational method for distribution process planning produces substantial savings. Many of these applications lead to problem generally known as Vehicle Routing Problem. The real-world applications are highly computationally demanding for larger instances. This article aims to show the possibilities and benefits of using hyperparameter search for solving the Periodic Vehicle Routing Problem for exhausted oil collection by execution on the supercomputing infrastructure using HyperLoom platform. HyperLoom is an open source platform for defining and executing scientific pipelines in a distributed environment. This experiment was run on the supercomputer Salomon operated by IT4Innovations.
12
Ma, Jun Yan, Xiao Ping Liao, Wei Xia e Xue Lian Yan. "Hyperparameter Estimation Based on Gaussian Process and its Application in Injection Molding". Advanced Materials Research 328-330 (setembro de 2011): 524–29. http://dx.doi.org/10.4028/www.scientific.net/amr.328-330.524.
Resumo:
As a powerful modeling tool, Gaussian process (GP) employs a Bayesian statistics approach and adopts a highly nonlinear regression technique for general scientific and engineering tasks. In the first step of constructing Gaussian process model is to estimate the best value of the hyperparameter which turned to be used in the second step where a satisfactory nonlinear model was fitted. In this paper, a modified Wolfe line search approach for hyperparameters estimation by maximizing the marginal likelihood based on conjugate gradient method is proposed. And then we analyze parameter correlation according to the value of hyperparameters to control the warpage which is a main defect for a thin shell structure part in injection molding.
13
Goh, Rui Ying, Lai Soon Lee, Hsin-Vonn Seow e Kathiresan Gopal. "Hybrid Harmony Search–Artificial Intelligence Models in Credit Scoring". Entropy 22, n.º 9 (4 de setembro de 2020): 989. http://dx.doi.org/10.3390/e22090989.
Resumo:
Credit scoring is an important tool used by financial institutions to correctly identify defaulters and non-defaulters. Support Vector Machines (SVM) and Random Forest (RF) are the Artificial Intelligence techniques that have been attracting interest due to their flexibility to account for various data patterns. Both are black-box models which are sensitive to hyperparameter settings. Feature selection can be performed on SVM to enable explanation with the reduced features, whereas feature importance computed by RF can be used for model explanation. The benefits of accuracy and interpretation allow for significant improvement in the area of credit risk and credit scoring. This paper proposes the use of Harmony Search (HS), to form a hybrid HS-SVM to perform feature selection and hyperparameter tuning simultaneously, and a hybrid HS-RF to tune the hyperparameters. A Modified HS (MHS) is also proposed with the main objective to achieve comparable results as the standard HS with a shorter computational time. MHS consists of four main modifications in the standard HS: (i) Elitism selection during memory consideration instead of random selection, (ii) dynamic exploration and exploitation operators in place of the original static operators, (iii) a self-adjusted bandwidth operator, and (iv) inclusion of additional termination criteria to reach faster convergence. Along with parallel computing, MHS effectively reduces the computational time of the proposed hybrid models. The proposed hybrid models are compared with standard statistical models across three different datasets commonly used in credit scoring studies. The computational results show that MHS-RF is most robust in terms of model performance, model explainability and computational time.
14
Qin, Chao, Yunfeng Zhang, Fangxun Bao, Caiming Zhang, Peide Liu e Peipei Liu. "XGBoost Optimized by Adaptive Particle Swarm Optimization for Credit Scoring". Mathematical Problems in Engineering 2021 (23 de março de 2021): 1–18. http://dx.doi.org/10.1155/2021/6655510.
Resumo:
Personal credit scoring is a challenging issue. In recent years, research has shown that machine learning has satisfactory performance in credit scoring. Because of the advantages of feature combination and feature selection, decision trees can match credit data which have high dimension and a complex correlation. Decision trees tend to overfitting yet. eXtreme Gradient Boosting is an advanced gradient enhanced tree that overcomes its shortcomings by integrating tree models. The structure of the model is determined by hyperparameters, which is aimed at the time-consuming and laborious problem of manual tuning, and the optimization method is employed for tuning. As particle swarm optimization describes the particle state and its motion law as continuous real numbers, the hyperparameter applicable to eXtreme Gradient Boosting can find its optimal value in the continuous search space. However, classical particle swarm optimization tends to fall into local optima. To solve this problem, this paper proposes an eXtreme Gradient Boosting credit scoring model that is based on adaptive particle swarm optimization. The swarm split, which is based on the clustering idea and two kinds of learning strategies, is employed to guide the particles to improve the diversity of the subswarms, in order to prevent the algorithm from falling into a local optimum. In the experiment, several traditional machine learning algorithms and popular ensemble learning classifiers, as well as four hyperparameter optimization methods (grid search, random search, tree-structured Parzen estimator, and particle swarm optimization), are considered for comparison. Experiments were performed with four credit datasets and seven KEEL benchmark datasets over five popular evaluation measures: accuracy, error rate (type I error and type II error), Brier score, and F 1 score. Results demonstrate that the proposed model outperforms other models on average. Moreover, adaptive particle swarm optimization performs better than the other hyperparameter optimization strategies.
15
Badriyah, Tessy, Dimas Bagus Santoso, Iwan Syarif e Daisy Rahmania Syarif. "Improving stroke diagnosis accuracy using hyperparameter optimized deep learning". International Journal of Advances in Intelligent Informatics 5, n.º 3 (17 de novembro de 2019): 256. http://dx.doi.org/10.26555/ijain.v5i3.427.
Resumo:
Stroke may cause death for anyone, including youngsters. One of the early stroke detection techniques is a Computerized Tomography (CT) scan. This research aimed to optimize hyperparameter in Deep Learning, Random Search and Bayesian Optimization for determining the right hyperparameter. The CT scan images were processed by scaling, grayscale, smoothing, thresholding, and morphological operation. Then, the images feature was extracted by the Gray Level Co-occurrence Matrix (GLCM). This research was performed a feature selection to select relevant features for reducing computing expenses, while deep learning based on hyperparameter setting was used to the data classification process. The experiment results showed that the Random Search had the best accuracy, while Bayesian Optimization excelled in optimization time.
16
Hashi, Emrana Kabir, e Md. Shahid Uz Zaman. "Developing a Hyperparameter Tuning Based Machine Learning Approach of Heart Disease Prediction". Journal of Applied Science & Process Engineering 7, n.º 2 (30 de outubro de 2020): 631–47. http://dx.doi.org/10.33736/jaspe.2639.2020.
Resumo:
Machine learning techniques are widely used in healthcare sectors to predict fatal diseases. The objective of this research was to develop and compare the performance of the traditional system with the proposed system that predicts the heart disease implementing the Logistic regression, K-nearest neighbor, Support vector machine, Decision tree, and Random Forest classification models. The proposed system helped to tune the hyperparameters using the grid search approach to the five mentioned classification algorithms. The performance of the heart disease prediction system is the major research issue. With the hyperparameter tuning model, it can be used to enhance the performance of the prediction models. The achievement of the traditional and proposed system was evaluated and compared in terms of accuracy, precision, recall, and F1 score. As the traditional system achieved accuracies between 81.97% and 90.16%., the proposed hyperparameter tuning model achieved accuracies in the range increased between 85.25% and 91.80%. These evaluations demonstrated that the proposed prediction approach is capable of achieving more accurate results compared with the traditional approach in predicting heart disease with the acquisition of feasible performance.
17
Chatzimparmpas, A., R. M. Martins, K. Kucher e A. Kerren. "VisEvol: Visual Analytics to Support Hyperparameter Search through Evolutionary Optimization". Computer Graphics Forum 40, n.º 3 (junho de 2021): 201–14. http://dx.doi.org/10.1111/cgf.14300.
18
Beck, Daniel, Trevor Cohn, Christian Hardmeier e Lucia Specia. "Learning Structural Kernels for Natural Language Processing". Transactions of the Association for Computational Linguistics 3 (dezembro de 2015): 461–73. http://dx.doi.org/10.1162/tacl_a_00151.
Resumo:
Structural kernels are a flexible learning paradigm that has been widely used in Natural Language Processing. However, the problem of model selection in kernel-based methods is usually overlooked. Previous approaches mostly rely on setting default values for kernel hyperparameters or using grid search, which is slow and coarse-grained. In contrast, Bayesian methods allow efficient model selection by maximizing the evidence on the training data through gradient-based methods. In this paper we show how to perform this in the context of structural kernels by using Gaussian Processes. Experimental results on tree kernels show that this procedure results in better prediction performance compared to hyperparameter optimization via grid search. The framework proposed in this paper can be adapted to other structures besides trees, e.g., strings and graphs, thereby extending the utility of kernel-based methods.
19
Suto, Jozsef. "The effect of hyperparameter search on artificial neural network in human activity recognition". Open Computer Science 11, n.º 1 (1 de janeiro de 2021): 411–22. http://dx.doi.org/10.1515/comp-2020-0227.
Resumo:
Abstract In the last decade, many researchers applied shallow and deep networks for human activity recognition (HAR). Currently, the trending research line in HAR is applying deep learning to extract features and classify activities from raw data. However, we observed that, authors of previous studies have not performed an efficient hyperparameter search on their artificial neural network (shallow or deep)-based classifier. Therefore, in this article, we demonstrate the effect of the random and Bayesian parameter search on a shallow neural network using five HAR databases. The result of this work shows that a shallow neural network with correct parameter optimization can achieve similar or even better recognition accuracy than the previous best deep classifier(s) on all databases. In addition, we draw conclusions about the advantages and disadvantages of the two hyperparameter search techniques according to the results.
20
Panichella, Annibale. "A Systematic Comparison of search-Based approaches for LDA hyperparameter tuning". Information and Software Technology 130 (fevereiro de 2021): 106411. http://dx.doi.org/10.1016/j.infsof.2020.106411.
21
Marques, Pedro, Matilda Rhode e Ilir Gashi. "Waste not: Using diverse neural networks from hyperparameter search for improved malware detection". Computers & Security 108 (setembro de 2021): 102339. http://dx.doi.org/10.1016/j.cose.2021.102339.
22
Kapočiūtė-Dzikienė, Jurgita, Kaspars Balodis e Raivis Skadiņš. "Intent Detection Problem Solving via Automatic DNN Hyperparameter Optimization". Applied Sciences 10, n.º 21 (22 de outubro de 2020): 7426. http://dx.doi.org/10.3390/app10217426.
Resumo:
Accurate intent detection-based chatbots are usually trained on larger datasets that are not available for some languages. Seeking the most accurate models, three English benchmark datasets that were human-translated into four morphologically complex languages (i.e., Estonian, Latvian, Lithuanian, Russian) were used. Two types of word embeddings (fastText and BERT), three types of deep neural network (DNN) classifiers (convolutional neural network (CNN); long short-term memory method (LSTM), and bidirectional LSTM (BiLSTM)), different DNN architectures (shallower and deeper), and various DNN hyperparameter values were investigated. DNN architecture and hyperparameter values were optimized automatically using the Bayesian method and random search. On three datasets of 2/5/8 intents for English, Estonian, Latvian, Lithuanian, and Russian languages, accuracies of 0.991/0.890/0.712, 0.972/0.890/0.644, 1.000/0.890/0.644, 0.981/0.872/0.712, and 0.972/0.881/0.661 were achieved, respectively. The BERT multilingual vectorization with the CNN classifier was proven to be a good choice for all datasets for all languages. Moreover, in the majority of models, the same set of optimal hyperparameter values was determined. The results obtained in this research were also compared with the previously reported values (where hyperparameter values of DNN models were selected by an expert). This comparison revealed that automatically optimized models are competitive or even more accurate when created with larger training datasets.
23
Soper, Daniel S. "Greed Is Good: Rapid Hyperparameter Optimization and Model Selection Using Greedy k-Fold Cross Validation". Electronics 10, n.º 16 (16 de agosto de 2021): 1973. http://dx.doi.org/10.3390/electronics10161973.
Resumo:
Selecting a final machine learning (ML) model typically occurs after a process of hyperparameter optimization in which many candidate models with varying structural properties and algorithmic settings are evaluated and compared. Evaluating each candidate model commonly relies on k-fold cross validation, wherein the data are randomly subdivided into k folds, with each fold being iteratively used as a validation set for a model that has been trained using the remaining folds. While many research studies have sought to accelerate ML model selection by applying metaheuristic and other search methods to the hyperparameter space, no consideration has been given to the k-fold cross validation process itself as a means of rapidly identifying the best-performing model. The current study rectifies this oversight by introducing a greedy k-fold cross validation method and demonstrating that greedy k-fold cross validation can vastly reduce the average time required to identify the best-performing model when given a fixed computational budget and a set of candidate models. This improved search time is shown to hold across a variety of ML algorithms and real-world datasets. For scenarios without a computational budget, this paper also introduces an early stopping algorithm based on the greedy cross validation method. The greedy early stopping method is shown to outperform a competing, state-of-the-art early stopping method both in terms of search time and the quality of the ML models selected by the algorithm. Since hyperparameter optimization is among the most time-consuming, computationally intensive, and monetarily expensive tasks in the broader process of developing ML-based solutions, the ability to rapidly identify optimal machine learning models using greedy cross validation has obvious and substantial benefits to organizations and researchers alike.
24
Kang, Seokho. "k-Nearest Neighbor Learning with Graph Neural Networks". Mathematics 9, n.º 8 (10 de abril de 2021): 830. http://dx.doi.org/10.3390/math9080830.
Resumo:
k-nearest neighbor (kNN) is a widely used learning algorithm for supervised learning tasks. In practice, the main challenge when using kNN is its high sensitivity to its hyperparameter setting, including the number of nearest neighbors k, the distance function, and the weighting function. To improve the robustness to hyperparameters, this study presents a novel kNN learning method based on a graph neural network, named kNNGNN. Given training data, the method learns a task-specific kNN rule in an end-to-end fashion by means of a graph neural network that takes the kNN graph of an instance to predict the label of the instance. The distance and weighting functions are implicitly embedded within the graph neural network. For a query instance, the prediction is obtained by performing a kNN search from the training data to create a kNN graph and passing it through the graph neural network. The effectiveness of the proposed method is demonstrated using various benchmark datasets for classification and regression tasks.
25
Liu, Ning, Xiaolong Ma, Zhiyuan Xu, Yanzhi Wang, Jian Tang e Jieping Ye. "AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates". Proceedings of the AAAI Conference on Artificial Intelligence 34, n.º 04 (3 de abril de 2020): 4876–83. http://dx.doi.org/10.1609/aaai.v34i04.5924.
Resumo:
Structured weight pruning is a representative model compression technique of DNNs to reduce the storage and computation requirements and accelerate inference. An automatic hyperparameter determination process is necessary due to the large number of flexible hyperparameters. This work proposes AutoCompress, an automatic structured pruning framework with the following key performance improvements: (i) effectively incorporate the combination of structured pruning schemes in the automatic process; (ii) adopt the state-of-art ADMM-based structured weight pruning as the core algorithm, and propose an innovative additional purification step for further weight reduction without accuracy loss; and (iii) develop effective heuristic search method enhanced by experience-based guided search, replacing the prior deep reinforcement learning technique which has underlying incompatibility with the target pruning problem. Extensive experiments on CIFAR-10 and ImageNet datasets demonstrate that AutoCompress is the key to achieve ultra-high pruning rates on the number of weights and FLOPs that cannot be achieved before. As an example, AutoCompress outperforms the prior work on automatic model compression by up to 33× in pruning rate (120× reduction in the actual parameter count) under the same accuracy. Significant inference speedup has been observed from the AutoCompress framework on actual measurements on smartphone. We release models of this work at anonymous link: http://bit.ly/2VZ63dS.
26
Schaer, Roger, Henning Müller e Adrien Depeursinge. "Optimized Distributed Hyperparameter Search and Simulation for Lung Texture Classification in CT Using Hadoop". Journal of Imaging 2, n.º 2 (7 de junho de 2016): 19. http://dx.doi.org/10.3390/jimaging2020019.
27
Bouktif, Salah, Ali Fiaz, Ali Ouni e Mohamed Adel Serhani. "Multi-Sequence LSTM-RNN Deep Learning and Metaheuristics for Electric Load Forecasting". Energies 13, n.º 2 (13 de janeiro de 2020): 391. http://dx.doi.org/10.3390/en13020391.
Resumo:
Short term electric load forecasting plays a crucial role for utility companies, as it allows for the efficient operation and management of power grid networks, optimal balancing between production and demand, as well as reduced production costs. As the volume and variety of energy data provided by building automation systems, smart meters, and other sources are continuously increasing, long short-term memory (LSTM) deep learning models have become an attractive approach for energy load forecasting. These models are characterized by their capabilities of learning long-term dependencies in collected electric data, which lead to accurate prediction results that outperform several alternative statistical and machine learning approaches. Unfortunately, applying LSTM models may not produce acceptable forecasting results, not only because of the noisy electric data but also due to the naive selection of its hyperparameter values. Therefore, an optimal configuration of an LSTM model is necessary to describe the electric consumption patterns and discover the time-series dynamics in the energy domain. Finding such an optimal configuration is, on the one hand, a combinatorial problem where selection is done from a very large space of choices; on the other hand, it is a learning problem where the hyperparameters should reflect the energy consumption domain knowledge, such as the influential time lags, seasonality, periodicity, and other temporal attributes. To handle this problem, we use in this paper metaheuristic-search-based algorithms, known by their ability to alleviate search complexity as well as their capacity to learn from the domain where they are applied, to find optimal or near-optimal values for the set of tunable LSTM hyperparameters in the electrical energy consumption domain. We tailor both a genetic algorithm (GA) and particle swarm optimization (PSO) to learn hyperparameters for load forecasting in the context of energy consumption of big data. The statistical analysis of the obtained result shows that the multi-sequence deep learning model tuned by the metaheuristic search algorithms provides more accurate results than the benchmark machine learning models and the LSTM model whose inputs and hyperparameters were established through limited experience and a discounted number of experimentations.
28
Le Dinh Phu, Cuong, e Dong Wang. "A Comparison of Machine Learning Methods to Predict Hospital Readmission of Diabetic Patient". Asia Proceedings of Social Sciences 7, n.º 2 (28 de março de 2021): 164–68. http://dx.doi.org/10.31580/apss.v7i2.1807.
Resumo:
Diabetes is a chronic disease whereby blood glucose is not metabolized in the body. Electronic health records (EHRs) (Yadav, P. et al., 2018). for each individual or a population have become important to standing developing trends of diseases. Machine learning helps provide accurate predictions higher than actual assessments. The main problem that we are trying to apply machine learning model and using EHRs that combines the strength of a machine learning model with various features and hyperparameter optimization or tuning. The hyperparameter optimization (Feurer, M., 2019) uses the random search optimization which minimizes a predefined loss function on given independent data. The evaluation on the method comparisons indicated that machine learning models has increased the ratio of metrics compared to previous models (Accuracy, Recall, F1 and AUC score) on the same public dataset that is reprocessed.
29
RADIUK, P. "AN APPROACH TO ACCELERATE THE TRAINING OF CONVOLUTIONAL NEURAL NETWORKS BY TUNING THE HYPERPARAMETERS OF LEARNING". Computer Systems and Information Technologies 2, n.º 2 (3 de novembro de 2020): 32–37. http://dx.doi.org/10.31891/csit-2020-2-5.
Resumo:
Over the last decade, a set of machine learning algorithms called deep learning has led to significant improvements in computer vision, natural language recognition and processing. This has led to the widespread use of a variety of commercial, learning-based products in various fields of human activity. Despite this success, the use of deep neural networks remains a black box. Today, the process of setting hyperparameters and designing a network architecture requires experience and a lot of trial and error and is based more on chance than on a scientific approach. At the same time, the task of simplifying deep learning is extremely urgent. To date, no simple ways have been invented to establish the optimal values of learning hyperparameters, namely learning speed, sample size, data set, learning pulse, and weight loss. Grid search and random search of hyperparameter space are extremely resource intensive. The choice of hyperparameters is critical for the training time and the final result. In addition, experts often choose one of the standard architectures (for example, ResNets and ready-made sets of hyperparameters. However, such kits are usually suboptimal for specific practical tasks. The presented work offers an approach to finding the optimal set of hyperparameters of learning ZNM. An integrated approach to all hyperparameters is valuable because there is an interdependence between them. The aim of the work is to develop an approach for setting a set of hyperparameters, which will reduce the time spent during the design of ZNM and ensure the efficiency of its work. In recent decades, the introduction of deep learning methods, in particular convolutional neural networks (CNNs), has led to impressive success in image and video processing. However, the training of CNN has been commonly mostly based on the employment of quasi-optimal hyperparameters. Such an approach usually requires huge computational and time costs to train the network and does not guarantee a satisfactory result. However, hyperparameters play a crucial role in the effectiveness of CNN, as diverse hyperparameters lead to models with significantly different characteristics. Poorly selected hyperparameters generally lead to low model performance. The issue of choosing optimal hyperparameters for CNN has not been resolved yet. The presented work proposes several practical approaches to setting hyperparameters, which allows reducing training time and increasing the accuracy of the model. The article considers the function of training validation loss during underfitting and overfitting. There are guidelines in the end to reach the optimization point. The paper also considers the regulation of learning rate and momentum to accelerate network training. All experiments are based on the widespread CIFAR-10 and CIFAR-100 datasets.
30
Choi, Jung Chan, Zhongqiang Liu, Suzanne Lacasse e Elin Skurtveit. "Leak-Off Pressure Using Weakly Correlated Geospatial Information and Machine Learning Algorithms". Geosciences 11, n.º 4 (19 de abril de 2021): 181. http://dx.doi.org/10.3390/geosciences11040181.
Resumo:
Leak-off pressure (LOP) is a key parameter to determine the allowable weight of drilling mud in a well and the in situ horizontal stress. The LOP test is run in situ and is frequently used by the petroleum industry. If the well pressure exceeds the LOP, wellbore instability may occur, with hydraulic fracturing and large mud losses in the formation. A reliable prediction of LOP is required to ensure safe and economical drilling operations. The prediction of LOP is challenging because it is affected by the usually complex earlier geological loading history, and the values of LOP and their measurements can vary significantly geospatially. This paper investigates the ability of machine learning algorithms to predict leak-off pressure on the basis of geospatial information of LOP measurements. About 3000 LOP test data were collected from 1800 exploration wells offshore Norway. Three machine learning algorithms (the deep neural network (DNN), random forest (RF), and support vector machine (SVM) algorithms) optimized by three hyperparameter search methods (the grid search, randomized search and Bayesian search) were compared with multivariate regression analysis. The Bayesian search algorithm needed fewer iterations than the grid search algorithms to find an optimal combination of hyperparameters. The three machine learning algorithms showed better performance than the multivariate linear regression when the features of the geospatial inputs were properly scaled. The RF algorithm gave the most promising results regardless of data scaling. If the data were not scaled, the DNN and SVM algorithms, even with optimized parameters, did not provide significantly improved test scores compared to the multivariate regression analysis. The analyses also showed that when the number of data points in a geographical setting is much smaller than that of other geographical areas, the prediction accuracy reduces significantly.
31
Wang, Yufei, Haiyang Zhang, Yongli An, Zhanlin Ji e Ivan Ganchev. "RG Hyperparameter Optimization Approach for Improved Indirect Prediction of Blood Glucose Levels by Boosting Ensemble Learning". Electronics 10, n.º 15 (27 de julho de 2021): 1797. http://dx.doi.org/10.3390/electronics10151797.
Resumo:
This paper proposes an RG hyperparameter optimization approach, based on a sequential use of random search (R) and grid search (G), for improving the blood glucose level prediction of boosting ensemble learning models. An indirect prediction of blood glucose levels in patients is performed, based on historical medical data collected by means of physical examination methods, using 40 human body’s health indicators. The conducted experiments with real clinical data proved that the proposed RG double optimization approach helps improve the prediction performance of four state-of-the-art boosting ensemble learning models enriched by it, achieving 1.47% to 24.40% MSE improvement and 0.75% to 11.54% RMSE improvement.
32
Haqmi Abas, Mohamad Aqib. "Agarwood Oil Quality Classification using Support Vector Classifier and Grid Search Cross Validation Hyperparameter Tuning". International Journal of Emerging Trends in Engineering Research 8, n.º 6 (25 de junho de 2020): 2551–56. http://dx.doi.org/10.30534/ijeter/2020/55862020.
33
Lin, Nan, Yongliang Chen, Haiqi Liu e Hanlin Liu. "A Comparative Study of Machine Learning Models with Hyperparameter Optimization Algorithm for Mapping Mineral Prospectivity". Minerals 11, n.º 2 (3 de fevereiro de 2021): 159. http://dx.doi.org/10.3390/min11020159.
Resumo:
Selecting internal hyperparameters, which can be set by the automatic search algorithm, is important to improve the generalization performance of machine learning models. In this study, the geological, remote sensing and geochemical data of the Lalingzaohuo area in Qinghai province were researched. A multi-source metallogenic information spatial data set was constructed by calculating the Youden index for selecting potential evidence layers. The model for mapping mineral prospectivity of the study area was established by combining two swarm intelligence optimization algorithms, namely the bat algorithm (BA) and the firefly algorithm (FA), with different machine learning models. The receiver operating characteristic (ROC) and prediction-area (P-A) curves were used for performance evaluation and showed that the two algorithms had an obvious optimization effect. The BA and FA differentiated in improving multilayer perceptron (MLP), AdaBoost and one-class support vector machine (OCSVM) models; thus, there was no optimization algorithm that was consistently superior to the other. However, the accuracy of the machine learning models was significantly enhanced after optimizing the hyperparameters. The area under curve (AUC) values of the ROC curve of the optimized machine learning models were all higher than 0.8, indicating that the hyperparameter optimization calculation was effective. In terms of individual model improvement, the accuracy of the FA-AdaBoost model was improved the most significantly, with the AUC value increasing from 0.8173 to 0.9597 and the prediction/area (P/A) value increasing from 3.156 to 10.765, where the mineral targets predicted by the model occupied 8.63% of the study area and contained 92.86% of the known mineral deposits. The targets predicted by the improved machine learning models are consistent with the metallogenic geological characteristics, indicating that the swarm intelligence optimization algorithm combined with the machine learning model is an efficient method for mineral prospectivity mapping.
34
Lee, Sanghyeop, Junyeob Kim, Hyeon Kang, Do-Young Kang e Jangsik Park. "Genetic Algorithm Based Deep Learning Neural Network Structure and Hyperparameter Optimization". Applied Sciences 11, n.º 2 (14 de janeiro de 2021): 744. http://dx.doi.org/10.3390/app11020744.
Resumo:
Alzheimer’s disease is one of the major challenges of population ageing, and diagnosis and prediction of the disease through various biomarkers is the key. While the application of deep learning as imaging technologies has recently expanded across the medical industry, empirical design of these technologies is very difficult. The main reason for this problem is that the performance of the Convolutional Neural Networks (CNN) differ greatly depending on the statistical distribution of the input dataset. Different hyperparameters also greatly affect the convergence of the CNN models. With this amount of information, selecting appropriate parameters for the network structure has became a large research area. Genetic Algorithm (GA), is a very popular technique to automatically select a high-performance network architecture. In this paper, we show the possibility of optimising the network architecture using GA, where its search space includes both network structure configuration and hyperparameters. To verify the performance of our Algorithm, we used an amyloid brain image dataset that is used for Alzheimer’s disease diagnosis. As a result, our algorithm outperforms Genetic CNN by 11.73% on a given classification task.
35
Kim, Seong-Hoon, Zong Woo Geem e Gi-Tae Han. "Hyperparameter Optimization Method Based on Harmony Search Algorithm to Improve Performance of 1D CNN Human Respiration Pattern Recognition System". Sensors 20, n.º 13 (1 de julho de 2020): 3697. http://dx.doi.org/10.3390/s20133697.
Resumo:
In this study, we propose a method to find an optimal combination of hyperparameters to improve the accuracy of respiration pattern recognition in a 1D (Dimensional) convolutional neural network (CNN). The proposed method is designed to integrate with a 1D CNN using the harmony search algorithm. In an experiment, we used the depth of the convolutional layer of the 1D CNN, the number and size of kernels in each layer, and the number of neurons in the dense layer as hyperparameters for optimization. The experimental results demonstrate that the proposed method provided a recognition rate for five respiration patterns of approximately 96.7% on average, which is an approximately 2.8% improvement over an existing method. In addition, the number of iterations required to derive the optimal combination of hyperparameters was 2,000,000 in the previous study. In contrast, the proposed method required only 3652 iterations.
36
Mallak, Ahlam, e Madjid Fathi. "A Hybrid Approach: Dynamic Diagnostic Rules for Sensor Systems in Industry 4.0 Generated by Online Hyperparameter Tuned Random Forest". Sci 2, n.º 3 (6 de agosto de 2020): 61. http://dx.doi.org/10.3390/sci2030061.
Resumo:
In this work, a hybrid component Fault Detection and Diagnosis (FDD) approach for industrial sensor systems is established and analyzed, to provide a hybrid schema that combines the advantages and eliminates the drawbacks of both model-based and data-driven methods of diagnosis. Moreover, it shines the light on a new utilization of Random Forest (RF) together with model-based diagnosis, beyond its ordinary data-driven application. RF is trained and hyperparameter tuned using three-fold cross validation over a random grid of parameters using random search, to finally generate diagnostic graphs as the dynamic, data-driven part of this system. This is followed by translating those graphs into model-based rules in the form of if-else statements, SQL queries or semantic queries such as SPARQL, in order to feed the dynamic rules into a structured model essential for further diagnosis. The RF hyperparameters are consistently updated online using the newly generated sensor data to maintain the dynamicity and accuracy of the generated graphs and rules thereafter. The architecture of the proposed method is demonstrated in a comprehensive manner, and the dynamic rules extraction phase is applied using a case study on condition monitoring of a hydraulic test rig using time-series multivariate sensor readings.
37
Mallak, Ahlam, e Madjid Fathi. "A Hybrid Approach: Dynamic Diagnostic Rules for Sensor Systems in Industry 4.0 Generated by Online Hyperparameter Tuned Random Forest". Sci 2, n.º 4 (24 de setembro de 2020): 61. http://dx.doi.org/10.3390/sci2040061.
Resumo:
In this work, a hybrid component Fault Detection and Diagnosis (FDD) approach for industrial sensor systems is established and analyzed, to provide a hybrid schema that combines the advantages and eliminates the drawbacks of both model-based and data-driven methods of diagnosis. Moreover, it shines the light on a new utilization of Random Forest (RF) together with model-based diagnosis, beyond its ordinary data-driven application. RF is trained and hyperparameter tuned using three-fold cross validation over a random grid of parameters using random search, to finally generate diagnostic graphs as the dynamic, data-driven part of this system. This is followed by translating those graphs into model-based rules in the form of if-else statements, SQL queries or semantic queries such as SPARQL, in order to feed the dynamic rules into a structured model essential for further diagnosis. The RF hyperparameters are consistently updated online using the newly generated sensor data to maintain the dynamicity and accuracy of the generated graphs and rules thereafter. The architecture of the proposed method is demonstrated in a comprehensive manner, and the dynamic rules extraction phase is applied using a case study on condition monitoring of a hydraulic test rig using time-series multivariate sensor readings.
38
Mallak, Ahlam, e Madjid Fathi. "A Hybrid Approach: Dynamic Diagnostic Rules for Sensor Systems in Industry 4.0 Generated by Online Hyperparameter Tuned Random Forest". Sci 2, n.º 4 (9 de outubro de 2020): 75. http://dx.doi.org/10.3390/sci2040075.
Resumo:
In this work, a hybrid component Fault Detection and Diagnosis (FDD) approach for industrial sensor systems is established and analyzed, to provide a hybrid schema that combines the advantages and eliminates the drawbacks of both model-based and data-driven methods of diagnosis. Moreover, it shines light on a new utilization of Random Forest (RF) together with model-based diagnosis, beyond its ordinary data-driven application. RF is trained and hyperparameter tuned using three-fold cross validation over a random grid of parameters using random search, to finally generate diagnostic graphs as the dynamic, data-driven part of this system. This is followed by translating those graphs into model-based rules in the form of if-else statements, SQL queries or semantic queries such as SPARQL, in order to feed the dynamic rules into a structured model essential for further diagnosis. The RF hyperparameters are consistently updated online using the newly generated sensor data to maintain the dynamicity and accuracy of the generated graphs and rules thereafter. The architecture of the proposed method is demonstrated in a comprehensive manner, and the dynamic rules extraction phase is applied using a case study on condition monitoring of a hydraulic test rig using time-series multivariate sensor readings.
39
Utsugi, Akio, e Toru Kumagai. "Bayesian Analysis of Mixtures of Factor Analyzers". Neural Computation 13, n.º 5 (1 de maio de 2001): 993–1002. http://dx.doi.org/10.1162/08997660151134299.
Resumo:
For Bayesian inference on the mixture of factor analyzers, natural conjugate priors on the parameters are introduced, and then a Gibbs sampler that generates parameter samples following the posterior is constructed. In addition, a deterministic estimation algorithm is derived by taking modes instead of samples from the conditional posteriors used in the Gibbs sampler. This is regarded as a maximum a posteriori estimation algorithm with hyperparameter search. The behaviors of the Gibbs sampler and the deterministic algorithm are compared on a simulation experiment.
40
Lee, Woo-Young, Seung-Min Park e Kwee-Bo Sim. "Optimal hyperparameter tuning of convolutional neural networks based on the parameter-setting-free harmony search algorithm". Optik 172 (novembro de 2018): 359–67. http://dx.doi.org/10.1016/j.ijleo.2018.07.044.
41
Nahhas, Faten Hamed, Helmi Z. M. Shafri, Maher Ibrahim Sameen, Biswajeet Pradhan e Shattri Mansor. "Deep Learning Approach for Building Detection Using LiDAR–Orthophoto Fusion". Journal of Sensors 2018 (5 de agosto de 2018): 1–12. http://dx.doi.org/10.1155/2018/7212307.
Resumo:
This paper reports on a building detection approach based on deep learning (DL) using the fusion of Light Detection and Ranging (LiDAR) data and orthophotos. The proposed method utilized object-based analysis to create objects, a feature-level fusion, an autoencoder-based dimensionality reduction to transform low-level features into compressed features, and a convolutional neural network (CNN) to transform compressed features into high-level features, which were used to classify objects into buildings and background. The proposed architecture was optimized for the grid search method, and its sensitivity to hyperparameters was analyzed and discussed. The proposed model was evaluated on two datasets selected from an urban area with different building types. Results show that the dimensionality reduction by the autoencoder approach from 21 features to 10 features can improve detection accuracy from 86.06% to 86.19% in the working area and from 77.92% to 78.26% in the testing area. The sensitivity analysis also shows that the selection of the hyperparameter values of the model significantly affects detection accuracy. The best hyperparameters of the model are 128 filters in the CNN model, the Adamax optimizer, 10 units in the fully connected layer of the CNN model, a batch size of 8, and a dropout of 0.2. These hyperparameters are critical to improving the generalization capacity of the model. Furthermore, comparison experiments with the support vector machine (SVM) show that the proposed model with or without dimensionality reduction outperforms the SVM models in the working area. However, the SVM model achieves better accuracy in the testing area than the proposed model without dimensionality reduction. This study generally shows that the use of an autoencoder in DL models can improve the accuracy of building recognition in fused LiDAR–orthophoto data.
42
Chen, Yi-Wei, Qingquan Song e Xia Hu. "Techniques for Automated Machine Learning". ACM SIGKDD Explorations Newsletter 22, n.º 2 (17 de janeiro de 2021): 35–50. http://dx.doi.org/10.1145/3447556.3447567.
Resumo:
Automated machine learning (AutoML) aims to find optimal machine learning solutions automatically given a problem description, its task type, and datasets. It could release the burden of data scientists from the multifarious manual tuning process and enable the access of domain experts to the off-the-shelf machine learning solutions without extensive experience. In this paper, we portray AutoML as a bi-level optimization problem, where one problem is nested within another to search the optimum in the search space, and review the current developments of AutoML in terms of three categories, automated feature engineering (AutoFE), automated model and hyperparameter tuning (AutoMHT), and automated deep learning (AutoDL). Stateof- the-art techniques in the three categories are presented. The iterative solver is proposed to generalize AutoML techniques. We summarize popular AutoML frameworks and conclude with current open challenges of AutoML.
43
Almaslukh, Bandar. "A Lightweight Deep Learning-Based Pneumonia Detection Approach for Energy-Efficient Medical Systems". Wireless Communications and Mobile Computing 2021 (21 de abril de 2021): 1–14. http://dx.doi.org/10.1155/2021/5556635.
Resumo:
Early detection of pneumonia disease can increase the survival rate of lung patients. Chest X-ray (CXR) images are the primarily means of detecting and diagnosing pneumonia. Detecting pneumonia from CXR images by a trained radiologist is a challenging task. It needs an automatic computer-aided diagnostic system to improve the accuracy of diagnosis. Developing a lightweight automatic pneumonia detection approach for energy-efficient medical systems plays an important role in improving the quality of healthcare with reduced costs and speedier response. Recent works have proposed to develop automated detection models using deep learning (DL) methods. However, the efficiency and effectiveness of these models need to be improved because they depend on the values of the models’ hyperparameters. Choosing suitable hyperparameter values is a critical task for constructing a lightweight and accurate model. In this paper, a lightweight DL approach is proposed using a pretrained DenseNet-121-based feature extraction method and a deep neural network- (DNN-) based method with a random search fine-tuning technique. The DenseNet-121 model is selected due to its ability to provide the best representation of lung features. The use of random search makes the tuning process faster and improves the efficiency and accuracy of the DNN model. An extensive set of experiments are conducted on a public dataset of CXR images using a set of evaluation metrics. The experiments show that the approach achieved 98.90% accuracy with an increase of 0.47% compared to the latest approach on the same dataset. Moreover, the experimental results demonstrate the approach that the average execution time for detection is very low, confirming its suitability for energy-efficient medical systems.
44
Grakova, Ekaterina, Jan Martinovič, Kateřina Slaninová, Kateřina Janurová, Vojtěch Cima, Martin Golasowski, Roberto Montemanni e Matteo Salani. "Setting the Configuration Parameters of the Algorithm for the Periodic Vehicle Routing Problem by HPC Power". MATEC Web of Conferences 296 (2019): 01009. http://dx.doi.org/10.1051/matecconf/201929601009.
Resumo:
The quality of an opt imal solut ion of the Vehicle Rout ing Problem is st rongly depended on the sett ing of the configurat ion parameters of the algorithm. The paper is focused on the int roduct ion of hyperparameter search for solving the Vehicle Rout ing Problem using a HyperLoom plat form for defining and execut ing scient ific pipelines in a dist ributed environment . To give a concrete example, we focused on Periodic Vehicle Rout ing Problem for the waste collect ion. HyperLoom plat form was used to define and execute the hyperparameters sweep pipeline. The heurist ic algorithm was tested on a real benchmark of the waste collect ion in Ostrava, Czech Republic. The aim of our ca se was to effect ively combine the minimizat ion of the total t ravelled distance and the opt imizat ion of the fairness of the routes in terms of the standard deviat ion of a tour length. The waste collect ion problem was very extensive and computat ionally demanding, so it was necessary to use high performance comput ing architecture for test ing a large number of different sett ings of configurat ion parameters. The experiments were run on the supercomputer Salomon operated by IT4Innovations Nat ional Supercomput ing Center in the Czech Republic.
45
Song, Shuang, Shugang Li, Tianjun Zhang, Li Ma, Shaobo Pan e Lu Gao. "Research on a Multi-Parameter Fusion Prediction Model of Pressure Relief Gas Concentration Based on RNN". Energies 14, n.º 5 (3 de março de 2021): 1384. http://dx.doi.org/10.3390/en14051384.
Resumo:
The effective prediction of gas concentration and the reasonable formulation of corresponding safety measures have important significance for improving the level of coal mine safety. To improve the accuracy of gas concentration prediction and enhance the applicability of the models, this paper starts with actual coal mine production monitoring data, improves the accuracy of gas concentration prediction through multi-parameter fusion prediction, and constructs a recurrent neural network (RNN)-based multi-parameter fusion prediction of coal face gas concentration. We determined the performance evaluation index of the model’s prediction method; used the grid search method to optimize the hyperparameters of the batch size; and used the number of neurons, the learning rate, the discard ratio, the network depth, and the early stopping method to prevent overfitting. The gas concentration prediction models—based on RNN and PSO-SVR and PSO-Adam-BP neural networks—were compared and analyzed experimentally with the mean absolute percentage error (MAPE) as the performance evaluation index. The result show that using the grid search method to adjust the batch size, the number of neurons, the learning rate, the discard ratio, and the network depth can effectively find the optimal hyperparameter combination. The training error can be reduced to 0.0195. Therefore, Adam’s optimized RNN gas concentration prediction model had higher accuracy and stability than the BP neural network and SVR. During training, the mean absolute error (MAE) could be reduced to 0.0573, and the root mean squared error (RMSE) could be reduced to 0.0167; however, the MAPE could be reduced to 0.3384% during prediction. The RNN gas concentration prediction model and parameter optimization method based on Adam optimization can effectively predict gas concentration. This method shows high accuracy in the prediction of gas concentration time series and can be used as a reference model for predicting mine gas concentration.
46
Schutera, Mark, Stefan Elser, Jochen Abhau, Ralf Mikut e Markus Reischl. "Strategies for supplementing recurrent neural network training for spatio-temporal prediction". at - Automatisierungstechnik 67, n.º 7 (26 de julho de 2019): 545–56. http://dx.doi.org/10.1515/auto-2018-0124.
Resumo:
Abstract In autonomous driving, prediction tasks address complex spatio-temporal data. This article describes the examination of Recurrent Neural Networks (RNNs) for object trajectory prediction in the image space. The proposed methods enhance the performance and spatio-temporal prediction capabilities of Recurrent Neural Networks. Two different data augmentation strategies and a hyperparameter search are implemented for this purpose. A conventional data augmentation strategy and a Generative Adversarial Network (GAN) based strategy are analyzed with respect to their ability to close the generalization gap of Recurrent Neural Networks. The results are then discussed using single-object tracklets provided by the KITTI Tracking Dataset. This work demonstrates the benefits of augmenting spatio-temporal data with GANs.
47
Huang, Jianzhong, Yuwan Cen, Nenggang Xie e Xiaohua Ye. "Inverse calculation of demolition robot based on gravitational search algorithm and differential evolution neural network". International Journal of Advanced Robotic Systems 17, n.º 3 (1 de maio de 2020): 172988142092529. http://dx.doi.org/10.1177/1729881420925298.
Resumo:
For the inverse calculation of laser-guided demolition robot, its global nonlinear mapping model from laser measuring point to joint cylinder stroke has been set up with an artificial neural network. Due to the contradiction between population diversity and convergence rate in the optimization of complex neural networks by using differential evolution, a gravitational search algorithm and differential evolution is proposed to accelerate the convergence rate of differential evolution population driven by gravity. Gravitational search algorithm and differential evolution is applied to optimize the inverse calculation neural network mapping model of demolition robot, and the algorithm simulation shows that gravity can effectively regulate the convergence process of differential evolution population. Compared with the standard differential evolution, the convergence speed and accuracy of gravitational search algorithm and differential evolution are significantly improved, which has better optimization stability. The calculation results show that the output accuracy of this gravitational and differential evolution neural network can meet the calculation requirements of the positioning control of demolition robot’s manipulator. The optimization using gravitational search algorithm and differential evolution is done with the connection weights of a neural network in this article, and as similar techniques can be applied to the other hyperparameter optimization problem. Moreover, such an inverse calculation method can provide a reference for the autonomous positioning of large hydraulic series manipulator, so as to improve the robotization level of construction machinery.
48
Demidova, Liliya A., e Artyom V. Gorchakov. "Research and Study of the Hybrid Algorithms Based on the Collective Behavior of Fish Schools and Classical Optimization Methods". Algorithms 13, n.º 4 (3 de abril de 2020): 85. http://dx.doi.org/10.3390/a13040085.
Resumo:
Inspired by biological systems, swarm intelligence algorithms are widely used to solve multimodal optimization problems. In this study, we consider the hybridization problem of an algorithm based on the collective behavior of fish schools. The algorithm is computationally inexpensive compared to other population-based algorithms. Accuracy of fish school search increases with the increase of predefined iteration count, but this also affects computation time required to find a suboptimal solution. We propose two hybrid approaches, intending to improve the evolutionary-inspired algorithm accuracy by using classical optimization methods, such as gradient descent and Newton’s optimization method. The study shows the effectiveness of the proposed hybrid algorithms, and the strong advantage of the hybrid algorithm based on fish school search and gradient descent. We provide a solution for the linearly inseparable exclusive disjunction problem using the developed algorithm and a perceptron with one hidden layer. To demonstrate the effectiveness of the algorithms, we visualize high dimensional loss surfaces near global extreme points. In addition, we apply the distributed version of the most effective hybrid algorithm to the hyperparameter optimization problem of a neural network.
49
Tuggener, Lukas, Mohammadreza Amirian, Fernando Benites, Pius von Däniken, Prakhar Gupta, Frank-Peter Schilling e Thilo Stadelmann. "Design Patterns for Resource-Constrained Automated Deep-Learning Methods". AI 1, n.º 4 (6 de novembro de 2020): 510–38. http://dx.doi.org/10.3390/ai1040031.
Resumo:
We present an extensive evaluation of a wide variety of promising design patterns for automated deep-learning (AutoDL) methods, organized according to the problem categories of the 2019 AutoDL challenges, which set the task of optimizing both model accuracy and search efficiency under tight time and computing constraints. We propose structured empirical evaluations as the most promising avenue to obtain design principles for deep-learning systems due to the absence of strong theoretical support. From these evaluations, we distill relevant patterns which give rise to neural network design recommendations. In particular, we establish (a) that very wide fully connected layers learn meaningful features faster; we illustrate (b) how the lack of pretraining in audio processing can be compensated by architecture search; we show (c) that in text processing deep-learning-based methods only pull ahead of traditional methods for short text lengths with less than a thousand characters under tight resource limitations; and lastly we present (d) evidence that in very data- and computing-constrained settings, hyperparameter tuning of more traditional machine-learning methods outperforms deep-learning systems.
50
Tam, Lydia, Wasif Bala, Jonathan Lavezo, Seth Lummus, Hannes Vogel e Kristen Yeom. "PATH-06. IMAGE-BASED MACHINE LEARNING CLASSIFIER FOR PEDIATRIC POSTERIOR FOSSA TUMOR HISTOPATHOLOGY". Neuro-Oncology 22, Supplement_3 (1 de dezembro de 2020): iii425. http://dx.doi.org/10.1093/neuonc/noaa222.642.
Resumo:
Abstract BACKGROUND Pediatric posterior fossa (PF) tumors can include astrocytomas, ependymomas, and medulloblastomas, all of which demonstrate unique histopathology. Whole slide image analyses can be time consuming and difficult. Therefore, we used machine learning to create a screenshot-based histopathology image classifier that can distinguish between types of pediatric PF tumors. METHODS We took 179 histopathology slides from Stanford University, dated from 2008–2019: 87 astrocytomas, 42 ependymomas, and 50 medulloblastomas, per pathology report. Each slide was viewed under a microscope at 20x. Then, a screenshot was taken of the region of interest representative of principal slide pathology, confirmed by a trained neuropathologist. These screenshots were used to train Resnet-18 models pre-trained on the ImageNet dataset and modified to predict three classes. Various models with different hyperparameters were trained using a random hyperparameter search method. Trained models were evaluated using 5-fold cross-validation, assigning 20% of the dataset for validation with each evaluation. Qualitative analysis of model performance was assessed by creating Class Activation Map (CAM) representations of image predictions. RESULTS The top performing Resnet-18 model achieved a cross-validation F1 of 0.967 on categorizing screenshots of tumor pathology into three types. Qualitative analysis using CAMs indicated the model was able to identify salient distinguishing features of each tumor type. CONCLUSIONS We present a PF lesion classifier capable of distinguishing between astrocytomas, ependymomas, and medulloblastomas based on a histopathology screenshot. Given its ease of use, this tool has potential as an educational tool in an academic setting.