Journal articles: 'Large dimensional learning'

1

Khan, Usman A., Soummya Kar, and José M. F. Moura. "Higher Dimensional Consensus: Learning in Large-Scale Networks." IEEE Transactions on Signal Processing 58, no. 5 (May 2010): 2836–49. http://dx.doi.org/10.1109/tsp.2010.2042482.

Full text

APA, Harvard, Vancouver, ISO, and other styles

2

Lin, Zhiping, Jiuwen Cao, Tao Chen, Yi Jin, Zhan-Li Sun, and Amaury Lendasse. "Extreme Learning Machine on High Dimensional and Large Data Applications." Mathematical Problems in Engineering 2015 (2015): 1–2. http://dx.doi.org/10.1155/2015/624903.

Full text

APA, Harvard, Vancouver, ISO, and other styles

3

Peng, Chong, Jie Cheng, and Qiang Cheng. "A Supervised Learning Model for High-Dimensional and Large-Scale Data." ACM Transactions on Intelligent Systems and Technology 8, no. 2 (January 18, 2017): 1–23. http://dx.doi.org/10.1145/2972957.

Full text

APA, Harvard, Vancouver, ISO, and other styles

4

Terol, Rafael Munoz, Alejandro Reina Reina, Saber Ziaei, and David Gil. "A Machine Learning Approach to Reduce Dimensional Space in Large Datasets." IEEE Access 8 (2020): 148181–92. http://dx.doi.org/10.1109/access.2020.3012836.

Full text

APA, Harvard, Vancouver, ISO, and other styles

5

Keriven, Nicolas, Anthony Bourrier, Rémi Gribonval, and Patrick Pérez. "Sketching for large-scale learning of mixture models." Information and Inference: A Journal of the IMA 7, no. 3 (December 22, 2017): 447–508. http://dx.doi.org/10.1093/imaiai/iax015.

Full text

Abstract:

Abstract Learning parameters from voluminous data can be prohibitive in terms of memory and computational requirements. We propose a ‘compressive learning’ framework, where we estimate model parameters from a sketch of the training data. This sketch is a collection of generalized moments of the underlying probability distribution of the data. It can be computed in a single pass on the training set and is easily computable on streams or distributed datasets. The proposed framework shares similarities with compressive sensing, which aims at drastically reducing the dimension of high-dimensional signals while preserving the ability to reconstruct them. To perform the estimation task, we derive an iterative algorithm analogous to sparse reconstruction algorithms in the context of linear inverse problems. We exemplify our framework with the compressive estimation of a Gaussian mixture model (GMM), providing heuristics on the choice of the sketching procedure and theoretical guarantees of reconstruction. We experimentally show on synthetic data that the proposed algorithm yields results comparable to the classical expectation-maximization technique while requiring significantly less memory and fewer computations when the number of database elements is large. We further demonstrate the potential of the approach on real large-scale data (over $10^{8}$ training samples) for the task of model-based speaker verification. Finally, we draw some connections between the proposed framework and approximate Hilbert space embedding of probability distributions using random features. We show that the proposed sketching operator can be seen as an innovative method to design translation-invariant kernels adapted to the analysis of GMMs. We also use this theoretical framework to derive preliminary information preservation guarantees, in the spirit of infinite-dimensional compressive sensing.

APA, Harvard, Vancouver, ISO, and other styles

6

Panos, Aristeidis, Petros Dellaportas, and Michalis K. Titsias. "Large scale multi-label learning using Gaussian processes." Machine Learning 110, no. 5 (April 14, 2021): 965–87. http://dx.doi.org/10.1007/s10994-021-05952-5.

Full text

Abstract:

AbstractWe introduce a Gaussian process latent factor model for multi-label classification that can capture correlations among class labels by using a small set of latent Gaussian process functions. To address computational challenges, when the number of training instances is very large, we introduce several techniques based on variational sparse Gaussian process approximations and stochastic optimization. Specifically, we apply doubly stochastic variational inference that sub-samples data instances and classes which allows us to cope with Big Data. Furthermore, we show it is possible and beneficial to optimize over inducing points, using gradient-based methods, even in very high dimensional input spaces involving up to hundreds of thousands of dimensions. We demonstrate the usefulness of our approach on several real-world large-scale multi-label learning problems.

APA, Harvard, Vancouver, ISO, and other styles

7

Cao, Jiuwen, and Zhiping Lin. "Extreme Learning Machines on High Dimensional and Large Data Applications: A Survey." Mathematical Problems in Engineering 2015 (2015): 1–13. http://dx.doi.org/10.1155/2015/103796.

Full text

Abstract:

Extreme learning machine (ELM) has been developed for single hidden layer feedforward neural networks (SLFNs). In ELM algorithm, the connections between the input layer and the hidden neurons are randomly assigned and remain unchanged during the learning process. The output connections are then tuned via minimizing the cost function through a linear system. The computational burden of ELM has been significantly reduced as the only cost is solving a linear system. The low computational complexity attracted a great deal of attention from the research community, especially for high dimensional and large data applications. This paper provides an up-to-date survey on the recent developments of ELM and its applications in high dimensional and large data. Comprehensive reviews on image processing, video processing, medical signal processing, and other popular large data applications with ELM are presented in the paper.

APA, Harvard, Vancouver, ISO, and other styles

8

Ju, Cheng, Susan Gruber, Samuel D. Lendle, Antoine Chambaz, Jessica M. Franklin, Richard Wyss, Sebastian Schneeweiss, and Mark J. van der Laan. "Scalable collaborative targeted learning for high-dimensional data." Statistical Methods in Medical Research 28, no. 2 (September 22, 2017): 532–54. http://dx.doi.org/10.1177/0962280217729845.

Full text

Abstract:

Robust inference of a low-dimensional parameter in a large semi-parametric model relies on external estimators of infinite-dimensional features of the distribution of the data. Typically, only one of the latter is optimized for the sake of constructing a well-behaved estimator of the low-dimensional parameter of interest. Optimizing more than one of them for the sake of achieving a better bias-variance trade-off in the estimation of the parameter of interest is the core idea driving the general template of the collaborative targeted minimum loss-based estimation procedure. The original instantiation of the collaborative targeted minimum loss-based estimation template can be presented as a greedy forward stepwise collaborative targeted minimum loss-based estimation algorithm. It does not scale well when the number p of covariates increases drastically. This motivates the introduction of a novel instantiation of the collaborative targeted minimum loss-based estimation template where the covariates are pre-ordered. Its time complexity is [Formula: see text] as opposed to the original [Formula: see text], a remarkable gain. We propose two pre-ordering strategies and suggest a rule of thumb to develop other meaningful strategies. Because it is usually unclear a priori which pre-ordering strategy to choose, we also introduce another instantiation called SL-C-TMLE algorithm that enables the data-driven choice of the better pre-ordering strategy given the problem at hand. Its time complexity is [Formula: see text] as well. The computational burden and relative performance of these algorithms were compared in simulation studies involving fully synthetic data or partially synthetic data based on a real world large electronic health database; and in analyses of three real, large electronic health databases. In all analyses involving electronic health databases, the greedy collaborative targeted minimum loss-based estimation algorithm is unacceptably slow. Simulation studies seem to indicate that our scalable collaborative targeted minimum loss-based estimation and SL-C-TMLE algorithms work well. All C-TMLEs are publicly available in a Julia software package.

APA, Harvard, Vancouver, ISO, and other styles

9

Loyola R, Diego G., Mattia Pedergnana, and Sebastián Gimeno García. "Smart sampling and incremental function learning for very large high dimensional data." Neural Networks 78 (June 2016): 75–87. http://dx.doi.org/10.1016/j.neunet.2015.09.001.

Full text

APA, Harvard, Vancouver, ISO, and other styles

10

Tran, Loc, Debrup Banerjee, Jihong Wang, Ashok J. Kumar, Frederic McKenzie, Yaohang Li, and Jiang Li. "High-dimensional MRI data analysis using a large-scale manifold learning approach." Machine Vision and Applications 24, no. 5 (April 19, 2013): 995–1014. http://dx.doi.org/10.1007/s00138-013-0499-8.

Full text

APA, Harvard, Vancouver, ISO, and other styles

11

Roth, Dan, Ming-Hsuan Yang, and Narendra Ahuja. "Learning to Recognize Three-Dimensional Objects." Neural Computation 14, no. 5 (May 1, 2002): 1071–103. http://dx.doi.org/10.1162/089976602753633394.

Full text

Abstract:

A learning account for the problem of object recognition is developed within the probably approximately correct (PAC) model of learnability. The key assumption underlying this work is that objects can be recognized (or discriminated) using simple representations in terms of syntactically simple relations over the raw image. Although the potential number of these simple relations could be huge, only a few of them are actually present in each observed image, and a fairly small number of those observed are relevant to discriminating an object. We show that these properties can be exploited to yield an efficient learning approach in terms of sample and computational complexity within the PAC model. No assumptions are needed on the distribution of the observed objects, and the learning performance is quantified relative to its experience. Most important, the success of learning an object representation is naturally tied to the ability to represent it as a function of some intermediate representations extracted from the image. We evaluate this approach in a large-scale experimental study in which the SNoW learning architecture is used to learn representations for the 100 objects in the Columbia Object Image Library. Experimental results exhibit good generalization and robustness properties of the SNoW-based method relative to other approaches. SNoW's recognition rate degrades more gracefully when the training data contains fewer views, and it shows similar behavior in some preliminary experiments with partially occluded objects.

APA, Harvard, Vancouver, ISO, and other styles

12

Alzu'bi, Ahmad, and Abdelrahman Abuarqoub. "Deep learning model with low-dimensional random projection for large-scale image search." Engineering Science and Technology, an International Journal 23, no. 4 (August 2020): 911–20. http://dx.doi.org/10.1016/j.jestch.2019.12.004.

Full text

APA, Harvard, Vancouver, ISO, and other styles

13

Cho, Hyeongmin, and Sangkyun Lee. "Data Quality Measures and Efficient Evaluation Algorithms for Large-Scale High-Dimensional Data." Applied Sciences 11, no. 2 (January 6, 2021): 472. http://dx.doi.org/10.3390/app11020472.

Full text

Abstract:

Machine learning has been proven to be effective in various application areas, such as object and speech recognition on mobile systems. Since a critical key to machine learning success is the availability of large training data, many datasets are being disclosed and published online. From a data consumer or manager point of view, measuring data quality is an important first step in the learning process. We need to determine which datasets to use, update, and maintain. However, not many practical ways to measure data quality are available today, especially when it comes to large-scale high-dimensional data, such as images and videos. This paper proposes two data quality measures that can compute class separability and in-class variability, the two important aspects of data quality, for a given dataset. Classical data quality measures tend to focus only on class separability; however, we suggest that in-class variability is another important data quality factor. We provide efficient algorithms to compute our quality measures based on random projections and bootstrapping with statistical benefits on large-scale high-dimensional data. In experiments, we show that our measures are compatible with classical measures on small-scale data and can be computed much more efficiently on large-scale high-dimensional datasets.

APA, Harvard, Vancouver, ISO, and other styles

14

Mei, Jiangyuan, Jian Hou, Jicheng Chen, and Hamid Reza Karimi. "A Fast Logdet Divergence Based Metric Learning Algorithm for Large Data Sets Classification." Abstract and Applied Analysis 2014 (2014): 1–9. http://dx.doi.org/10.1155/2014/463981.

Full text

Abstract:

Large data sets classification is widely used in many industrial applications. It is a challenging task to classify large data sets efficiently, accurately, and robustly, as large data sets always contain numerous instances with high dimensional feature space. In order to deal with this problem, in this paper we present an online Logdet divergence based metric learning (LDML) model by making use of the powerfulness of metric learning. We firstly generate a Mahalanobis matrix via learning the training data with LDML model. Meanwhile, we propose a compressed representation for high dimensional Mahalanobis matrix to reduce the computation complexity in each iteration. The final Mahalanobis matrix obtained this way measures the distances between instances accurately and serves as the basis of classifiers, for example, thek-nearest neighbors classifier. Experiments on benchmark data sets demonstrate that the proposed algorithm compares favorably with the state-of-the-art methods.

APA, Harvard, Vancouver, ISO, and other styles

15

Wang, Jianzhong. "Semi-supervised learning using multiple one-dimensional embedding based adaptive interpolation." International Journal of Wavelets, Multiresolution and Information Processing 14, no. 02 (March 2016): 1640002. http://dx.doi.org/10.1142/s0219691316400026.

Full text

Abstract:

We propose a novel semi-supervised learning (SSL) scheme using adaptive interpolation on multiple one-dimensional (1D) embedded data. For a given high-dimensional dataset, we smoothly map it onto several different 1D sequences, so that the labeled subset is converted to a 1D subset for each of these sequences. Applying the cubic interpolation of the labeled subset, we obtain a subset of unlabeled points, which are assigned to the same label in all interpolations. Selecting a proportion of these points at random and adding them to the current labeled subset, we build a larger labeled subset for the next interpolation. Repeating the embedding and interpolation, we enlarge the labeled subset gradually, and finally reach a labeled set with a reasonable large size, based on which the final classifier is constructed. We explore the use of the proposed scheme in the classification of handwritten digits and show promising results.

APA, Harvard, Vancouver, ISO, and other styles

16

Xu, Qingzhen. "A Novel Machine Learning Strategy Based on Two-Dimensional Numerical Models in Financial Engineering." Mathematical Problems in Engineering 2013 (2013): 1–6. http://dx.doi.org/10.1155/2013/659809.

Full text

Abstract:

Machine learning is the most commonly used technique to address larger and more complex tasks by analyzing the most relevant information already present in databases. In order to better predict the future trend of the index, this paper proposes a two-dimensional numerical model for machine learning to simulate major U.S. stock market index and uses a nonlinear implicit finite-difference method to find numerical solutions of the two-dimensional simulation model. The proposed machine learning method uses partial differential equations to predict the stock market and can be extensively used to accelerate large-scale data processing on the history database. The experimental results show that the proposed algorithm reduces the prediction error and improves forecasting precision.

APA, Harvard, Vancouver, ISO, and other styles

17

Yochum, Phatpicha, Liang Chang, Tianlong Gu, and Manli Zhu. "Learning Sentiment over Network Embedding for Recommendation System." International Journal of Machine Learning and Computing 11, no. 1 (January 2021): 12–20. http://dx.doi.org/10.18178/ijmlc.2021.11.1.1008.

Full text

Abstract:

With the rapid development of Internet, various unstructured information, such as user-generated content, textual reviews, and implicit or explicit feedbacks have grown continuously. Though structured knowledge bases (KBs) which consist of a large number of triples exhibit great advantages in recommendation field recently. In this paper, we propose a novel approach to learn sentiment over network embedding for recommendation system based on the knowledge graph which we have been built, that is, we integrate the network embedding method with the sentiment of user reviews. Specifically, we use the typical network embedding method node2vec to embed the large-scale structured data into a low-dimensional vector space to capture the internal semantic information of users and attractions and apply the user weight scoring which is the combination of user review ratings and textual reviews to get similar attractions among users. Experimental results on real-world dataset verified the superior recommendation performance on precision, recall, and F-measure of our approach compared with state-of-the-art baselines.

APA, Harvard, Vancouver, ISO, and other styles

18

Matz, Rebecca L., Cori L. Fata-Hartley, Lynmarie A. Posey, James T. Laverty, Sonia M. Underwood, Justin H. Carmel, Deborah G. Herrington, et al. "Evaluating the extent of a large-scale transformation in gateway science courses." Science Advances 4, no. 10 (October 2018): eaau0554. http://dx.doi.org/10.1126/sciadv.aau0554.

Full text

Abstract:

We evaluate the impact of an institutional effort to transform undergraduate science courses using an approach based on course assessments. The approach is guided byA Framework for K-12 Science Educationand focuses on scientific and engineering practices, crosscutting concepts, and core ideas, together called three-dimensional learning. To evaluate the extent of change, we applied the Three-dimensional Learning Assessment Protocol to 4 years of chemistry, physics, and biology course exams. Changes in exams differed by discipline and even by course, apparently depending on an interplay between departmental culture, course organization, and perceived course ownership, demonstrating the complex nature of transformation in higher education. We conclude that while transformation must be supported at all organizational levels, ultimately, change is controlled by factors at the course and departmental levels.

APA, Harvard, Vancouver, ISO, and other styles

19

Stojkovic, Ivan, and Zoran Obradovic. "Sparse Learning of the Disease Severity Score for High-Dimensional Data." Complexity 2017 (2017): 1–11. http://dx.doi.org/10.1155/2017/7120691.

Full text

Abstract:

Learning disease severity scores automatically from collected measurements may aid in the quality of both healthcare and scientific understanding. Some steps in that direction have been taken and machine learning algorithms for extracting scoring functions from data have been proposed. Given the rapid increase in both quantity and diversity of data measured and stored, the large amount of information is becoming one of the challenges for learning algorithms. In this work, we investigated the direction of the problem where the dimensionality of measured variables is large. Learning the severity score in such cases brings the issue of which of measured features are relevant. We have proposed a novel approach by combining desirable properties of existing formulations, which compares favorably to alternatives in accuracy and especially in the robustness of the learned scoring function. The proposed formulation has a nonsmooth penalty that induces sparsity. This problem is solved by addressing a dual formulation which is smooth and allows an efficient optimization. The proposed approach might be used as an effective and reliable tool for both scoring function learning and biomarker discovery, as demonstrated by identifying a stable set of genes related to influenza symptoms’ severity, which are enriched in immune-related processes.

APA, Harvard, Vancouver, ISO, and other styles

20

Espadoto, Mateus, Nina Sumiko Tomita Hirata, and Alexandru C. Telea. "Deep learning multidimensional projections." Information Visualization 19, no. 3 (May 18, 2020): 247–69. http://dx.doi.org/10.1177/1473871620909485.

Full text

Abstract:

Dimensionality reduction methods, also known as projections, are often used to explore multidimensional data in machine learning, data science, and information visualization. However, several such methods, such as the well-known t-distributed stochastic neighbor embedding and its variants, are computationally expensive for large datasets, suffer from stability problems, and cannot directly handle out-of-sample data. We propose a learning approach to construct any such projections. We train a deep neural network based on sample set drawn from a given data universe, and their corresponding two-dimensional projections, compute with any user-chosen technique. Next, we use the network to infer projections of any dataset from the same universe. Our approach generates projections with similar characteristics as the learned ones, is computationally two to four orders of magnitude faster than existing projection methods, has no complex-to-set user parameters, handles out-of-sample data in a stable manner, and can be used to learn any projection technique. We demonstrate our proposal on several real-world high-dimensional datasets from machine learning.

APA, Harvard, Vancouver, ISO, and other styles

21

Xu, Suwa, Bochao Jia, and Faming Liang. "Learning Moral Graphs in Construction of High-Dimensional Bayesian Networks for Mixed Data." Neural Computation 31, no. 6 (June 2019): 1183–214. http://dx.doi.org/10.1162/neco_a_01190.

Full text

Abstract:

Bayesian networks have been widely used in many scientific fields for describing the conditional independence relationships for a large set of random variables. This letter proposes a novel algorithm, the so-called p-learning algorithm, for learning moral graphs for high-dimensional Bayesian networks. The moral graph is a Markov network representation of the Bayesian network and also the key to construction of the Bayesian network for constraint-based algorithms. The consistency of the p-learning algorithm is justified under the small- n, large- p scenario. The numerical results indicate that the p-learning algorithm significantly outperforms the existing ones, such as the PC, grow-shrink, incremental association, semi-interleaved hiton, hill-climbing, and max-min hill-climbing. Under the sparsity assumption, the p-learning algorithm has a computational complexity of O(p2) even in the worst case, while the existing algorithms have a computational complexity of O(p3) in the worst case.

APA, Harvard, Vancouver, ISO, and other styles

22

Zhang, Peng Hao. "Study Speech Recognition System Based on Manifold Learning." Applied Mechanics and Materials 380-384 (August 2013): 3762–65. http://dx.doi.org/10.4028/www.scientific.net/amm.380-384.3762.

Full text

Abstract:

This paper conducts a comprehensive research and discussion on the relevant technologies and manifold learning.Traditional MFCC phonetic feature will lead a slower learning speed on account of it has high dimension and is large in data quantities. In order to solve this problem, we introduce a manifold learning, putting forward two new extraction methods of MFCC-Manifold phonetic feature. We can reduce dimensions by making use of ISOMAP algorithm which bases on the classical MDS (Multidimensional scaling). Introducing geodesic distance to replace the original European distance data will make twenty-four dimensional data, which using the traditional MFCC feature extraction down to ten dimensional data.

APA, Harvard, Vancouver, ISO, and other styles

23

Basak, Jayanta. "Learning Hough Transform: A Neural Network Model." Neural Computation 13, no. 3 (March 1, 2001): 651–76. http://dx.doi.org/10.1162/089976601300014501.

Full text

Abstract:

A single-layered Hough transform network is proposed that accepts image coordinates of each object pixel as input and produces a set of outputs that indicate the belongingness of the pixel to a particular structure (e.g., a straight line). The network is able to learn adaptively the parametric forms of the linear segments present in the image. It is designed for learning and identification not only of linear segments in two-dimensional images but also the planes and hyperplanes in the higher-dimensional spaces. It provides an efficient representation of visual information embedded in the connection weights. The network not only reduces the large space requirement, as in the case of classical Hough transform, but also represents the parameters with high precision.

APA, Harvard, Vancouver, ISO, and other styles

24

Farrell, Annie, Guiming Wang, Scott A. Rush, James A. Martin, Jerrold L. Belant, Adam B. Butler, and Dave Godwin. "Machine learning of large‐scale spatial distributions of wild turkeys with high‐dimensional environmental data." Ecology and Evolution 9, no. 10 (April 24, 2019): 5938–49. http://dx.doi.org/10.1002/ece3.5177.

Full text

APA, Harvard, Vancouver, ISO, and other styles

25

BECKER, SEBASTIAN, PATRICK CHERIDITO, ARNULF JENTZEN, and TIMO WELTI. "Solving high-dimensional optimal stopping problems using deep learning." European Journal of Applied Mathematics 32, no. 3 (April 27, 2021): 470–514. http://dx.doi.org/10.1017/s0956792521000073.

Full text

Abstract:

Nowadays many financial derivatives, such as American or Bermudan options, are of early exercise type. Often the pricing of early exercise options gives rise to high-dimensional optimal stopping problems, since the dimension corresponds to the number of underlying assets. High-dimensional optimal stopping problems are, however, notoriously difficult to solve due to the well-known curse of dimensionality. In this work, we propose an algorithm for solving such problems, which is based on deep learning and computes, in the context of early exercise option pricing, both approximations of an optimal exercise strategy and the price of the considered option. The proposed algorithm can also be applied to optimal stopping problems that arise in other areas where the underlying stochastic process can be efficiently simulated. We present numerical results for a large number of example problems, which include the pricing of many high-dimensional American and Bermudan options, such as Bermudan max-call options in up to 5000 dimensions. Most of the obtained results are compared to reference values computed by exploiting the specific problem design or, where available, to reference values from the literature. These numerical results suggest that the proposed algorithm is highly effective in the case of many underlyings, in terms of both accuracy and speed.

APA, Harvard, Vancouver, ISO, and other styles

26

Hao, Lin, Juncheol Kim, Sookhee Kwon, and Il Do Ha. "Deep Learning-Based Survival Analysis for High-Dimensional Survival Data." Mathematics 9, no. 11 (May 28, 2021): 1244. http://dx.doi.org/10.3390/math9111244.

Full text

Abstract:

With the development of high-throughput technologies, more and more high-dimensional or ultra-high-dimensional genomic data are being generated. Therefore, effectively analyzing such data has become a significant challenge. Machine learning (ML) algorithms have been widely applied for modeling nonlinear and complicated interactions in a variety of practical fields such as high-dimensional survival data. Recently, multilayer deep neural network (DNN) models have made remarkable achievements. Thus, a Cox-based DNN prediction survival model (DNNSurv model), which was built with Keras and TensorFlow, was developed. However, its results were only evaluated on the survival datasets with high-dimensional or large sample sizes. In this paper, we evaluated the prediction performance of the DNNSurv model using ultra-high-dimensional and high-dimensional survival datasets and compared it with three popular ML survival prediction models (i.e., random survival forest and the Cox-based LASSO and Ridge models). For this purpose, we also present the optimal setting of several hyperparameters, including the selection of a tuning parameter. The proposed method demonstrated via data analysis that the DNNSurv model performed well overall as compared with the ML models, in terms of the three main evaluation measures (i.e., concordance index, time-dependent Brier score, and the time-dependent AUC) for survival prediction performance.

APA, Harvard, Vancouver, ISO, and other styles

27

Vijayakumar, Sethu, Aaron D'Souza, and Stefan Schaal. "Incremental Online Learning in High Dimensions." Neural Computation 17, no. 12 (December 1, 2005): 2602–34. http://dx.doi.org/10.1162/089976605774320557.

Full text

Abstract:

Locally weighted projection regression (LWPR) is a new algorithm for incremental nonlinear function approximation in high-dimensional spaces with redundant and irrelevant input dimensions. At its core, it employs nonparametric regression with locally linear models. In order to stay computationally efficient and numerically robust, each local model performs the regression analysis with a small number of univariate regressions in selected directions in input space in the spirit of partial least squares regression. We discuss when and how local learning techniques can successfully work in high-dimensional spaces and review the various techniques for local dimensionality reduction before finally deriving the LWPR algorithm. The properties of LWPR are that it (1) learns rapidly with second-order learning methods based on incremental training, (2) uses statistically sound stochastic leave-one-out cross validation for learning without the need to memorize training data, (3) adjusts its weighting kernels based on only local information in order to minimize the danger of negative interference of incremental learning, (4) has a computational complexity that is linear in the number of inputs, and (5) can deal with a large number of—possibly redundant—inputs, as shown in various empirical evaluations with up to 90 dimensional data sets. For a probabilistic interpretation, predictive variance and confidence intervals are derived. To our knowledge, LWPR is the first truly incremental spatially localized learning method that can successfully and efficiently operate in very high-dimensional spaces.

APA, Harvard, Vancouver, ISO, and other styles

28

Xu, Jiali, Qian Yin, Ping Guo, and Xin Zheng. "Two-dimensional multifibre spectral image correction based on machine learning techniques." Monthly Notices of the Royal Astronomical Society 499, no. 2 (September 19, 2020): 1972–84. http://dx.doi.org/10.1093/mnras/staa2883.

Full text

Abstract:

ABSTRACT Owing to the limited size and imperfections of the optical components in a spectrometer, aberrations inevitably make their way into 2D multifibre spectral images in the Large Sky Area Multi-Object Fiber Spectroscopy Telescope (LAMOST), which leads to obvious spatial variation of the point spread functions (PSFs). However, if spatially variant PSFs are estimated directly, the large storage and intensive computational requirements result in the deconvolution spectrum extraction method becoming intractable. In this paper, we propose a novel method to solve the problem of spatial variation of the PSFs through image aberration correction. When CCD image aberrations are corrected, the convolution kernel can be approximated by only one spatially invariant PSF. Specifically, a novel method based on machine learning is proposed to calibrate the distorted spectral images. The method includes many techniques, such as total least squares (TLS) algorithm, self-supervised learning and multilayer feed-forward neural networksnetworks, and it makes use of a special training set sampling scheme combining 2D distortion features in a flat-field spectrum and calibration lamp spectrum. The calibration experiments on the LAMOST CCD images show that the proposed method is feasible. Furthermore, the spectrum extraction results before and after calibration are compared, and the experimental results show that the characteristics of the extracted 1D waveform are closer to those of an ideal optics system after image correction, and that the PSF of the corrected object spectrum estimated by the blind deconvolution method is nearly centrosymmetric, which indicates that our proposed method can significantly reduce the complexity of spectrum extraction and improve extraction accuracy.

APA, Harvard, Vancouver, ISO, and other styles

29

Zhang, Tong. "Learning Bounds for Kernel Regression Using Effective Data Dimensionality." Neural Computation 17, no. 9 (September 1, 2005): 2077–98. http://dx.doi.org/10.1162/0899766054323008.

Full text

Abstract:

Kernel methods can embed finite-dimensional data into infinite-dimensional feature spaces. In spite of the large underlying feature dimensionality, kernel methods can achieve good generalization ability. This observation is often wrongly interpreted, and it has been used to argue that kernel learning can magically avoid the “curse-of-dimensionality” phenomenon encountered in statistical estimation problems. This letter shows that although using kernel representation, one can embed data into an infinite-dimensional feature space; the effective dimensionality of this embedding, which determines the learning complexity of the underlying kernel machine, is usually small. In particular, we introduce an algebraic definition of a scale-sensitive effective dimension associated with a kernel representation. Based on this quantity, we derive upper bounds on the generalization performance of some kernel regression methods. Moreover, we show that the resulting convergent rates are optimal under various circumstances.

APA, Harvard, Vancouver, ISO, and other styles

30

Yin, Yanqing, and Jiang Hu. "On the limit of the spectral distribution of large-dimensional random quaternion covariance matrices." Random Matrices: Theory and Applications 06, no. 02 (January 10, 2017): 1750004. http://dx.doi.org/10.1142/s2010326317500046.

Full text

Abstract:

The use of quaternions and quaternion matrices in practice, such as in machine learning, adaptive filtering, vector sensing and image processing, has recently been rapidly gaining in popularity. In this paper, by applying random matrix theory, we investigate the spectral distribution of large-dimensional quaternion covariance matrices when the quaternion samples are drawn from a population that satisfies a mild moment condition. We also apply the result to several common models.

APA, Harvard, Vancouver, ISO, and other styles

31

Ghattas, Omar, and Karen Willcox. "Learning physics-based models from data: perspectives from inverse problems and model reduction." Acta Numerica 30 (May 2021): 445–554. http://dx.doi.org/10.1017/s0962492921000064.

Full text

Abstract:

This article addresses the inference of physics models from data, from the perspectives of inverse problems and model reduction. These fields develop formulations that integrate data into physics-based models while exploiting the fact that many mathematical models of natural and engineered systems exhibit an intrinsically low-dimensional solution manifold. In inverse problems, we seek to infer uncertain components of the inputs from observations of the outputs, while in model reduction we seek low-dimensional models that explicitly capture the salient features of the input–output map through approximation in a low-dimensional subspace. In both cases, the result is a predictive model that reflects data-driven learning yet deeply embeds the underlying physics, and thus can be used for design, control and decision-making, often with quantified uncertainties. We highlight recent developments in scalable and efficient algorithms for inverse problems and model reduction governed by large-scale models in the form of partial differential equations. Several illustrative applications to large-scale complex problems across different domains of science and engineering are provided.

APA, Harvard, Vancouver, ISO, and other styles

32

Huang, Xiao, Qingquan Song, Fan Yang, and Xia Hu. "Large-Scale Heterogeneous Feature Embedding." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 3878–85. http://dx.doi.org/10.1609/aaai.v33i01.33013878.

Full text

Abstract:

Feature embedding aims to learn a low-dimensional vector representation for each instance to preserve the information in its features. These representations can benefit various offthe-shelf learning algorithms. While embedding models for a single type of features have been well-studied, real-world instances often contain multiple types of correlated features or even information within a different modality such as networks. Existing studies such as multiview learning show that it is promising to learn unified vector representations from all sources. However, high computational costs of incorporating heterogeneous information limit the applications of existing algorithms. The number of instances and dimensions of features in practice are often large. To bridge the gap, we propose a scalable framework FeatWalk, which can model and incorporate instance similarities in terms of different types of features into a unified embedding representation. To enable the scalability, FeatWalk does not directly calculate any similarity measure, but provides an alternative way to simulate the similarity-based random walks among instances to extract the local instance proximity and preserve it in a set of instance index sequences. These sequences are homogeneous with each other. A scalable word embedding algorithm is applied to them to learn a joint embedding representation of instances. Experiments on four real-world datasets demonstrate the efficiency and effectiveness of FeatWalk.

APA, Harvard, Vancouver, ISO, and other styles

33

Erfani, Sarah M., Sutharshan Rajasegarar, Shanika Karunasekera, and Christopher Leckie. "High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning." Pattern Recognition 58 (October 2016): 121–34. http://dx.doi.org/10.1016/j.patcog.2016.03.028.

Full text

APA, Harvard, Vancouver, ISO, and other styles

34

Jiang, Youhe, Huaxi Gu, Yunfeng Lu, and Xiaoshan Yu. "2D-HRA: Two-Dimensional Hierarchical Ring-Based All-Reduce Algorithm in Large-Scale Distributed Machine Learning." IEEE Access 8 (2020): 183488–94. http://dx.doi.org/10.1109/access.2020.3028367.

Full text

APA, Harvard, Vancouver, ISO, and other styles

35

Lyons, John Thomas, and Tuhfe Göçmen. "Applied Machine Learning Techniques for Performance Analysis in Large Wind Farms." Energies 14, no. 13 (June 23, 2021): 3756. http://dx.doi.org/10.3390/en14133756.

Full text

Abstract:

As the amount of information collected by wind turbines continues to grow, so too does the potential of its leveraging. The application of machine learning techniques as an advanced analytic tool has proven effective in solving tasks whose inherent complexity can outreach expert-based ability. Such is the case presented by this study, in which the dataset to be leveraged is high-dimensional (79 turbines × 7 SCADA channels) and high-frequency (1 Hz). In this paper, a series of machine learning techniques is applied to the retrospective power performance analysis of a withheld test set containing SCADA data collectively representing 2 full days worth of operation at the Horns Rev I offshore wind farm. A sequential machine-learning based methodology is thoroughly explored, refined, then applied to the power performance analysis task of identifying instances of abnormal behaviour; namely instances of wind turbine under and over-performance. The results of the final analysis suggest that a normal behaviour model (NBM), consisting of a uniquely constructed artificial neural network (ANN) variant trained on abnormality filtered dataset, indeed proves effective in accomplishing the power performance analysis objective. Instances of over and under performance captured by the developed NBM network are presented and discussed, including the operation status of the turbines and the uncertainty embedded in the prediction results.

APA, Harvard, Vancouver, ISO, and other styles

36

Chang, Neng-Chieh. "Double/debiased machine learning for difference-in-differences models." Econometrics Journal 23, no. 2 (February 4, 2020): 177–91. http://dx.doi.org/10.1093/ectj/utaa001.

Full text

Abstract:

Summary This paper provides an orthogonal extension of the semiparametric difference-in-differences estimator proposed in earlier literature. The proposed estimator enjoys the so-called Neyman orthogonality (Chernozhukov et al., 2018), and thus it allows researchers to flexibly use a rich set of machine learning methods in the first-step estimation. It is particularly useful when researchers confront a high-dimensional data set in which the number of potential control variables is larger than the sample size and the conventional nonparametric estimation methods, such as kernel and sieve estimators, do not apply. I apply this orthogonal difference-in-differences estimator to evaluate the effect of tariff reduction on corruption. The empirical results show that tariff reduction decreases corruption in large magnitude.

APA, Harvard, Vancouver, ISO, and other styles

37

R.Sudha Rani, P., and Dr K.Kiran Kumar. "An Improved Particle Swarm Optimization based classification model for high dimensional medical disease prediction." International Journal of Engineering & Technology 7, no. 2.7 (March 18, 2018): 546. http://dx.doi.org/10.14419/ijet.v7i2.7.10880.

Full text

Abstract:

Recently, machine learning techniques have become popular and widely accepted for medical disease detection and classification on high dimensional datasets. Classification models is one of the essential model in machine learning models for medical disease prediction due to its fast processing speed, high efficiency and noisy datasets. Traditional machine learning models are failed to estimate the disease patterns with high true positive rate due to large number of features and data size. In this paper, a novel particle swarm optimization based hybrid classifier was implemented for medical disease prediction with high dimensions. The main objective of the feature selection based hybrid classifier is to classify the high dimensional data for large medical feature set. Proposed filtered based hybrid classifier is usually designed and implemented to improve the medical prediction rate on high dimensional data. In this work, we have used different ensemble learning models such ACO+NN, PSO+ELM, PSO+WELM to analyze the performance of proposed model(IPSO+WELM). Experimental results are evaluated on different types of medical datasets including lung cancer, diabetes, ovarian, and DLBCL-Stanford. Performance results show that proposed IPSO+WELM with ensemble model has high computational efficiency in terms of true positive rate, error rate and accuracy.

APA, Harvard, Vancouver, ISO, and other styles

38

Michau, Gabriel, Yang Hu, Thomas Palmé, and Olga Fink. "Feature learning for fault detection in high-dimensional condition monitoring signals." Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability 234, no. 1 (August 24, 2019): 104–15. http://dx.doi.org/10.1177/1748006x19868335.

Full text

Abstract:

Complex industrial systems are continuously monitored by a large number of heterogeneous sensors. The diversity of their operating conditions and the possible fault types make it impossible to collect enough data for learning all the possible fault patterns. This article proposes an integrated automatic unsupervised feature learning and one-class classification for fault detection that uses data on healthy conditions only for its training. The approach is based on stacked extreme learning machines (namely hierarchical extreme learning machines) and comprises an autoencoder, performing unsupervised feature learning, stacked with a one-class classifier monitoring the distance of the test data to the training healthy class, thereby assessing the health of the system. This study provides a comprehensive evaluation of hierarchical extreme learning machines fault detection capability compared to other machine learning approaches, such as stand-alone one-class classifiers (extreme learning machines and support vector machines); these same one-class classifiers combined with traditional dimensionality reduction methods (principal component analysis) and a deep belief network. The performance is first evaluated on a synthetic dataset that encompasses typical characteristics of condition monitoring data. Subsequently, the approach is evaluated on a real case study of a power plant fault. The proposed algorithm for fault detection, combining feature learning with the one-class classifier, demonstrates a better performance, particularly in cases where condition monitoring data contain several non-informative signals.

APA, Harvard, Vancouver, ISO, and other styles

39

Sun, Linfeng, Zhongrui Wang, Jinbao Jiang, Yeji Kim, Bomin Joo, Shoujun Zheng, Seungyeon Lee, Woo Jong Yu, Bai-Sun Kong, and Heejun Yang. "In-sensor reservoir computing for language learning via two-dimensional memristors." Science Advances 7, no. 20 (May 2021): eabg1455. http://dx.doi.org/10.1126/sciadv.abg1455.

Full text

Abstract:

The dynamic processing of optoelectronic signals carrying temporal and sequential information is critical to various machine learning applications including language processing and computer vision. Despite extensive efforts to emulate the visual cortex of human brain, large energy/time overhead and extra hardware costs are incurred by the physically separated sensing, memory, and processing units. The challenge is further intensified by the tedious training of conventional recurrent neural networks for edge deployment. Here, we report in-sensor reservoir computing for language learning. High dimensionality, nonlinearity, and fading memory for the in-sensor reservoir were achieved via two-dimensional memristors based on tin sulfide (SnS), uniquely having dual-type defect states associated with Sn and S vacancies. Our in-sensor reservoir computing demonstrates an accuracy of 91% to classify short sentences of language, thus shedding light on a low training cost and the real-time solution for processing temporal and sequential signals for machine learning applications at the edge.

APA, Harvard, Vancouver, ISO, and other styles

40

Amjad, Muhammad. "The Value of Manifold Learning Algorithms in Simplifying Complex Datasets for More Efficacious Analysis." Sciential - McMaster Undergraduate Science Journal, no. 5 (December 4, 2020): 13–20. http://dx.doi.org/10.15173/sciential.v1i5.2537.

Full text

Abstract:

Advances in manifold learning have proven to be of great benefit in reducing the dimensionality of large complex datasets. Elements in an intricate dataset will typically belong in high-dimensional space as the number of individual features or independent variables will be extensive. However, these elements can be integrated into a low-dimensional manifold with well-defined parameters. By constructing a low-dimensional manifold and embedding it into high-dimensional feature space, the dataset can be simplified for easier interpretation. In spite of this elemental dimensionality reduction, the dataset’s constituents do not lose any information, but rather filter it with the hopes of elucidating the appropriate knowledge. This paper will explore the importance of this method of data analysis, its applications, and its extensions into topological data analysis.

APA, Harvard, Vancouver, ISO, and other styles

41

Kong, Jie, Quansen Sun, Mithun Mukherjee, and Jaime Lloret. "Low-Rank Hypergraph Hashing for Large-Scale Remote Sensing Image Retrieval." Remote Sensing 12, no. 7 (April 4, 2020): 1164. http://dx.doi.org/10.3390/rs12071164.

Full text

Abstract:

As remote sensing (RS) images increase dramatically, the demand for remote sensing image retrieval (RSIR) is growing, and has received more and more attention. The characteristics of RS images, e.g., large volume, diversity and high complexity, make RSIR more challenging in terms of speed and accuracy. To reduce the retrieval complexity of RSIR, a hashing technique has been widely used for RSIR, mapping high-dimensional data into a low-dimensional Hamming space while preserving the similarity structure of data. In order to improve hashing performance, we propose a new hash learning method, named low-rank hypergraph hashing (LHH), to accomplish for the large-scale RSIR task. First, LHH employs a l2-1 norm to constrain the projection matrix to reduce the noise and redundancy among features. In addition, low-rankness is also imposed on the projection matrix to exploit its global structure. Second, LHH uses hypergraphs to capture the high-order relationship among data, and is very suitable to explore the complex structure of RS images. Finally, an iterative algorithm is developed to generate high-quality hash codes and efficiently solve the proposed optimization problem with a theoretical convergence guarantee. Extensive experiments are conducted on three RS image datasets and one natural image dataset that are publicly available. The experimental results demonstrate that the proposed LHH outperforms the existing hashing learning in RSIR tasks.

APA, Harvard, Vancouver, ISO, and other styles

42

Kang, Zhao, Yiwei Lu, Yuanzhang Su, Changsheng Li, and Zenglin Xu. "Similarity Learning via Kernel Preserving Embedding." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 4057–64. http://dx.doi.org/10.1609/aaai.v33i01.33014057.

Full text

Abstract:

Data similarity is a key concept in many data-driven applications. Many algorithms are sensitive to similarity measures. To tackle this fundamental problem, automatically learning of similarity information from data via self-expression has been developed and successfully applied in various models, such as low-rank representation, sparse subspace learning, semisupervised learning. However, it just tries to reconstruct the original data and some valuable information, e.g., the manifold structure, is largely ignored. In this paper, we argue that it is beneficial to preserve the overall relations when we extract similarity information. Specifically, we propose a novel similarity learning framework by minimizing the reconstruction error of kernel matrices, rather than the reconstruction error of original data adopted by existing work. Taking the clustering task as an example to evaluate our method, we observe considerable improvements compared to other state-ofthe-art methods. More importantly, our proposed framework is very general and provides a novel and fundamental building block for many other similarity-based tasks. Besides, our proposed kernel preserving opens up a large number of possibilities to embed high-dimensional data into low-dimensional space.

APA, Harvard, Vancouver, ISO, and other styles

43

Dong, Naghedolfeizi, Aberra, and Zeng. "Spectral–Spatial Discriminant Feature Learning for Hyperspectral Image Classification." Remote Sensing 11, no. 13 (June 29, 2019): 1552. http://dx.doi.org/10.3390/rs11131552.

Full text

Abstract:

Sparse representation classification (SRC) is being widely applied to target detection in hyperspectral images (HSI). However, due to the problem in HSI that high-dimensional data contain redundant information, SRC methods may fail to achieve high classification performance, even with a large number of spectral bands. Selecting a subset of predictive features in a high-dimensional space is an important and challenging problem for hyperspectral image classification. In this paper, we propose a novel discriminant feature learning (DFL) method, which combines spectral and spatial information into a hypergraph Laplacian. First, a subset of discriminative features is selected, which preserve the spectral structure of data and the inter- and intra-class constraints on labeled training samples. A feature evaluator is obtained by semi-supervised learning with the hypergraph Laplacian. Secondly, the selected features are mapped into a further lower-dimensional eigenspace through a generalized eigendecomposition of the Laplacian matrix. The finally extracted discriminative features are used in a joint sparsity-model algorithm. Experiments conducted with benchmark data sets and different experimental settings show that our proposed method increases classification accuracy and outperforms the state-of-the-art HSI classification methods.

APA, Harvard, Vancouver, ISO, and other styles

44

Pes, Barbara. "Learning from High-Dimensional and Class-Imbalanced Datasets Using Random Forests." Information 12, no. 8 (July 21, 2021): 286. http://dx.doi.org/10.3390/info12080286.

Full text

Abstract:

Class imbalance and high dimensionality are two major issues in several real-life applications, e.g., in the fields of bioinformatics, text mining and image classification. However, while both issues have been extensively studied in the machine learning community, they have mostly been treated separately, and little research has been thus far conducted on which approaches might be best suited to deal with datasets that are class-imbalanced and high-dimensional at the same time (i.e., with a large number of features). This work attempts to give a contribution to this challenging research area by studying the effectiveness of hybrid learning strategies that involve the integration of feature selection techniques, to reduce the data dimensionality, with proper methods that cope with the adverse effects of class imbalance (in particular, data balancing and cost-sensitive methods are considered). Extensive experiments have been carried out across datasets from different domains, leveraging a well-known classifier, the Random Forest, which has proven to be effective in high-dimensional spaces and has also been successfully applied to imbalanced tasks. Our results give evidence of the benefits of such a hybrid approach, when compared to using only feature selection or imbalance learning methods alone.

APA, Harvard, Vancouver, ISO, and other styles

45

Tizhoosh, Hamid R. "Opposition-Based Reinforcement Learning." Journal of Advanced Computational Intelligence and Intelligent Informatics 10, no. 4 (July 20, 2006): 578–85. http://dx.doi.org/10.20965/jaciii.2006.p0578.

Full text

Abstract:

Reinforcement learning is a machine intelligence scheme for learning in highly dynamic, probabilistic environments. By interaction with the environment, reinforcement agents learn optimal control policies, especially in the absence of a priori knowledge and/or a sufficiently large amount of training data. Despite its advantages, however, reinforcement learning suffers from a major drawback - high calculation cost because convergence to an optimal solution usually requires that all states be visited frequently to ensure that policy is reliable. This is not always possible, however, due to the complex, high-dimensional state space in many applications. This paper introduces opposition-based reinforcement learning, inspired by opposition-based learning, to speed up convergence. Considering opposite actions simultaneously enables individual states to be updated more than once shortening exploration and expediting convergence. Three versions of Q-learning algorithm will be given as examples. Experimental results for the grid world problem of different sizes demonstrate the superior performance of the proposed approach.

APA, Harvard, Vancouver, ISO, and other styles

46

Hammoudeh, Ahmad. "Route selection for a three-dimensional elevator using deep reinforcement learning." Building Services Engineering Research and Technology 41, no. 4 (September 19, 2019): 480–91. http://dx.doi.org/10.1177/0143624419876079.

Full text

Abstract:

While the car of the conventional elevator system moves only vertically in one dimension (up and down), the car of the three-dimensional elevator system travels in three perpendicular dimensions. The elevator moves through a vertical shaft to a certain floor and then the elevator serves multiple passengers distributed among different rooms at that floor. The controller decides which route should be taken to serve the passengers. This article proposes the use of deep reinforcement learning to select a route for the three-dimensional elevator. Deep reinforcement learning method learns from experiencing a large number of scenarios generated using Monte Carlo simulation offline. Once trained, deep reinforcement learning can select the route online. Numerical experimentations are used to show the superiority of deep reinforcement learning in finding an optimum or near optimum-route instantaneously. Although deep reinforcement learning is closer to finding the optimum route than other methods, finding an optimum route is not always guaranteed. Deep reinforcement learning has some limitations that include the long training time and the difficulties in training the neural networks. Practical application:Multidimensional elevators have been of expanding interest to the elevator industry as well as to traffic analysis engineers. This article demonstrates that deep reinforcement learning surpasses other methods in finding an optimum or near-optimum route for the three-dimensional elevator, and it also overcomes the challenges of the non-intelligent methods. This article can help enterprises that develop multidimensional elevators in overcoming the challenges of the controller in addition to boosting the feasibility of multidimensional elevators.

APA, Harvard, Vancouver, ISO, and other styles

47

Visser, Max. "Teaching giants to learn: lessons from army learning in World War II." Learning Organization 24, no. 3 (April 10, 2017): 159–68. http://dx.doi.org/10.1108/tlo-09-2016-0060.

Full text

Abstract:

Purpose This paper aims to discuss the “truism” that learning organizations cannot be large organizations and, conversely, that large organizations cannot be learning organizations. This paper analyzes learning in the German and US armies in the Second World War, based on a four-dimensional model of the learning organization. Design/methodology/approach The paper entails a secondary analysis of historical and military sources and data. Findings It is found that the German and US armies differed in learning capacity, which can be plausibly, but not exclusively, related to differences in the battlefield performance between those armies in the Second World War. Research limitations/implications The research scope of the paper is limited to the analysis of two particular armies in the Second World War. Implications of theory reside in the importance of organizational learning capacity and its dimensions for learning in current organizations. Practical implications The paper has clear practical implications for large organizations wishing to become effective and responsible learning organizations. Originality/value This is among the first organizational papers to analyze army learning in the Second World War and to derive lessons from that analysis for current large organizations.

APA, Harvard, Vancouver, ISO, and other styles

48

Vinci, Giuseppe, Peter Freeman, Jeffrey Newman, Larry Wasserman, and Christopher Genovese. "Estimating the distribution of Galaxy Morphologies on a continuous space." Proceedings of the International Astronomical Union 10, S306 (May 2014): 68–71. http://dx.doi.org/10.1017/s1743921314013568.

Full text

Abstract:

AbstractThe incredible variety of galaxy shapes cannot be summarized by human defined discrete classes of shapes without causing a possibly large loss of information. Dictionary learning and sparse coding allow us to reduce the high dimensional space of shapes into a manageable low dimensional continuous vector space. Statistical inference can be done in the reduced space via probability distribution estimation and manifold estimation.

APA, Harvard, Vancouver, ISO, and other styles

49

Wang, Chien-Chih, Chun-Heng Huang, and Chih-Jen Lin. "Subsampled Hessian Newton Methods for Supervised Learning." Neural Computation 27, no. 8 (August 2015): 1766–95. http://dx.doi.org/10.1162/neco_a_00751.

Full text

Abstract:

Newton methods can be applied in many supervised learning approaches. However, for large-scale data, the use of the whole Hessian matrix can be time-consuming. Recently, subsampled Newton methods have been proposed to reduce the computational time by using only a subset of data for calculating an approximation of the Hessian matrix. Unfortunately, we find that in some situations, the running speed is worse than the standard Newton method because cheaper but less accurate search directions are used. In this work, we propose some novel techniques to improve the existing subsampled Hessian Newton method. The main idea is to solve a two-dimensional subproblem per iteration to adjust the search direction to better minimize the second-order approximation of the function value. We prove the theoretical convergence of the proposed method. Experiments on logistic regression, linear SVM, maximum entropy, and deep networks indicate that our techniques significantly reduce the running time of the subsampled Hessian Newton method. The resulting algorithm becomes a compelling alternative to the standard Newton method for large-scale data classification.

APA, Harvard, Vancouver, ISO, and other styles

50

Ma, Wenye. "Projective Quadratic Regression for Online Learning." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 5093–100. http://dx.doi.org/10.1609/aaai.v34i04.5951.

Full text

Abstract:

This paper considers online convex optimization (OCO) problems - the paramount framework for online learning algorithm design. The loss function of learning task in OCO setting is based on streaming data so that OCO is a powerful tool to model large scale applications such as online recommender systems. Meanwhile, real-world data are usually of extreme high-dimensional due to modern feature engineering techniques so that the quadratic regression is impractical. Factorization Machine as well as its variants are efficient models for capturing feature interactions with low-rank matrix model but they can't fulfill the OCO setting due to their non-convexity. In this paper, We propose a projective quadratic regression (PQR) model. First, it can capture the import second-order feature information. Second, it is a convex model, so the requirements of OCO are fulfilled and the global optimal solution can be achieved. Moreover, existing modern online optimization methods such as Online Gradient Descent (OGD) or Follow-The-Regularized-Leader (FTRL) can be applied directly. In addition, by choosing a proper hyper-parameter, we show that it has the same order of space and time complexity as the linear model and thus can handle high-dimensional data. Experimental results demonstrate the performance of the proposed PQR model in terms of accuracy and efficiency by comparing with the state-of-the-art methods.

APA, Harvard, Vancouver, ISO, and other styles

Journal articles on the topic 'Large dimensional learning'

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles