Готові списки джерел за темами / Distributed optimization and learning

Зміст

  1. Статті в журналах
  2. Дисертації
  3. Книги
  4. Частини книг
  5. Тези доповідей конференцій
  6. Звіти організацій

Добірка наукової літератури з теми "Distributed optimization and learning"

Автор: Grafiati
Опубліковано: 15 лютого 2025

Оформте джерело за APA, MLA, Chicago, Harvard та іншими стилями

Оберіть тип джерела:

Ознайомтеся зі списками актуальних статей, книг, дисертацій, тез та інших наукових джерел на тему "Distributed optimization and learning".

Біля кожної праці в переліку літератури доступна кнопка «Додати до бібліографії». Скористайтеся нею – і ми автоматично оформимо бібліографічне посилання на обрану працю в потрібному вам стилі цитування: APA, MLA, «Гарвард», «Чикаго», «Ванкувер» тощо.

Також ви можете завантажити повний текст наукової публікації у форматі «.pdf» та прочитати онлайн анотацію до роботи, якщо відповідні параметри наявні в метаданих.

Статті в журналах з теми "Distributed optimization and learning"

1

Kamalesh, Kamalesh, and Dr Gobi Natesan. "Machine Learning-Driven Analysis of Distributed Computing Systems: Exploring Optimization and Efficiency." International Journal of Research Publication and Reviews 5, no. 3 (March 9, 2024): 3979–83. http://dx.doi.org/10.55248/gengpi.5.0324.0786.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
2

Mertikopoulos, Panayotis, E. Veronica Belmega, Romain Negrel, and Luca Sanguinetti. "Distributed Stochastic Optimization via Matrix Exponential Learning." IEEE Transactions on Signal Processing 65, no. 9 (May 1, 2017): 2277–90. http://dx.doi.org/10.1109/tsp.2017.2656847.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
3

Gratton, Cristiano, Naveen K. D. Venkategowda, Reza Arablouei, and Stefan Werner. "Privacy-Preserved Distributed Learning With Zeroth-Order Optimization." IEEE Transactions on Information Forensics and Security 17 (2022): 265–79. http://dx.doi.org/10.1109/tifs.2021.3139267.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
4

Blot, Michael, David Picard, Nicolas Thome, and Matthieu Cord. "Distributed optimization for deep learning with gossip exchange." Neurocomputing 330 (February 2019): 287–96. http://dx.doi.org/10.1016/j.neucom.2018.11.002.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
5

Young, M. Todd, Jacob D. Hinkle, Ramakrishnan Kannan, and Arvind Ramanathan. "Distributed Bayesian optimization of deep reinforcement learning algorithms." Journal of Parallel and Distributed Computing 139 (May 2020): 43–52. http://dx.doi.org/10.1016/j.jpdc.2019.07.008.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
6

Nedic, Angelia. "Distributed Gradient Methods for Convex Machine Learning Problems in Networks: Distributed Optimization." IEEE Signal Processing Magazine 37, no. 3 (May 2020): 92–101. http://dx.doi.org/10.1109/msp.2020.2975210.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
7

Lin, I.-Cheng. "Learning and Optimization over Robust Networked Systems." ACM SIGMETRICS Performance Evaluation Review 52, no. 3 (January 9, 2025): 23–26. https://doi.org/10.1145/3712170.3712179.

Повний текст джерела
Анотація:
Research Summary: 1 Introduction Networked systems are ubiquitous in our daily lives, playing a critical role across a wide range of scientific fields, including communication, machine learning, optimization, control, biology, economics, and social sciences. In machine learning and optimization, distributed learning and distributed optimization exploit the structure of such networked systems, which enables training models across multiple users [1] and solving complex problems by partitioning tasks among interconnected agents [2]. This significantly enhances both scalability and efficiency of the learning/optimization task. In control systems, for example, networked architectures synchronize the operations of different parts of the system like sensors, controllers etc. across distributed environments, ensuring the overall system's stability and performance [3]. In economics, networked systems were utilized to capture the interdependencies between different part of the economic system, allowing further insights of market dynamics [4]. In the social sciences, specifically social networks, study the propagation of idea, behaviors and opinions, offering a deeper understanding of societal structures and its dynamics [5].
Стилі APA, Harvard, Vancouver, ISO та ін.
8

Gao, Hongchang. "Distributed Stochastic Nested Optimization for Emerging Machine Learning Models: Algorithm and Theory." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 13 (June 26, 2023): 15437. http://dx.doi.org/10.1609/aaai.v37i13.26804.

Повний текст джерела
Анотація:
Traditional machine learning models can be formulated as the expected risk minimization (ERM) problem: minw∈Rd Eξ [l(w; ξ)], where w ∈ Rd denotes the model parameter, ξ represents training samples, l(·) is the loss function. Numerous optimization algorithms, such as stochastic gradient descent (SGD), have been developed to solve the ERM problem. However, a wide range of emerging machine learning models are beyond this class of optimization problems, such as model-agnostic meta-learning (Finn, Abbeel, and Levine 2017). Of particular interest of my research is the stochastic nested optimization (SNO) problem, whose objective function has a nested structure. Specifically, I have been focusing on two instances of this kind of problem: stochastic compositional optimization (SCO) problems, which cover meta-learning, area-under-the-precision recall-curve optimization, contrastive self-supervised learning, etc., and stochastic bilevel optimization (SBO) problems, which can be applied to meta-learning, hyperparameter optimization, neural network architecture search, etc. With the emergence of large-scale distributed data, such as the user data generated on mobile devices or intelligent hardware, it is imperative to develop distributed optimization algorithms for SNO (Distributed SNO). A significant challenge for optimizing distributed SNO problems lies in that the stochastic (hyper-)gradient is a biased estimation of the full gradient. Thus, existing distributed optimization algorithms when applied to them suffer from slow convergence rates. In this talk, I will discuss my recent works about distributed SCO (Gao and Huang 2021; Gao, Li, and Huang 2022) and distributed SBO (Gao, Gu, and Thai 2022; Gao 2022) under both centralized and decentralized settings, including algorithmic details about reducing the bias of stochastic gradient, theoretical convergence rate, and practical machine learning applications, and then highlight challenges for future research.
Стилі APA, Harvard, Vancouver, ISO та ін.
9

Choi, Dojin, Jiwon Wee, Sangho Song, Hyeonbyeong Lee, Jongtae Lim, Kyoungsoo Bok, and Jaesoo Yoo. "k-NN Query Optimization for High-Dimensional Index Using Machine Learning." Electronics 12, no. 11 (May 24, 2023): 2375. http://dx.doi.org/10.3390/electronics12112375.

Повний текст джерела
Анотація:
In this study, we propose three k-nearest neighbor (k-NN) optimization techniques for a distributed, in-memory-based, high-dimensional indexing method to speed up content-based image retrieval. The proposed techniques perform distributed, in-memory, high-dimensional indexing-based k-NN query optimization: a density-based optimization technique that performs k-NN optimization using data distribution; a cost-based optimization technique using query processing cost statistics; and a learning-based optimization technique using a deep learning model, based on query logs. The proposed techniques were implemented on Spark, which supports a master/slave model for large-scale distributed processing. We showed the superiority and validity of the proposed techniques through various performance evaluations, based on high-dimensional data.
Стилі APA, Harvard, Vancouver, ISO та ін.
10

Yang, Peng, and Ping Li. "Distributed Primal-Dual Optimization for Online Multi-Task Learning." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 6631–38. http://dx.doi.org/10.1609/aaai.v34i04.6139.

Повний текст джерела
Анотація:
Conventional online multi-task learning algorithms suffer from two critical limitations: 1) Heavy communication caused by delivering high velocity of sequential data to a central machine; 2) Expensive runtime complexity for building task relatedness. To address these issues, in this paper we consider a setting where multiple tasks are geographically located in different places, where one task can synchronize data with others to leverage knowledge of related tasks. Specifically, we propose an adaptive primal-dual algorithm, which not only captures task-specific noise in adversarial learning but also carries out a projection-free update with runtime efficiency. Moreover, our model is well-suited to decentralized periodic-connected tasks as it allows the energy-starved or bandwidth-constraint tasks to postpone the update. Theoretical results demonstrate the convergence guarantee of our distributed algorithm with an optimal regret. Empirical results confirm that the proposed model is highly effective on various real-world datasets.
Стилі APA, Harvard, Vancouver, ISO та ін.
Більше джерел

Дисертації з теми "Distributed optimization and learning"

1

Funkquist, Mikaela, and Minghua Lu. "Distributed Optimization Through Deep Reinforcement Learning." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-293878.

Повний текст джерела
Анотація:
Reinforcement learning methods allows self-learningagents to play video- and board games autonomously. Thisproject aims to study the efficiency of the reinforcement learningalgorithms Q-learning and deep Q-learning for dynamical multi-agent problems. The goal is to train robots to optimally navigatethrough a warehouse without colliding.A virtual environment was created, in which the learning algo-rithms were tested by simulating moving agents. The algorithms’efficiency was evaluated by how fast the agents learned to performpredetermined tasks.The results show that Q-learning excels in simple problemswith few agents, quickly solving systems with two active agents.Deep Q-learning proved to be better suited for complex systemscontaining several agents, though cases of sub-optimal movementwere still possible. Both algorithms showed great potential fortheir respective areas however improvements still need to be madefor any real-world use.
Förstärkningsinlärningsmetoder tillåter självlärande enheter att spela video- och brädspel autonomt. Projektet siktar på att studera effektiviteten hos förstärkningsinlärningsmetoderna Q-learning och deep Q-learning i dynamiska problem. Målet är att träna upp robotar så att de kan röra sig genom ett varuhus på bästa sätt utan att kollidera. En virtuell miljö skapades, i vilken algoritmerna testades genom att simulera agenter som rörde sig. Algoritmernas effektivitet utvärderades av hur snabbt agenterna lärde sig att utföra förutbestämda uppgifter. Resultatet visar att Q-learning fungerar bra för enkla problem med få agenter, där system med två aktiva agenter löstes snabbt. Deep Q-learning fungerar bättre för mer komplexa system som innehåller fler agenter, men fall med suboptimala rörelser uppstod. Båda algoritmerna visade god potential inom deras respektive områden, däremot måste förbättringar göras innan de kan användas i verkligheten.
Kandidatexjobb i elektroteknik 2020, KTH, Stockholm
Стилі APA, Harvard, Vancouver, ISO та ін.
2

Konečný, Jakub. "Stochastic, distributed and federated optimization for machine learning." Thesis, University of Edinburgh, 2017. http://hdl.handle.net/1842/31478.

Повний текст джерела
Анотація:
We study optimization algorithms for the finite sum problems frequently arising in machine learning applications. First, we propose novel variants of stochastic gradient descent with a variance reduction property that enables linear convergence for strongly convex objectives. Second, we study distributed setting, in which the data describing the optimization problem does not fit into a single computing node. In this case, traditional methods are inefficient, as the communication costs inherent in distributed optimization become the bottleneck. We propose a communication-efficient framework which iteratively forms local subproblems that can be solved with arbitrary local optimization algorithms. Finally, we introduce the concept of Federated Optimization/Learning, where we try to solve the machine learning problems without having data stored in any centralized manner. The main motivation comes from industry when handling user-generated data. The current prevalent practice is that companies collect vast amounts of user data and store them in datacenters. An alternative we propose is not to collect the data in first place, and instead occasionally use the computational power of users' devices to solve the very same optimization problems, while alleviating privacy concerns at the same time. In such setting, minimization of communication rounds is the primary goal, and we demonstrate that solving the optimization problems in such circumstances is conceptually tractable.
Стилі APA, Harvard, Vancouver, ISO та ін.
3

Armond, Kenneth C. Jr. "Distributed Support Vector Machine Learning." ScholarWorks@UNO, 2008. http://scholarworks.uno.edu/td/711.

Повний текст джерела
Анотація:
Support Vector Machines (SVMs) are used for a growing number of applications. A fundamental constraint on SVM learning is the management of the training set. This is because the order of computations goes as the square of the size of the training set. Typically, training sets of 1000 (500 positives and 500 negatives, for example) can be managed on a PC without hard-drive thrashing. Training sets of 10,000 however, simply cannot be managed with PC-based resources. For this reason most SVM implementations must contend with some kind of chunking process to train parts of the data at a time (10 chunks of 1000, for example, to learn the 10,000). Sequential and multi-threaded chunking methods provide a way to run the SVM on large datasets while retaining accuracy. The multi-threaded distributed SVM described in this thesis is implemented using Java RMI, and has been developed to run on a network of multi-core/multi-processor computers.
Стилі APA, Harvard, Vancouver, ISO та ін.
4

Patvarczki, Jozsef. "Layout Optimization for Distributed Relational Databases Using Machine Learning." Digital WPI, 2012. https://digitalcommons.wpi.edu/etd-dissertations/291.

Повний текст джерела
Анотація:
A common problem when running Web-based applications is how to scale-up the database. The solution to this problem usually involves having a smart Database Administrator determine how to spread the database tables out amongst computers that will work in parallel. Laying out database tables across multiple machines so they can act together as a single efficient database is hard. Automated methods are needed to help eliminate the time required for database administrators to create optimal configurations. There are four operators that we consider that can create a search space of possible database layouts: 1) denormalizing, 2) horizontally partitioning, 3) vertically partitioning, and 4) fully replicating. Textbooks offer general advice that is useful for dealing with extreme cases - for instance you should fully replicate a table if the level of insert to selects is close to zero. But even this seemingly obvious statement is not necessarily one that will lead to a speed up once you take into account that some nodes might be a bottle neck. There can be complex interactions between the 4 different operators which make it even more difficult to predict what the best thing to do is. Instead of using best practices to do database layout, we need a system that collects empirical data on when these 4 different operators are effective. We have implemented a state based search technique to try different operators, and then we used the empirically measured data to see if any speed up occurred. We recognized that the costs of creating the physical database layout are potentially large, but it is necessary since we want to know the "Ground Truth" about what is effective and under what conditions. After creating a dataset where these four different operators have been applied to make different databases, we can employ machine learning to induce rules to help govern the physical design of the database across an arbitrary number of computer nodes. This learning process, in turn, would allow the database placement algorithm to get better over time as it trains over a set of examples. What this algorithm calls for is that it will try to learn 1) "What is a good database layout for a particular application given a query workload?" and 2) "Can this algorithm automatically improve itself in making recommendations by using machine learned rules to try to generalize when it makes sense to apply each of these operators?" There has been considerable research done in parallelizing databases where large amounts of data are shipped from one node to another to answer a single query. Sometimes the costs of shipping the data back and forth might be high, so in this work we assume that it might be more efficient to create a database layout where each query can be answered by a single node. To make this assumption requires that all the incoming query templates are known beforehand. This requirement can easily be satisfied in the case of a Web-based application due to the characteristic that users typically interact with the system through a web interface such as web forms. In this case, unseen queries are not necessarily answerable, without first possibly reconstructing the data on a single machine. Prior knowledge of these exact query templates allows us to select the best possible database table placements across multiple nodes. But in the case of trying to improve the efficiency of a Web-based application, a web site provider might feel that they are willing to suffer the inconvenience of not being able to answer an arbitrary query, if they are in turn provided with a system that runs more efficiently.
Стилі APA, Harvard, Vancouver, ISO та ін.
5

Ouyang, Hua. "Optimal stochastic and distributed algorithms for machine learning." Diss., Georgia Institute of Technology, 2013. http://hdl.handle.net/1853/49091.

Повний текст джерела
Анотація:
Stochastic and data-distributed optimization algorithms have received lots of attention from the machine learning community due to the tremendous demand from the large-scale learning and the big-data related optimization. A lot of stochastic and deterministic learning algorithms are proposed recently under various application scenarios. Nevertheless, many of these algorithms are based on heuristics and their optimality in terms of the generalization error is not sufficiently justified. In this talk, I will explain the concept of an optimal learning algorithm, and show that given a time budget and proper hypothesis space, only those achieving the lower bounds of the estimation error and the optimization error are optimal. Guided by this concept, we investigated the stochastic minimization of nonsmooth convex loss functions, a central problem in machine learning. We proposed a novel algorithm named Accelerated Nonsmooth Stochastic Gradient Descent, which exploits the structure of common nonsmooth loss functions to achieve optimal convergence rates for a class of problems including SVMs. It is the first stochastic algorithm that can achieve the optimal O(1/t) rate for minimizing nonsmooth loss functions. The fast rates are confirmed by empirical comparisons with state-of-the-art algorithms including the averaged SGD. The Alternating Direction Method of Multipliers (ADMM) is another flexible method to explore function structures. In the second part we proposed stochastic ADMM that can be applied to a general class of convex and nonsmooth functions, beyond the smooth and separable least squares loss used in lasso. We also demonstrate the rates of convergence for our algorithm under various structural assumptions of the stochastic function: O(1/sqrt{t}) for convex functions and O(log t/t) for strongly convex functions. A novel application named Graph-Guided SVM is proposed to demonstrate the usefulness of our algorithm. We also extend the scalability of stochastic algorithms to nonlinear kernel machines, where the problem is formulated as a constrained dual quadratic optimization. The simplex constraint can be handled by the classic Frank-Wolfe method. The proposed stochastic Frank-Wolfe methods achieve comparable or even better accuracies than state-of-the-art batch and online kernel SVM solvers, and are significantly faster. The last part investigates the problem of data-distributed learning. We formulate it as a consensus-constrained optimization problem and solve it with ADMM. It turns out that the underlying communication topology is a key factor in achieving a balance between a fast learning rate and computation resource consumption. We analyze the linear convergence behavior of consensus ADMM so as to characterize the interplay between the communication topology and the penalty parameters used in ADMM. We observe that given optimal parameters, the complete bipartite and the master-slave graphs exhibit the fastest convergence, followed by bi-regular graphs.
Стилі APA, Harvard, Vancouver, ISO та ін.
6

El, Gamal Mostafa. "Distributed Statistical Learning under Communication Constraints." Digital WPI, 2017. https://digitalcommons.wpi.edu/etd-dissertations/314.

Повний текст джерела
Анотація:
"In this thesis, we study distributed statistical learning, in which multiple terminals, connected by links with limited capacity, cooperate to perform a learning task. As the links connecting the terminals have limited capacity, the messages exchanged between the terminals have to be compressed. The goal of this thesis is to investigate how to compress the data observations at multiple terminals and how to use the compressed data for inference. We first focus on the distributed parameter estimation problem, in which terminals send messages related to their local observations using limited rates to a fusion center that will obtain an estimate of a parameter related to the observations of all terminals. It is well known that if the transmission rates are in the Slepian-Wolf region, the fusion center can fully recover all observations and hence can construct an estimator having the same performance as that of the centralized case. One natural question is whether Slepian-Wolf rates are necessary to achieve the same estimation performance as that of the centralized case. In this thesis, we show that the answer to this question is negative. We then examine the optimality of data dimensionality reduction via sufficient statistics compression in distributed parameter estimation problems. The data dimensionality reduction step is often needed especially if the data has a very high dimension and the communication rate is not as high as the one characterized above. We show that reducing the dimensionality by extracting sufficient statistics of the parameter to be estimated does not degrade the overall estimation performance in the presence of communication constraints. We further analyze the optimal estimation performance in the presence of communication constraints and we verify the derived bound using simulations. Finally, we study distributed optimization problems, for which we examine the randomized distributed coordinate descent algorithm with quantized updates. In the literature, the iteration complexity of the randomized distributed coordinate descent algorithm has been characterized under the assumption that machines can exchange updates with an infinite precision. We consider a practical scenario in which the messages exchange occurs over channels with finite capacity, and hence the updates have to be quantized. We derive sufficient conditions on the quantization error such that the algorithm with quantized update still converge."
Стилі APA, Harvard, Vancouver, ISO та ін.
7

Dai, Wei. "Learning with Staleness." Research Showcase @ CMU, 2018. http://repository.cmu.edu/dissertations/1209.

Повний текст джерела
Анотація:
A fundamental assumption behind most machine learning (ML) algorithms and analyses is the sequential execution. That is, any update to the ML model can be immediately applied and the new model is always available for the next algorithmic step. This basic assumption, however, can be costly to realize, when the computation is carried out across multiple machines, linked by commodity networks that are usually 104 times slower than the memory speed due to fundamental hardware limitations. As a result, concurrent ML computation in the distributed settings often needs to handle delayed updates and perform learning in the presence of staleness. This thesis characterizes learning with staleness from three directions: (1) We extend the theoretical analyses of a number of classical ML algorithms, including stochastic gradient descent, proximal gradient descent on non-convex problems, and Frank-Wolfe algorithms, to explicitly incorporate staleness into their convergence characterizations. (2)We conduct simulation and large-scale distributed experiments to study the empirical effects of staleness on ML algorithms under indeterministic executions. Our results reveal that staleness is a key parameter governing the convergence speed for all considered ML algorithms, with varied ramifications. (3) We design staleness-minimizing parameter server systems by optimizing synchronization methods to effectively reduce the runtime staleness. The proposed optimization of a bounded consistency model utilizes the additional network bandwidths to communicate updates eagerly, relieving users of the burden to tune the staleness level. By minimizing staleness at the framework level, our system stabilizes diverging optimization paths and substantially accelerates convergence across ML algorithms without any modification to the ML programs.
Стилі APA, Harvard, Vancouver, ISO та ін.
8

Lu, Yumao. "Kernel optimization and distributed learning algorithms for support vector machines." Diss., Restricted to subscribing institutions, 2005. http://uclibs.org/PID/11984.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
9

Dinh, The Canh. "Distributed Algorithms for Fast and Personalized Federated Learning." Thesis, The University of Sydney, 2023. https://hdl.handle.net/2123/30019.

Повний текст джерела
Анотація:
The significant increase in the number of cutting-edge user equipment (UE) results in the phenomenal growth of the data volume generated at the edge. This shift fuels the booming trend of an emerging technique named Federated Learning. In contrast to traditional methods in which data is collected and processed centrally, FL builds a global model from contributions of UE's model without sending private data then effectively ensures data privacy. However, FL faces challenges in non-identically distributed (non-IID) data, communication cost, and convergence rate. Firstly, we propose first-order optimization FL algorithms named FedApprox and FEDL to improve the convergence rate. We propose FedApprox exploiting proximal stochastic variance-reduced gradient methods and extract insights from convergence conditions via the algorithm’s parameter control. We then propose FEDL to handle heterogeneous UE data and characterize the trade-off between local computation and global communication. Experimentally, FedApprox outperforms vanilla FedAvg while FEDL outperforms FedApprox and FedAvg. Secondly, we consider the communication between edges to be more costly than local computational overhead. We propose DONE, a distributed approximate Newton-type algorithm for communication-efficient federated edge learning. DONE approximates Newton direction using classical Richardson iteration on each edge. Experimentally, DONE attains a comparable performance to Newton’s method and outperforms first-order algorithms. Finally, we address the non-IID issue by proposing pFedMe, a personalized FL algorithm using Moreau envelopes. pFedMe achieves quadratic speedup for strongly convex and sublinear speedup of order 2/3 for smooth nonconvex objectives. We then propose FedU, a Federated Multitask Learning algorithm using Laplacian regularization to leverage the relationships among the users' models. Experimentally, pFedMe excels FedAvg and Per-FedAvg while FedU outperforms pFedMe and MOCHA.
Стилі APA, Harvard, Vancouver, ISO та ін.
10

Reddi, Sashank Jakkam. "New Optimization Methods for Modern Machine Learning." Research Showcase @ CMU, 2017. http://repository.cmu.edu/dissertations/1116.

Повний текст джерела
Анотація:
Modern machine learning systems pose several new statistical, scalability, privacy and ethical challenges. With the advent of massive datasets and increasingly complex tasks, scalability has especially become a critical issue in these systems. In this thesis, we focus on fundamental challenges related to scalability, such as computational and communication efficiency, in modern machine learning applications. The underlying central message of this thesis is that classical statistical thinking leads to highly effective optimization methods for modern big data applications. The first part of the thesis investigates optimization methods for solving large-scale nonconvex Empirical Risk Minimization (ERM) problems. Such problems have surged into prominence, notably through deep learning, and have led to exciting progress. However, our understanding of optimization methods suitable for these problems is still very limited. We develop and analyze a new line of optimization methods for nonconvex ERM problems, based on the principle of variance reduction. We show that our methods exhibit fast convergence to stationary points and improve the state-of-the-art in several nonconvex ERM settings, including nonsmooth and constrained ERM. Using similar principles, we also develop novel optimization methods that provably converge to second-order stationary points. Finally, we show that the key principles behind our methods can be generalized to overcome challenges in other important problems such as Bayesian inference. The second part of the thesis studies two critical aspects of modern distributed machine learning systems — asynchronicity and communication efficiency of optimization methods. We study various asynchronous stochastic algorithms with fast convergence for convex ERM problems and show that these methods achieve near-linear speedups in sparse settings common to machine learning. Another key factor governing the overall performance of a distributed system is its communication efficiency. Traditional optimization algorithms used in machine learning are often ill-suited for distributed environments with high communication cost. To address this issue, we dis- cuss two different paradigms to achieve communication efficiency of algorithms in distributed environments and explore new algorithms with better communication complexity.
Стилі APA, Harvard, Vancouver, ISO та ін.
Більше джерел

Книги з теми "Distributed optimization and learning"

1

Jiang, Jiawei, Bin Cui, and Ce Zhang. Distributed Machine Learning and Gradient Optimization. Singapore: Springer Singapore, 2022. http://dx.doi.org/10.1007/978-981-16-3420-8.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
2

Wang, Huiwei, Huaqing Li, and Bo Zhou. Distributed Optimization, Game and Learning Algorithms. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-33-4528-7.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
3

Joshi, Gauri. Optimization Algorithms for Distributed Machine Learning. Cham: Springer International Publishing, 2023. http://dx.doi.org/10.1007/978-3-031-19067-4.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
4

Tatarenko, Tatiana. Game-Theoretic Learning and Distributed Optimization in Memoryless Multi-Agent Systems. Cham: Springer International Publishing, 2017. http://dx.doi.org/10.1007/978-3-319-65479-9.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
5

Oblinger, Diana G. Distributed learning. Boulder, Colo: CAUSE, 1996.

Знайти повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
6

Majhi, Sudhan, Rocío Pérez de Prado, and Chandrappa Dasanapura Nanjundaiah, eds. Distributed Computing and Optimization Techniques. Singapore: Springer Nature Singapore, 2022. http://dx.doi.org/10.1007/978-981-19-2281-7.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
7

Giselsson, Pontus, and Anders Rantzer, eds. Large-Scale and Distributed Optimization. Cham: Springer International Publishing, 2018. http://dx.doi.org/10.1007/978-3-319-97478-1.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
8

Lü, Qingguo, Xiaofeng Liao, Huaqing Li, Shaojiang Deng, and Shanfu Gao. Distributed Optimization in Networked Systems. Singapore: Springer Nature Singapore, 2023. http://dx.doi.org/10.1007/978-981-19-8559-1.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
9

Abdulrahman Younis Ali Younis Kalbat. Distributed and Large-Scale Optimization. [New York, N.Y.?]: [publisher not identified], 2016.

Знайти повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
10

Otto, Daniel, Gianna Scharnberg, Michael Kerres, and Olaf Zawacki-Richter, eds. Distributed Learning Ecosystems. Wiesbaden: Springer Fachmedien Wiesbaden, 2023. http://dx.doi.org/10.1007/978-3-658-38703-7.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
Більше джерел

Частини книг з теми "Distributed optimization and learning"

1

Joshi, Gauri, and Shiqiang Wang. "Communication-Efficient Distributed Optimization Algorithms." In Federated Learning, 125–43. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-030-96896-0_6.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
2

Jiang, Jiawei, Bin Cui, and Ce Zhang. "Distributed Gradient Optimization Algorithms." In Distributed Machine Learning and Gradient Optimization, 57–114. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-16-3420-8_3.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
3

Jiang, Jiawei, Bin Cui, and Ce Zhang. "Distributed Machine Learning Systems." In Distributed Machine Learning and Gradient Optimization, 115–66. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-16-3420-8_4.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
4

Joshi, Gauri. "Distributed Optimization in Machine Learning." In Synthesis Lectures on Learning, Networks, and Algorithms, 1–12. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-031-19067-4_1.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
5

Lin, Zhouchen, Huan Li, and Cong Fang. "ADMM for Distributed Optimization." In Alternating Direction Method of Multipliers for Machine Learning, 207–40. Singapore: Springer Nature Singapore, 2022. http://dx.doi.org/10.1007/978-981-16-9840-8_6.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
6

Jiang, Jiawei, Bin Cui, and Ce Zhang. "Basics of Distributed Machine Learning." In Distributed Machine Learning and Gradient Optimization, 15–55. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-16-3420-8_2.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
7

Scheidegger, Carre, Arpit Shah, and Dan Simon. "Distributed Learning with Biogeography-Based Optimization." In Lecture Notes in Computer Science, 203–15. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011. http://dx.doi.org/10.1007/978-3-642-21827-9_21.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
8

González-Mendoza, Miguel, Neil Hernández-Gress, and André Titli. "Quadratic Optimization Fine Tuning for the Learning Phase of SVM." In Advanced Distributed Systems, 347–57. Berlin, Heidelberg: Springer Berlin Heidelberg, 2005. http://dx.doi.org/10.1007/11533962_31.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
9

Wang, Huiwei, Huaqing Li, and Bo Zhou. "Cooperative Distributed Optimization in Multiagent Networks with Delays." In Distributed Optimization, Game and Learning Algorithms, 1–17. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-33-4528-7_1.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
10

Wang, Huiwei, Huaqing Li, and Bo Zhou. "Constrained Consensus of Multi-agent Systems with Time-Varying Topology." In Distributed Optimization, Game and Learning Algorithms, 19–37. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-33-4528-7_2.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.

Тези доповідей конференцій з теми "Distributed optimization and learning"

1

Patil, Aditya, Sanket Lodha, Sonal Deshmukh, Rupali S. Joshi, Vaishali Patil, and Sudhir Chitnis. "Battery Optimization Using Machine Learning." In 2024 IEEE International Conference on Blockchain and Distributed Systems Security (ICBDS), 1–5. IEEE, 2024. https://doi.org/10.1109/icbds61829.2024.10837428.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
2

Khan, Malak Abid Ali, Luo Senlin, Hongbin Ma, Abdul Khalique Shaikh, Ahlam Almusharraf, and Imran Khan Mirani. "Optimization of LoRa for Distributed Environments Based on Machine Learning." In 2024 IEEE Asia Pacific Conference on Wireless and Mobile (APWiMob), 137–42. IEEE, 2024. https://doi.org/10.1109/apwimob64015.2024.10792952.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
3

Chao, Liangchen, Bo Zhang, Hengpeng Guo, Fangheng Ji, and Junfeng Li. "UAV Swarm Collaborative Transmission Optimization for Machine Learning Tasks." In 2024 IEEE 30th International Conference on Parallel and Distributed Systems (ICPADS), 504–11. IEEE, 2024. http://dx.doi.org/10.1109/icpads63350.2024.00072.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
4

Shamir, Ohad, and Nathan Srebro. "Distributed stochastic optimization and learning." In 2014 52nd Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE, 2014. http://dx.doi.org/10.1109/allerton.2014.7028543.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
5

Hulse, Daniel, Brandon Gigous, Kagan Tumer, Christopher Hoyle, and Irem Y. Tumer. "Towards a Distributed Multiagent Learning-Based Design Optimization Method." In ASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. American Society of Mechanical Engineers, 2017. http://dx.doi.org/10.1115/detc2017-68042.

Повний текст джерела
Анотація:
Complex engineered systems create many design challenges for engineers and organizations because of the interactions between subsystems and desire for optimality. In some conceptual-level optimizations, the design problem is simplified to consider the most important variables in an all-in-one optimization framework. This work introduces a stochastic optimization method which uses a distributed multiagent design method in which action-value based learning agents make individual design choices for each component. These agents use a probabilistic action-selection strategy based on the learned objective values of each action. This distributed multiagent system is applied to a simple quadrotor optimization problem in an all-in-one optimization framework, and compared with the performance of centralized methods. Results show the multiagent system is capable of finding comparable designs to centralized methods in a similar amount of computational time. This demonstrates the potential merit of a multiagent approach for complex systems design.
Стилі APA, Harvard, Vancouver, ISO та ін.
6

Li, Naihao, Jiaqi Wang, Xu Liu, Lanfeng Wang, and Long Zhang. "Contrastive Learning-based Meta-Learning Sequential Recommendation." In 2024 International Conference on Distributed Computing and Optimization Techniques (ICDCOT). IEEE, 2024. http://dx.doi.org/10.1109/icdcot61034.2024.10515699.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
7

Vaidya, Nitin H. "Security and Privacy for Distributed Optimization & Distributed Machine Learning." In PODC '21: ACM Symposium on Principles of Distributed Computing. New York, NY, USA: ACM, 2021. http://dx.doi.org/10.1145/3465084.3467485.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
8

Liao, Leonardo, and Yongqiang Wu. "Distributed Polytope ARTMAP: A Vigilance-Free ART Network for Distributed Supervised Learning." In 2009 International Joint Conference on Computational Sciences and Optimization, CSO. IEEE, 2009. http://dx.doi.org/10.1109/cso.2009.63.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
9

Wang, Shoujin, Fan Wang, and Yu Zhang. "Learning Rate Decay Algorithm Based on Mutual Information in Deep Learning." In 2024 International Conference on Distributed Computing and Optimization Techniques (ICDCOT). IEEE, 2024. http://dx.doi.org/10.1109/icdcot61034.2024.10515368.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
10

Anand, Aditya, Lakshay Rastogi, Ansh Agarwaal, and Shashank Bhardwaj. "Refraction-Learning Based Whale Optimization Algorithm with Opposition-Learning and Adaptive Parameter Optimization." In 2024 Third International Conference on Distributed Computing and Electrical Circuits and Electronics (ICDCECE). IEEE, 2024. http://dx.doi.org/10.1109/icdcece60827.2024.10548420.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.

Звіти організацій з теми "Distributed optimization and learning"

1

Stuckey, Peter, and Toby Walsh. Learning within Optimization. Fort Belvoir, VA: Defense Technical Information Center, April 2013. http://dx.doi.org/10.21236/ada575367.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
2

Nygard, Kendall E. Distributed Optimization in Aircraft Mission Scheduling. Fort Belvoir, VA: Defense Technical Information Center, May 1995. http://dx.doi.org/10.21236/ada300064.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
3

Meyer, Robert R. Large-Scale Optimization Via Distributed Systems. Fort Belvoir, VA: Defense Technical Information Center, November 1989. http://dx.doi.org/10.21236/ada215136.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
4

Shead, Timothy, Jonathan Berry, Cynthia Phillips, and Jared Saia. Information-Theoretically Secure Distributed Machine Learning. Office of Scientific and Technical Information (OSTI), November 2019. http://dx.doi.org/10.2172/1763277.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
5

Graesser, Arthur C., and Robert A. Wisher. Question Generation as a Learning Multiplier in Distributed Learning Environments. Fort Belvoir, VA: Defense Technical Information Center, October 2001. http://dx.doi.org/10.21236/ada399456.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
6

Voon, B. K., and M. A. Austin. Structural Optimization in a Distributed Computing Environment. Fort Belvoir, VA: Defense Technical Information Center, January 1991. http://dx.doi.org/10.21236/ada454846.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
7

Hays, Robert T. Theoretical Foundation for Advanced Distributed Learning Research. Fort Belvoir, VA: Defense Technical Information Center, May 2001. http://dx.doi.org/10.21236/ada385457.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
8

Chen, J. S. J. Distributed-query optimization in fragmented data-base systems. Office of Scientific and Technical Information (OSTI), August 1987. http://dx.doi.org/10.2172/7183881.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
9

Nocedal, Jorge. Nonlinear Optimization Methods for Large-Scale Learning. Office of Scientific and Technical Information (OSTI), October 2019. http://dx.doi.org/10.2172/1571768.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
10

Lumsdaine, Andrew. Scalable Second Order Optimization for Machine Learning. Office of Scientific and Technical Information (OSTI), May 2022. http://dx.doi.org/10.2172/1984057.

Повний текст джерела
Стилі APA, Harvard, Vancouver, ISO та ін.
Також вас можуть зацікавити розширені добірки на тему "Distributed optimization and learning" для конкретних типів джерел:
Статті в журналах Дисертації Книги
Ми пропонуємо знижки на всі преміум-плани для авторів, чиї праці увійшли до тематичних добірок літератури. Зв'яжіться з нами, щоб отримати унікальний промокод!

До бібліографії