Log in

Relevant bibliographies by topics / Off-Policy learning / Journal articles

To see the other types of publications on this topic, follow the link: Off-Policy learning.

Journal articles on the topic 'Off-Policy learning'

Author: Grafiati

Published: 1 June 2024

Last updated: 31 July 2025

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles

Select a source type:

Consult the top 50 journal articles for your research on the topic 'Off-Policy learning.'

Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.

You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.

Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.

1

Meng, Wenjia, Qian Zheng, Gang Pan, and Yilong Yin. "Off-Policy Proximal Policy Optimization." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 8 (2023): 9162–70. http://dx.doi.org/10.1609/aaai.v37i8.26099.

Full text

Abstract:

Proximal Policy Optimization (PPO) is an important reinforcement learning method, which has achieved great success in sequential decision-making problems. However, PPO faces the issue of sample inefficiency, which is due to the PPO cannot make use of off-policy data. In this paper, we propose an Off-Policy Proximal Policy Optimization method (Off-Policy PPO) that improves the sample efficiency of PPO by utilizing off-policy data. Specifically, we first propose a clipped surrogate objective function that can utilize off-policy data and avoid excessively large policy updates. Next, we theoretica

APA, Harvard, Vancouver, ISO, and other styles

2

Schmitt, Simon, John Shawe-Taylor, and Hado van Hasselt. "Chaining Value Functions for Off-Policy Learning." Proceedings of the AAAI Conference on Artificial Intelligence 36, no. 8 (2022): 8187–95. http://dx.doi.org/10.1609/aaai.v36i8.20792.

Full text

Abstract:

To accumulate knowledge and improve its policy of behaviour, a reinforcement learning agent can learn `off-policy' about policies that differ from the policy used to generate its experience. This is important to learn counterfactuals, or because the experience was generated out of its own control. However, off-policy learning is non-trivial, and standard reinforcement-learning algorithms can be unstable and divergent. In this paper we discuss a novel family of off-policy prediction algorithms which are convergent by construction. The idea is to first learn on-policy about the data-generating b

APA, Harvard, Vancouver, ISO, and other styles

3

Yang, Shangdong, Dingyuanhao Sun, and Xingguo Chen. "Off-Policy Temporal Difference Learning with Bellman Residuals." Mathematics 12, no. 22 (2024): 3603. http://dx.doi.org/10.3390/math12223603.

Full text

Abstract:

In reinforcement learning, off-policy temporal difference learning methods have gained significant attention due to their flexibility in utilizing existing data. However, traditional off-policy temporal difference methods often suffer from poor convergence and stability when handling complex problems. To address these issues, this paper proposes an off-policy temporal difference algorithm with Bellman residuals (TDBR). By incorporating Bellman residuals, the proposed algorithm effectively improves the convergence and stability of the off-policy learning process. This paper first introduces the

APA, Harvard, Vancouver, ISO, and other styles

4

Cief, Matej, Branislav Kveton, and Michal Kompan. "Cross-Validated Off-Policy Evaluation." Proceedings of the AAAI Conference on Artificial Intelligence 39, no. 15 (2025): 16073–81. https://doi.org/10.1609/aaai.v39i15.33765.

Full text

Abstract:

We study estimator selection and hyper-parameter tuning in off-policy evaluation. Although cross-validation is the most popular method for model selection in supervised learning, off-policy evaluation relies mostly on theory, which provides only limited guidance to practitioners. We show how to use cross-validation for off-policy evaluation. This challenges a popular belief that cross-validation in off-policy evaluation is not feasible. We evaluate our method empirically and show that it addresses a variety of use cases.

APA, Harvard, Vancouver, ISO, and other styles

5

Xu, Da, Yuting Ye, Chuanwei Ruan, and Bo Yang. "Towards Robust Off-Policy Learning for Runtime Uncertainty." Proceedings of the AAAI Conference on Artificial Intelligence 36, no. 9 (2022): 10101–9. http://dx.doi.org/10.1609/aaai.v36i9.21249.

Full text

Abstract:

Off-policy learning plays a pivotal role in optimizing and evaluating policies prior to the online deployment. However, during the real-time serving, we observe varieties of interventions and constraints that cause inconsistency between the online and offline setting, which we summarize and term as runtime uncertainty. Such uncertainty cannot be learned from the logged data due to its abnormality and rareness nature. To assert a certain level of robustness, we perturb the off-policy estimators along an adversarial direction in view of the runtime uncertainty. It allows the resulting estimators

APA, Harvard, Vancouver, ISO, and other styles

6

Peters, James F., and Christopher Henry. "Approximation spaces in off-policy Monte Carlo learning." Engineering Applications of Artificial Intelligence 20, no. 5 (2007): 667–75. http://dx.doi.org/10.1016/j.engappai.2006.11.005.

Full text

APA, Harvard, Vancouver, ISO, and other styles

7

Yu, Jiayu, Jingyao Li, Shuai Lü, and Shuai Han. "Mixed experience sampling for off-policy reinforcement learning." Expert Systems with Applications 251 (October 2024): 124017. http://dx.doi.org/10.1016/j.eswa.2024.124017.

Full text

APA, Harvard, Vancouver, ISO, and other styles

8

Cetin, Edoardo, and Oya Celiktutan. "Learning Pessimism for Reinforcement Learning." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 6 (2023): 6971–79. http://dx.doi.org/10.1609/aaai.v37i6.25852.

Full text

Abstract:

Off-policy deep reinforcement learning algorithms commonly compensate for overestimation bias during temporal-difference learning by utilizing pessimistic estimates of the expected target returns. In this work, we propose Generalized Pessimism Learning (GPL), a strategy employing a novel learnable penalty to enact such pessimism. In particular, we propose to learn this penalty alongside the critic with dual TD-learning, a new procedure to estimate and minimize the magnitude of the target returns bias with trivial computational cost. GPL enables us to accurately counteract overestimation bias t

APA, Harvard, Vancouver, ISO, and other styles

9

Sun, Mingfei, Sam Devlin, Katja Hofmann, and Shimon Whiteson. "Deterministic and Discriminative Imitation (D2-Imitation): Revisiting Adversarial Imitation for Sample Efficiency." Proceedings of the AAAI Conference on Artificial Intelligence 36, no. 8 (2022): 8378–85. http://dx.doi.org/10.1609/aaai.v36i8.20813.

Full text

Abstract:

Sample efficiency is crucial for imitation learning methods to be applicable in real-world applications. Many studies improve sample efficiency by extending adversarial imitation to be off-policy regardless of the fact that these off-policy extensions could either change the original objective or involve complicated optimization. We revisit the foundation of adversarial imitation and propose an off-policy sample efficient approach that requires no adversarial training or min-max optimization. Our formulation capitalizes on two key insights: (1) the similarity between the Bellman equation and t

APA, Harvard, Vancouver, ISO, and other styles

10

Yang, Shangdong, Shuaiqiang Zhang, and Xingguo Chen. "Online Attentive Kernel-Based Off-Policy Temporal Difference Learning." Applied Sciences 14, no. 23 (2024): 11114. http://dx.doi.org/10.3390/app142311114.

Full text

Abstract:

Temporal difference (TD) learning is a powerful framework for value function approximation in reinforcement learning. However, standard TD methods often struggle with feature representation and off-policy learning challenges. In this paper, we propose a novel framework, online attentive kernel-based off-policy TD learning, and in combination with well-known algorithms, introduce OAKGTD2, OAKTDC, and OAKETD. This framework uses two-timescale optimization. In the slow-timescale, a sparse representation of state features is learned using an online attentive kernel-based method. In the fast-timesc

APA, Harvard, Vancouver, ISO, and other styles

11

Chen, Xingguo, Wangrong Qin, Yu Gong, Shangdong Yang, and Wenhao Wang. "On Convergence Rate of MRetrace." Mathematics 12, no. 18 (2024): 2930. http://dx.doi.org/10.3390/math12182930.

Full text

Abstract:

Off-policy is a key setting for reinforcement learning algorithms. In recent years, the stability of off-policy learning for value-based reinforcement learning has been guaranteed even when combined with linear function approximation and bootstrapping. Convergence rate analysis is currently a hot topic. However, the convergence rates of learning algorithms vary, and analyzing the reasons behind this remains an open problem. In this paper, we propose an essentially simplified version of a convergence rate to generate general off-policy temporal difference learning algorithms. We emphasize that

APA, Harvard, Vancouver, ISO, and other styles

12

Kong, Seung-Hyun, I. Made Aswin Nahrendra, and Dong-Hee Paek. "Enhanced Off-Policy Reinforcement Learning With Focused Experience Replay." IEEE Access 9 (2021): 93152–64. http://dx.doi.org/10.1109/access.2021.3085142.

Full text

APA, Harvard, Vancouver, ISO, and other styles

13

Li, Lihong. "A perspective on off-policy evaluation in reinforcement learning." Frontiers of Computer Science 13, no. 5 (2019): 911–12. http://dx.doi.org/10.1007/s11704-019-9901-7.

Full text

APA, Harvard, Vancouver, ISO, and other styles

14

Luo, Biao, Huai-Ning Wu, and Tingwen Huang. "Off-Policy Reinforcement Learning for $ H_\infty $ Control Design." IEEE Transactions on Cybernetics 45, no. 1 (2015): 65–76. http://dx.doi.org/10.1109/tcyb.2014.2319577.

Full text

APA, Harvard, Vancouver, ISO, and other styles

15

Hao, Longyan, Chaoli Wang, and Yibo Shi. "Quadratic Tracking Control of Linear Stochastic Systems with Unknown Dynamics Using Average Off-Policy Q-Learning Method." Mathematics 12, no. 10 (2024): 1533. http://dx.doi.org/10.3390/math12101533.

Full text

Abstract:

This article investigates the optimal tracking control problem for data-based stochastic discrete-time linear systems. An average off-policy Q-learning algorithm is proposed to solve the optimal control problem with random disturbances. Compared with the existing off-policy reinforcement learning (RL) algorithm, the proposed average off-policy Q-learning algorithm avoids the assumption of an initial stability control. First, a pole placement strategy is used to design an initial stable control for systems with unknown dynamics. Second, the initial stable control is used to design a data-based

APA, Harvard, Vancouver, ISO, and other styles

16

Jain, Arushi, Gandharv Patil, Ayush Jain, Khimya Khetarpal, and Doina Precup. "Variance Penalized On-Policy and Off-Policy Actor-Critic." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 9 (2021): 7899–907. http://dx.doi.org/10.1609/aaai.v35i9.16964.

Full text

Abstract:

Reinforcement learning algorithms are typically geared towards optimizing the expected return of an agent. However, in many practical applications, low variance in the return is desired to ensure the reliability of an algorithm. In this paper, we propose on-policy and off-policy actor-critic algorithms that optimize a performance criterion involving both mean and variance in the return. Previous work uses the second moment of return to estimate the variance indirectly. Instead, we use a much simpler recently proposed direct variance estimator which updates the estimates incrementally using tem

APA, Harvard, Vancouver, ISO, and other styles

17

Yang, Yana, Meng Xi, Huiao Dai, Jiabao Wen, and Jiachen Yang. "Z-Score Experience Replay in Off-Policy Deep Reinforcement Learning." Sensors 24, no. 23 (2024): 7746. https://doi.org/10.3390/s24237746.

Full text

Abstract:

Reinforcement learning, as a machine learning method that does not require pre-training data, seeks the optimal policy through the continuous interaction between an agent and its environment. It is an important approach to solving sequential decision-making problems. By combining it with deep learning, deep reinforcement learning possesses powerful perception and decision-making capabilities and has been widely applied to various domains to tackle complex decision problems. Off-policy reinforcement learning separates exploration and exploitation by storing and replaying interaction experiences

APA, Harvard, Vancouver, ISO, and other styles

18

Zhang, Hengrui, Youfang Lin, Shuo Shen, Sheng Han, and Kai Lv. "Enhancing Off-Policy Constrained Reinforcement Learning through Adaptive Ensemble C Estimation." Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 19 (2024): 21770–78. http://dx.doi.org/10.1609/aaai.v38i19.30177.

Full text

Abstract:

In the domain of real-world agents, the application of Reinforcement Learning (RL) remains challenging due to the necessity for safety constraints. Previously, Constrained Reinforcement Learning (CRL) has predominantly focused on on-policy algorithms. Although these algorithms exhibit a degree of efficacy, their interactivity efficiency in real-world settings is sub-optimal, highlighting the demand for more efficient off-policy methods. However, off-policy CRL algorithms grapple with challenges in precise estimation of the C-function, particularly due to the fluctuations in the constrained Lag

APA, Harvard, Vancouver, ISO, and other styles

19

Gelada, Carles, and Marc G. Bellemare. "Off-Policy Deep Reinforcement Learning by Bootstrapping the Covariate Shift." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 3647–55. http://dx.doi.org/10.1609/aaai.v33i01.33013647.

Full text

Abstract:

In this paper we revisit the method of off-policy corrections for reinforcement learning (COP-TD) pioneered by Hallak et al. (2017). Under this method, online updates to the value function are reweighted to avoid divergence issues typical of off-policy learning. While Hallak et al.’s solution is appealing, it cannot easily be transferred to nonlinear function approximation. First, it requires a projection step onto the probability simplex; second, even though the operator describing the expected behavior of the off-policy learning algorithm is convergent, it is not known to be a contraction ma

APA, Harvard, Vancouver, ISO, and other styles

20

Zhang, Shangtong, Bo Liu, and Shimon Whiteson. "Mean-Variance Policy Iteration for Risk-Averse Reinforcement Learning." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 12 (2021): 10905–13. http://dx.doi.org/10.1609/aaai.v35i12.17302.

Full text

Abstract:

We present a mean-variance policy iteration (MVPI) framework for risk-averse control in a discounted infinite horizon MDP optimizing the variance of a per-step reward random variable. MVPI enjoys great flexibility in that any policy evaluation method and risk-neutral control method can be dropped in for risk-averse control off the shelf, in both on- and off-policy settings. This flexibility reduces the gap between risk-neutral control and risk-averse control and is achieved by working on a novel augmented MDP directly. We propose risk-averse TD3 as an example instantiating MVPI, which outperfo

APA, Harvard, Vancouver, ISO, and other styles

21

Tennenholtz, Guy, Uri Shalit, and Shie Mannor. "Off-Policy Evaluation in Partially Observable Environments." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 06 (2020): 10276–83. http://dx.doi.org/10.1609/aaai.v34i06.6590.

Full text

Abstract:

This work studies the problem of batch off-policy evaluation for Reinforcement Learning in partially observable environments. Off-policy evaluation under partial observability is inherently prone to bias, with risk of arbitrarily large errors. We define the problem of off-policy evaluation for Partially Observable Markov Decision Processes (POMDPs) and establish what we believe is the first off-policy evaluation result for POMDPs. In addition, we formulate a model in which observed and unobserved variables are decoupled into two dynamic processes, called a Decoupled POMDP. We show how off-poli

APA, Harvard, Vancouver, ISO, and other styles

22

Xiao, Teng, and Suhang Wang. "Towards Off-Policy Learning for Ranking Policies with Logged Feedback." Proceedings of the AAAI Conference on Artificial Intelligence 36, no. 8 (2022): 8700–8707. http://dx.doi.org/10.1609/aaai.v36i8.20849.

Full text

Abstract:

Probabilistic learning to rank (LTR) has been the dominating approach for optimizing the ranking metric, but cannot maximize long-term rewards. Reinforcement learning models have been proposed to maximize user long-term rewards by formulating the recommendation as a sequential decision-making problem, but could only achieve inferior accuracy compared to LTR counterparts, primarily due to the lack of online interactions and the characteristics of ranking. In this paper, we propose a new off-policy value ranking (VR) algorithm that can simultaneously maximize user long-term rewards and optimize

APA, Harvard, Vancouver, ISO, and other styles

23

Li, Jinna, Hamidreza Modares, Tianyou Chai, Frank L. Lewis, and Lihua Xie. "Off-Policy Reinforcement Learning for Synchronization in Multiagent Graphical Games." IEEE Transactions on Neural Networks and Learning Systems 28, no. 10 (2017): 2434–45. http://dx.doi.org/10.1109/tnnls.2016.2609500.

Full text

APA, Harvard, Vancouver, ISO, and other styles

24

Ali, Raja Farrukh, Kevin Duong, Nasik Muhammad Nafi, and William Hsu. "Multi-Horizon Learning in Procedurally-Generated Environments for Off-Policy Reinforcement Learning (Student Abstract)." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 13 (2023): 16150–51. http://dx.doi.org/10.1609/aaai.v37i13.26935.

Full text

Abstract:

Value estimates at multiple timescales can help create advanced discounting functions and allow agents to form more effective predictive models of their environment. In this work, we investigate learning over multiple horizons concurrently for off-policy reinforcement learning by using an advantage-based action selection method and introducing architectural improvements. Our proposed agent learns over multiple horizons simultaneously, while using either exponential or hyperbolic discounting functions. We implement our approach on Rainbow, a value-based off-policy algorithm, and test on Procgen

APA, Harvard, Vancouver, ISO, and other styles

25

Nakamura, Yutaka, Takeshi Mori, Yoichi Tokita, Tomohiro Shibata, and Shin Ishii. "Off-Policy Natural Policy Gradient Method for a Biped Walking Using a CPG Controller." Journal of Robotics and Mechatronics 17, no. 6 (2005): 636–44. http://dx.doi.org/10.20965/jrm.2005.p0636.

Full text

Abstract:

Referring to the mechanism of animals’ rhythmic movements, motor control schemes using a central pattern generator (CPG) controller have been studied. We previously proposed reinforcement learning (RL) called the CPG-actor-critic model, as an autonomous learning framework for a CPG controller. Here, we propose an off-policy natural policy gradient RL algorithm for the CPG-actor-critic model, to solve the “exploration-exploitation” problem by meta-controlling “behavior policy.” We apply this RL algorithm to an automatic control problem using a biped robot simulator. Computer simulation demonstr

APA, Harvard, Vancouver, ISO, and other styles

26

Wang, Mingyang, Zhenshan Bing, Xiangtong Yao, et al. "Meta-Reinforcement Learning Based on Self-Supervised Task Representation Learning." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 8 (2023): 10157–65. http://dx.doi.org/10.1609/aaai.v37i8.26210.

Full text

Abstract:

Meta-reinforcement learning enables artificial agents to learn from related training tasks and adapt to new tasks efficiently with minimal interaction data. However, most existing research is still limited to narrow task distributions that are parametric and stationary, and does not consider out-of-distribution tasks during the evaluation, thus, restricting its application. In this paper, we propose MoSS, a context-based Meta-reinforcement learning algorithm based on Self-Supervised task representation learning to address this challenge. We extend meta-RL to broad non-parametric task distribut

APA, Harvard, Vancouver, ISO, and other styles

27

Nguyen, Ba Quoc Anh, Ngoc Trung Dang, Thanh Tung Le, and Phuong Nam Dao. "On-policy and Off-policy Q-learning algorithms with policy iteration for two-wheeled inverted pendulum systems." Robotics and Autonomous Systems 193 (November 2025): 105111. https://doi.org/10.1016/j.robot.2025.105111.

Full text

APA, Harvard, Vancouver, ISO, and other styles

28

Cao, Jiaqing, Quan Liu, Fei Zhu, Qiming Fu, and Shan Zhong. "Gradient temporal-difference learning for off-policy evaluation using emphatic weightings." Information Sciences 580 (November 2021): 311–30. http://dx.doi.org/10.1016/j.ins.2021.08.082.

Full text

APA, Harvard, Vancouver, ISO, and other styles

29

Tian, Chang, An Liu, Guan Huang, and Wu Luo. "Successive Convex Approximation Based Off-Policy Optimization for Constrained Reinforcement Learning." IEEE Transactions on Signal Processing 70 (2022): 1609–24. http://dx.doi.org/10.1109/tsp.2022.3158737.

Full text

APA, Harvard, Vancouver, ISO, and other styles

30

Karimpanal, Thommen George, and Erik Wilhelm. "Identification and off-policy learning of multiple objectives using adaptive clustering." Neurocomputing 263 (November 2017): 39–47. http://dx.doi.org/10.1016/j.neucom.2017.04.074.

Full text

APA, Harvard, Vancouver, ISO, and other styles

31

Kiumarsi, Bahare, Frank L. Lewis, and Zhong-Ping Jiang. "H∞ control of linear discrete-time systems: Off-policy reinforcement learning." Automatica 78 (April 2017): 144–52. http://dx.doi.org/10.1016/j.automatica.2016.12.009.

Full text

APA, Harvard, Vancouver, ISO, and other styles

32

Li, Jinna, Zhenfei Xiao, and Ping Li. "Discrete-Time Multi-Player Games Based on Off-Policy Q-Learning." IEEE Access 7 (2019): 134647–59. http://dx.doi.org/10.1109/access.2019.2939384.

Full text

APA, Harvard, Vancouver, ISO, and other styles

33

Kiumarsi, Bahare, Wei Kang, and Frank L. Lewis. "H∞ Control of Nonaffine Aerial Systems Using Off-policy Reinforcement Learning." Unmanned Systems 04, no. 01 (2016): 51–60. http://dx.doi.org/10.1142/s2301385016400069.

Full text

Abstract:

This paper presents a completely model-free [Formula: see text] optimal tracking solution to the control of a general class of nonlinear nonaffine systems in the presence of the input constraints. The proposed method is motivated by nonaffine unmanned aerial vehicle (UAV) system as a real application. First, a general class of nonlinear nonaffine system dynamics is presented as an affine system in terms of a nonlinear function of the control input. It is shown that the optimal control of nonaffine systems may not have an admissible solution if the utility function is not defined properly. More

APA, Harvard, Vancouver, ISO, and other styles

34

Lian, Bosen, Wenqian Xue, Yijing Xie, Frank L. Lewis, and Ali Davoudi. "Off-policy inverse Q-learning for discrete-time antagonistic unknown systems." Automatica 155 (September 2023): 111171. http://dx.doi.org/10.1016/j.automatica.2023.111171.

Full text

APA, Harvard, Vancouver, ISO, and other styles

35

Zhang, Ruiyi, Tong Yu, Yilin Shen, and Hongxia Jin. "Text-Based Interactive Recommendation via Offline Reinforcement Learning." Proceedings of the AAAI Conference on Artificial Intelligence 36, no. 10 (2022): 11694–702. http://dx.doi.org/10.1609/aaai.v36i10.21424.

Full text

Abstract:

Interactive recommendation with natural-language feedback can provide richer user feedback and has demonstrated advantages over traditional recommender systems. However, the classical online paradigm involves iteratively collecting experience via interaction with users, which is expensive and risky. We consider an offline interactive recommendation to exploit arbitrary experience collected by multiple unknown policies. A direct application of policy learning with such fixed experience suffers from the distribution shift. To tackle this issue, we develop a behavior-agnostic off-policy correctio

APA, Harvard, Vancouver, ISO, and other styles

36

Xu, Z., L. Cao, and X. Chen. "Deep Reinforcement Learning with Adaptive Update Target Combination." Computer Journal 63, no. 7 (2019): 995–1003. http://dx.doi.org/10.1093/comjnl/bxz066.

Full text

Abstract:

Abstract Simple and efficient exploration remains a core challenge in deep reinforcement learning. While many exploration methods can be applied to high-dimensional tasks, these methods manually adjust exploration parameters according to domain knowledge. This paper proposes a novel method that can automatically balance exploration and exploitation, as well as combine on-policy and off-policy update targets through a dynamic weighted way based on value difference. The proposed method does not directly affect the probability of a selected action but utilizes the value difference produced during

APA, Harvard, Vancouver, ISO, and other styles

37

Chaudhari, Shreyas, David Arbour, Georgios Theocharous, and Nikos Vlassis. "Distributional Off-Policy Evaluation for Slate Recommendations." Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 8 (2024): 8265–73. http://dx.doi.org/10.1609/aaai.v38i8.28667.

Full text

Abstract:

Recommendation strategies are typically evaluated by using previously logged data, employing off-policy evaluation methods to estimate their expected performance. However, for strategies that present users with slates of multiple items, the resulting combinatorial action space renders many of these methods impractical. Prior work has developed estimators that leverage the structure in slates to estimate the expected off-policy performance, but the estimation of the entire performance distribution remains elusive. Estimating the complete distribution allows for a more comprehensive evaluation o

APA, Harvard, Vancouver, ISO, and other styles

38

Kim, Man-Je, Hyunsoo Park, and Chang Wook Ahn. "Nondominated Policy-Guided Learning in Multi-Objective Reinforcement Learning." Electronics 11, no. 7 (2022): 1069. http://dx.doi.org/10.3390/electronics11071069.

Full text

Abstract:

Control intelligence is a typical field where there is a trade-off between target objectives, and researchers in this field have longed for artificial intelligence that achieves the target objectives. Multi-objective deep reinforcement learning was sufficient to satisfy this need. In particular, multi-objective deep reinforcement learning methods based on policy optimization are leading the optimization of control intelligence. However, multi-objective reinforcement learning has difficulties when finding various Pareto optimals of multi-objectives due to the greedy nature of reinforcement lear

APA, Harvard, Vancouver, ISO, and other styles

39

Hollenstein, Jakob, Georg Martius, and Justus Piater. "Colored Noise in PPO: Improved Exploration and Performance through Correlated Action Sampling." Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 11 (2024): 12466–72. http://dx.doi.org/10.1609/aaai.v38i11.29139.

Full text

Abstract:

Proximal Policy Optimization (PPO), a popular on-policy deep reinforcement learning method, employs a stochastic policy for exploration. In this paper, we propose a colored noise-based stochastic policy variant of PPO. Previous research highlighted the importance of temporal correlation in action noise for effective exploration in off-policy reinforcement learning. Building on this, we investigate whether correlated noise can also enhance exploration in on-policy methods like PPO. We discovered that correlated noise for action selection improves learning performance and outperforms the current

APA, Harvard, Vancouver, ISO, and other styles

40

Ren, He, Jing Dai, Huaguang Zhang, and Kun Zhang. "Off-policy integral reinforcement learning algorithm in dealing with nonzero sum game for nonlinear distributed parameter systems." Transactions of the Institute of Measurement and Control 42, no. 15 (2020): 2919–28. http://dx.doi.org/10.1177/0142331220932634.

Full text

Abstract:

Benefitting from the technology of integral reinforcement learning, the nonzero sum (NZS) game for distributed parameter systems is effectively solved in this paper when the information of system dynamics are unavailable. The Karhunen-Loève decomposition (KLD) is employed to convert the partial differential equation (PDE) systems into high-order ordinary differential equation (ODE) systems. Moreover, the off-policy IRL technology is introduced to design the optimal strategies for the NZS game. To confirm that the presented algorithm will converge to the optimal value functions, the traditional

APA, Harvard, Vancouver, ISO, and other styles

41

Yang, Hyunjun, Hyeonjun Park, and Kyungjae Lee. "A Selective Portfolio Management Algorithm with Off-Policy Reinforcement Learning Using Dirichlet Distribution." Axioms 11, no. 12 (2022): 664. http://dx.doi.org/10.3390/axioms11120664.

Full text

Abstract:

Existing methods in portfolio management deterministically produce an optimal portfolio. However, according to modern portfolio theory, there exists a trade-off between a portfolio’s expected returns and risks. Therefore, the optimal portfolio does not exist definitively, but several exist, and using only one deterministic portfolio is disadvantageous for risk management. We proposed Dirichlet Distribution Trader (DDT), an algorithm that calculates multiple optimal portfolios by taking Dirichlet Distribution as a policy. The DDT algorithm makes several optimal portfolios according to risk leve

APA, Harvard, Vancouver, ISO, and other styles

42

Shahid, Asad Ali, Dario Piga, Francesco Braghin, and Loris Roveda. "Continuous control actions learning and adaptation for robotic manipulation through reinforcement learning." Autonomous Robots 46, no. 3 (2022): 483–98. http://dx.doi.org/10.1007/s10514-022-10034-z.

Full text

Abstract:

AbstractThis paper presents a learning-based method that uses simulation data to learn an object manipulation task using two model-free reinforcement learning (RL) algorithms. The learning performance is compared across on-policy and off-policy algorithms: Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC). In order to accelerate the learning process, the fine-tuning procedure is proposed that demonstrates the continuous adaptation of on-policy RL to new environments, allowing the learned policy to adapt and execute the (partially) modified task. A dense reward function is designed

APA, Harvard, Vancouver, ISO, and other styles

43

Levine, Alexander, and Soheil Feizi. "Goal-Conditioned Q-learning as Knowledge Distillation." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 7 (2023): 8500–8509. http://dx.doi.org/10.1609/aaai.v37i7.26024.

Full text

Abstract:

Many applications of reinforcement learning can be formalized as goal-conditioned environments, where, in each episode, there is a "goal" that affects the rewards obtained during that episode but does not affect the dynamics. Various techniques have been proposed to improve performance in goal-conditioned environments, such as automatic curriculum generation and goal relabeling. In this work, we explore a connection between off-policy reinforcement learning in goal-conditioned settings and knowledge distillation. In particular: the current Q-value function and the target Q-value estimate are b

APA, Harvard, Vancouver, ISO, and other styles

44

Kim та Park. "Exploration with Multiple Random ε-Buffers in Off-Policy Deep Reinforcement Learning". Symmetry 11, № 11 (2019): 1352. http://dx.doi.org/10.3390/sym11111352.

Full text

Abstract:

In terms of deep reinforcement learning (RL), exploration is highly significant in achieving better generalization. In benchmark studies, ε-greedy random actions have been used to encourage exploration and prevent over-fitting, thereby improving generalization. Deep RL with random ε-greedy policies, such as deep Q-networks (DQNs), can demonstrate efficient exploration behavior. A random ε-greedy policy exploits additional replay buffers in an environment of sparse and binary rewards, such as in the real-time online detection of network securities by verifying whether the network is “normal or

APA, Harvard, Vancouver, ISO, and other styles

45

Liu, Mushuang, Yan Wan, Frank L. Lewis, and Victor G. Lopez. "Adaptive Optimal Control for Stochastic Multiplayer Differential Games Using On-Policy and Off-Policy Reinforcement Learning." IEEE Transactions on Neural Networks and Learning Systems 31, no. 12 (2020): 5522–33. http://dx.doi.org/10.1109/tnnls.2020.2969215.

Full text

APA, Harvard, Vancouver, ISO, and other styles

46

Suttle, Wesley, Zhuoran Yang, Kaiqing Zhang, Zhaoran Wang, Tamer Başar, and Ji Liu. "A Multi-Agent Off-Policy Actor-Critic Algorithm for Distributed Reinforcement Learning." IFAC-PapersOnLine 53, no. 2 (2020): 1549–54. http://dx.doi.org/10.1016/j.ifacol.2020.12.2021.

Full text

APA, Harvard, Vancouver, ISO, and other styles

47

Stanković, Miloš S., Marko Beko, and Srdjan S. Stanković. "Distributed Gradient Temporal Difference Off-policy Learning With Eligibility Traces: Weak Convergence." IFAC-PapersOnLine 53, no. 2 (2020): 1563–68. http://dx.doi.org/10.1016/j.ifacol.2020.12.2184.

Full text

APA, Harvard, Vancouver, ISO, and other styles

48

Li, Jinna, Zhenfei Xiao, Tianyou Chai, Frank L. Lewis, and Sarangapani Jagannathan. "Off-Policy Q-Learning for Anti-Interference Control of Multi-Player Systems." IFAC-PapersOnLine 53, no. 2 (2020): 9189–94. http://dx.doi.org/10.1016/j.ifacol.2020.12.2180.

Full text

APA, Harvard, Vancouver, ISO, and other styles

49

Chen, Ning, Shuhan Luo, Jiayang Dai, Biao Luo, and Weihua Gui. "Optimal Control of Iron-Removal Systems Based on Off-Policy Reinforcement Learning." IEEE Access 8 (2020): 149730–40. http://dx.doi.org/10.1109/access.2020.3015801.

Full text

APA, Harvard, Vancouver, ISO, and other styles

50

Hachiya, Hirotaka, Takayuki Akiyama, Masashi Sugiayma, and Jan Peters. "Adaptive importance sampling for value function approximation in off-policy reinforcement learning." Neural Networks 22, no. 10 (2009): 1399–410. http://dx.doi.org/10.1016/j.neunet.2009.01.002.

Full text

APA, Harvard, Vancouver, ISO, and other styles

We offer discounts on all premium plans for authors whose works are included in thematic literature selections. Contact us to get a unique promo code!