Log in

Relevant bibliographies by topics / Structured continuous time Markov decision processes / Journal articles

To see the other types of publications on this topic, follow the link: Structured continuous time Markov decision processes.

Journal articles on the topic 'Structured continuous time Markov decision processes'

Author: Grafiati

Published: 9 March 2023

Last updated: 31 July 2025

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles

Select a source type:

Consult the top 50 journal articles for your research on the topic 'Structured continuous time Markov decision processes.'

Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.

You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.

Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.

1

Shelton, C. R., and G. Ciardo. "Tutorial on Structured Continuous-Time Markov Processes." Journal of Artificial Intelligence Research 51 (December 23, 2014): 725–78. http://dx.doi.org/10.1613/jair.4415.

Full text

Abstract:

A continuous-time Markov process (CTMP) is a collection of variables indexed by a continuous quantity, time. It obeys the Markov property that the distribution over a future variable is independent of past variables given the state at the present time. We introduce continuous-time Markov process representations and algorithms for filtering, smoothing, expected sufficient statistics calculations, and model estimation, assuming no prior knowledge of continuous-time processes but some basic knowledge of probability and statistics. We begin by describing "flat" or unstructured Markov processes and

APA, Harvard, Vancouver, ISO, and other styles

2

D'Amico, Guglielmo, Jacques Janssen, and Raimondo Manca. "Monounireducible Nonhomogeneous Continuous Time Semi-Markov Processes Applied to Rating Migration Models." Advances in Decision Sciences 2012 (October 16, 2012): 1–12. http://dx.doi.org/10.1155/2012/123635.

Full text

Abstract:

Monounireducible nonhomogeneous semi- Markov processes are defined and investigated. The mono- unireducible topological structure is a sufficient condition that guarantees the absorption of the semi-Markov process in a state of the process. This situation is of fundamental importance in the modelling of credit rating migrations because permits the derivation of the distribution function of the time of default. An application in credit rating modelling is given in order to illustrate the results.

APA, Harvard, Vancouver, ISO, and other styles

3

Beutler, Frederick J., and Keith W. Ross. "Uniformization for semi-Markov decision processes under stationary policies." Journal of Applied Probability 24, no. 3 (1987): 644–56. http://dx.doi.org/10.2307/3214096.

Full text

Abstract:

Uniformization permits the replacement of a semi-Markov decision process (SMDP) by a Markov chain exhibiting the same average rewards for simple (non-randomized) policies. It is shown that various anomalies may occur, especially for stationary (randomized) policies; uniformization introduces virtual jumps with concomitant action changes not present in the original process. Since these lead to discrepancies in the average rewards for stationary processes, uniformization can be accepted as valid only for simple policies.We generalize uniformization to yield consistent results for stationary poli

APA, Harvard, Vancouver, ISO, and other styles

4

Beutler, Frederick J., and Keith W. Ross. "Uniformization for semi-Markov decision processes under stationary policies." Journal of Applied Probability 24, no. 03 (1987): 644–56. http://dx.doi.org/10.1017/s0021900200031375.

Full text

Abstract:

Uniformization permits the replacement of a semi-Markov decision process (SMDP) by a Markov chain exhibiting the same average rewards for simple (non-randomized) policies. It is shown that various anomalies may occur, especially for stationary (randomized) policies; uniformization introduces virtual jumps with concomitant action changes not present in the original process. Since these lead to discrepancies in the average rewards for stationary processes, uniformization can be accepted as valid only for simple policies. We generalize uniformization to yield consistent results for stationary pol

APA, Harvard, Vancouver, ISO, and other styles

5

Dibangoye, Jilles Steeve, Christopher Amato, Olivier Buffet, and François Charpillet. "Optimally Solving Dec-POMDPs as Continuous-State MDPs." Journal of Artificial Intelligence Research 55 (February 24, 2016): 443–97. http://dx.doi.org/10.1613/jair.4623.

Full text

Abstract:

Decentralized partially observable Markov decision processes (Dec-POMDPs) provide a general model for decision-making under uncertainty in decentralized settings, but are difficult to solve optimally (NEXP-Complete). As a new way of solving these problems, we introduce the idea of transforming a Dec-POMDP into a continuous-state deterministic MDP with a piecewise-linear and convex value function. This approach makes use of the fact that planning can be accomplished in a centralized offline manner, while execution can still be decentralized. This new Dec-POMDP formulation, which we call an occu

APA, Harvard, Vancouver, ISO, and other styles

6

Chornei, Ruslan. "Local Control in Gordon-Newell Networks." NaUKMA Research Papers. Computer Science 7 (May 12, 2025): 120–29. https://doi.org/10.18523/2617-3808.2024.7.120-129.

Full text

Abstract:

We examine continuous-time stochastic processes with a general compact state space, which is organized by a fundamental graph defining a neighborhood structure of states. These neighborhoods establish local interactions among the coordinates of the spatial process. At any given moment, the random state of the system, as described by the stochastic process, forms a random field concerning the neighborhood graph.The process is assumed to have a semi-Markov temporal property, and its transition kernels exhibit a spatial Markov property relative to the basic graph. Additionally, a local control st

APA, Harvard, Vancouver, ISO, and other styles

7

Pazis, Jason, and Ronald Parr. "Sample Complexity and Performance Bounds for Non-Parametric Approximate Linear Programming." Proceedings of the AAAI Conference on Artificial Intelligence 27, no. 1 (2013): 782–88. http://dx.doi.org/10.1609/aaai.v27i1.8696.

Full text

Abstract:

One of the most difficult tasks in value function approximation for Markov Decision Processes is finding an approximation architecture that is expressive enough to capture the important structure in the value function, while at the same time not overfitting the training samples. Recent results in non-parametric approximate linear programming (NP-ALP), have demonstrated that this can be done effectively using nothing more than a smoothness assumption on the value function. In this paper we extend these results to the case where samples come from real world transitions instead of the full Bellma

APA, Harvard, Vancouver, ISO, and other styles

8

Abid, Amira, Fathi Abid, and Bilel Kaffel. "CDS-based implied probability of default estimation." Journal of Risk Finance 21, no. 4 (2020): 399–422. http://dx.doi.org/10.1108/jrf-05-2019-0079.

Full text

Abstract:

Purpose This study aims to shed more light on the relationship between probability of default, investment horizons and rating classes to make decision-making processes more efficient. Design/methodology/approach Based on credit default swaps (CDS) spreads, a methodology is implemented to determine the implied default probability and the implied rating, and then to estimate the term structure of the market-implied default probability and the transition matrix of implied rating. The term structure estimation in discrete time is conducted with the Nelson and Siegel model and in continuous time wi

APA, Harvard, Vancouver, ISO, and other styles

9

Mironov, Aleksey, Anna Mironova, and Vyacheslav Burlov. "Mathematical modeling of preemptive management by the stages complex of administrative production." Applied Mathematics and Control Sciences, no. 4 (December 12, 2022): 174–97. http://dx.doi.org/10.15593/2499-9873/2022.4.10.

Full text

Abstract:

In line with the fundamental reform of the Russia administrative legislation, this article discusses the synthesis of a geoinformation system for preventive management of the full cycle of production on affairs about administrative offenses. According to the Anokhin - Sudakov's theory of functional systems, the tasks of forming a structural image and synthesizing a mathematical model to manage the administrative process stages, as well as the tasks of substantiating the mathematical criterion and its structural and functional implementation to prevent violations of a reasonable time in product

APA, Harvard, Vancouver, ISO, and other styles

10

Puterman, Martin L., and F. A. Van der Duyn Schouten. "Markov Decision Processes With Continuous Time Parameter." Journal of the American Statistical Association 80, no. 390 (1985): 491. http://dx.doi.org/10.2307/2287942.

Full text

APA, Harvard, Vancouver, ISO, and other styles

11

Mahmoud, Marwan, and Sami Ben Slama. "Peer-to-Peer Energy Trading Case Study Using an AI-Powered Community Energy Management System." Applied Sciences 13, no. 13 (2023): 7838. http://dx.doi.org/10.3390/app13137838.

Full text

Abstract:

The Internet of Energy (IoE) is a topic that industry and academics find intriguing and promising, since it can aid in developing technology for smart cities. This study suggests an innovative energy system with peer-to-peer trading and more sophisticated residential energy storage system management. It proposes a smart residential community strategy that includes household customers and nearby energy storage installations. Without constructing new energy-producing facilities, users can consume affordable renewable energy by exchanging energy with the community energy pool. The community energ

APA, Harvard, Vancouver, ISO, and other styles

12

Fu, Yaqing. "Variance Optimization for Continuous-Time Markov Decision Processes." Open Journal of Statistics 09, no. 02 (2019): 181–95. http://dx.doi.org/10.4236/ojs.2019.92014.

Full text

APA, Harvard, Vancouver, ISO, and other styles

13

Guo, Xianping, and Yi Zhang. "Constrained total undiscounted continuous-time Markov decision processes." Bernoulli 23, no. 3 (2017): 1694–736. http://dx.doi.org/10.3150/15-bej793.

Full text

APA, Harvard, Vancouver, ISO, and other styles

14

Zhang, Yi. "Continuous-Time Markov Decision Processes with Exponential Utility." SIAM Journal on Control and Optimization 55, no. 4 (2017): 2636–60. http://dx.doi.org/10.1137/16m1086261.

Full text

APA, Harvard, Vancouver, ISO, and other styles

15

Dufour, François, and Alexei B. Piunovskiy. "Impulsive Control for Continuous-Time Markov Decision Processes." Advances in Applied Probability 47, no. 1 (2015): 106–27. http://dx.doi.org/10.1239/aap/1427814583.

Full text

Abstract:

In this paper our objective is to study continuous-time Markov decision processes on a general Borel state space with both impulsive and continuous controls for the infinite time horizon discounted cost. The continuous-time controlled process is shown to be nonexplosive under appropriate hypotheses. The so-called Bellman equation associated to this control problem is studied. Sufficient conditions ensuring the existence and the uniqueness of a bounded measurable solution to this optimality equation are provided. Moreover, it is shown that the value function of the optimization problem under co

APA, Harvard, Vancouver, ISO, and other styles

16

Dufour, François, and Alexei B. Piunovskiy. "Impulsive Control for Continuous-Time Markov Decision Processes." Advances in Applied Probability 47, no. 01 (2015): 106–27. http://dx.doi.org/10.1017/s0001867800007722.

Full text

Abstract:

In this paper our objective is to study continuous-time Markov decision processes on a general Borel state space with both impulsive and continuous controls for the infinite time horizon discounted cost. The continuous-time controlled process is shown to be nonexplosive under appropriate hypotheses. The so-called Bellman equation associated to this control problem is studied. Sufficient conditions ensuring the existence and the uniqueness of a bounded measurable solution to this optimality equation are provided. Moreover, it is shown that the value function of the optimization problem under co

APA, Harvard, Vancouver, ISO, and other styles

17

Piunovskiy, Alexey. "Realizable Strategies in Continuous-Time Markov Decision Processes." SIAM Journal on Control and Optimization 56, no. 1 (2018): 473–95. http://dx.doi.org/10.1137/17m1138959.

Full text

APA, Harvard, Vancouver, ISO, and other styles

18

Hu, Qiying. "Continuous time shock markov decision processes with discounted criterion." Optimization 25, no. 2-3 (1992): 271–83. http://dx.doi.org/10.1080/02331939208843824.

Full text

APA, Harvard, Vancouver, ISO, and other styles

19

Wei, Qingda. "Mean–semivariance optimality for continuous-time Markov decision processes." Systems & Control Letters 125 (March 2019): 67–74. http://dx.doi.org/10.1016/j.sysconle.2019.02.001.

Full text

APA, Harvard, Vancouver, ISO, and other styles

20

Guo, Xianping, XinYuan Song, and Junyu Zhang. "Bias optimality for multichain continuous-time Markov decision processes." Operations Research Letters 37, no. 5 (2009): 317–21. http://dx.doi.org/10.1016/j.orl.2009.04.005.

Full text

APA, Harvard, Vancouver, ISO, and other styles

21

Zhang, Lanlan, and Xianping Guo. "Constrained continuous-time Markov decision processes with average criteria." Mathematical Methods of Operations Research 67, no. 2 (2007): 323–40. http://dx.doi.org/10.1007/s00186-007-0154-0.

Full text

APA, Harvard, Vancouver, ISO, and other styles

22

Hu, Q. Y. "Nonstationary Continuous Time Markov Decision Processes with Discounted Criterion." Journal of Mathematical Analysis and Applications 180, no. 1 (1993): 60–70. http://dx.doi.org/10.1006/jmaa.1993.1382.

Full text

APA, Harvard, Vancouver, ISO, and other styles

23

Hu, Qiying. "Continuous Time Markov Decision Processes with Discounted Moment Criterion." Journal of Mathematical Analysis and Applications 203, no. 1 (1996): 1–12. http://dx.doi.org/10.1006/jmaa.1996.9999.

Full text

APA, Harvard, Vancouver, ISO, and other styles

24

Piunovskiy, Alexey, and Yi Zhang. "The Transformation Method for Continuous-Time Markov Decision Processes." Journal of Optimization Theory and Applications 154, no. 2 (2012): 691–712. http://dx.doi.org/10.1007/s10957-012-0015-8.

Full text

APA, Harvard, Vancouver, ISO, and other styles

25

Bartocci, Ezio, Luca Bortolussi, Tomáš Brázdil, Dimitrios Milios, and Guido Sanguinetti. "Policy learning in continuous-time Markov decision processes using Gaussian Processes." Performance Evaluation 116 (November 2017): 84–100. http://dx.doi.org/10.1016/j.peva.2017.08.007.

Full text

APA, Harvard, Vancouver, ISO, and other styles

26

Guo, Xianping. "Constrained Optimization for Average Cost Continuous-Time Markov Decision Processes." IEEE Transactions on Automatic Control 52, no. 6 (2007): 1139–43. http://dx.doi.org/10.1109/tac.2007.899040.

Full text

APA, Harvard, Vancouver, ISO, and other styles

27

Xianping Guo and Xinyuan Song. "Mean-Variance Criteria for Finite Continuous-Time Markov Decision Processes." IEEE Transactions on Automatic Control 54, no. 9 (2009): 2151–57. http://dx.doi.org/10.1109/tac.2009.2023833.

Full text

APA, Harvard, Vancouver, ISO, and other styles

28

Piunovskiy, Alexey. "Randomized and Relaxed Strategies in Continuous-Time Markov Decision Processes." SIAM Journal on Control and Optimization 53, no. 6 (2015): 3503–33. http://dx.doi.org/10.1137/15m1014012.

Full text

APA, Harvard, Vancouver, ISO, and other styles

29

Piunovskiy, A. B. "DISCOUNTED CONTINUOUS TIME MARKOV DECISION PROCESSES: THE CONVEX ANALYTIC APPROACH." IFAC Proceedings Volumes 38, no. 1 (2005): 31–36. http://dx.doi.org/10.3182/20050703-6-cz-1902.00357.

Full text

APA, Harvard, Vancouver, ISO, and other styles

30

Guo, Xianping, Mantas Vykertas, and Yi Zhang. "Absorbing Continuous-Time Markov Decision Processes with Total Cost Criteria." Advances in Applied Probability 45, no. 2 (2013): 490–519. http://dx.doi.org/10.1239/aap/1370870127.

Full text

Abstract:

In this paper we study absorbing continuous-time Markov decision processes in Polish state spaces with unbounded transition and cost rates, and history-dependent policies. The performance measure is the expected total undiscounted costs. For the unconstrained problem, we show the existence of a deterministic stationary optimal policy, whereas, for the constrained problems with N constraints, we show the existence of a mixed stationary optimal policy, where the mixture is over no more than N+1 deterministic stationary policies. Furthermore, the strong duality result is obtained for the associat

APA, Harvard, Vancouver, ISO, and other styles

31

Guo, Xianping, Mantas Vykertas, and Yi Zhang. "Absorbing Continuous-Time Markov Decision Processes with Total Cost Criteria." Advances in Applied Probability 45, no. 02 (2013): 490–519. http://dx.doi.org/10.1017/s0001867800006418.

Full text

Abstract:

In this paper we study absorbing continuous-time Markov decision processes in Polish state spaces with unbounded transition and cost rates, and history-dependent policies. The performance measure is the expected total undiscounted costs. For the unconstrained problem, we show the existence of a deterministic stationary optimal policy, whereas, for the constrained problems withNconstraints, we show the existence of a mixed stationary optimal policy, where the mixture is over no more thanN+1 deterministic stationary policies. Furthermore, the strong duality result is obtained for the associated

APA, Harvard, Vancouver, ISO, and other styles

32

Anselmi, Jonatha, François Dufour, and Tomás Prieto-Rumeau. "Computable approximations for average Markov decision processes in continuous time." Journal of Applied Probability 55, no. 2 (2018): 571–92. http://dx.doi.org/10.1017/jpr.2018.36.

Full text

Abstract:

Abstract In this paper we study the numerical approximation of the optimal long-run average cost of a continuous-time Markov decision process, with Borel state and action spaces, and with bounded transition and reward rates. Our approach uses a suitable discretization of the state and action spaces to approximate the original control model. The approximation error for the optimal average reward is then bounded by a linear combination of coefficients related to the discretization of the state and action spaces, namely, the Wasserstein distance between an underlying probability measure μ and a m

APA, Harvard, Vancouver, ISO, and other styles

33

Guo, Xianping, Yonghui Huang, and Yi Zhang. "Constrained Continuous-Time Markov Decision Processes on the Finite Horizon." Applied Mathematics & Optimization 75, no. 2 (2016): 317–41. http://dx.doi.org/10.1007/s00245-016-9352-6.

Full text

APA, Harvard, Vancouver, ISO, and other styles

34

Ye, Liuer, and Xianping Guo. "Continuous-Time Markov Decision Processes with State-Dependent Discount Factors." Acta Applicandae Mathematicae 121, no. 1 (2012): 5–27. http://dx.doi.org/10.1007/s10440-012-9669-3.

Full text

APA, Harvard, Vancouver, ISO, and other styles

35

Guo, Xianping, and Xinyuan Song. "Discounted continuous-time constrained Markov decision processes in Polish spaces." Annals of Applied Probability 21, no. 5 (2011): 2016–49. http://dx.doi.org/10.1214/10-aap749.

Full text

APA, Harvard, Vancouver, ISO, and other styles

36

Zhu, Quan-xin. "Variance minimization for continuous-time Markov decision processes: two approaches." Applied Mathematics-A Journal of Chinese Universities 25, no. 4 (2010): 400–410. http://dx.doi.org/10.1007/s11766-010-2428-1.

Full text

APA, Harvard, Vancouver, ISO, and other styles

37

Zhang, Junyu, and Xi-Ren Cao. "Continuous-time Markov decision processes with nth-bias optimality criteria." Automatica 45, no. 7 (2009): 1628–38. http://dx.doi.org/10.1016/j.automatica.2009.03.009.

Full text

APA, Harvard, Vancouver, ISO, and other styles

38

Wei, Qingda. "Finite approximation for finite-horizon continuous-time Markov decision processes." 4OR 15, no. 1 (2016): 67–84. http://dx.doi.org/10.1007/s10288-016-0321-3.

Full text

APA, Harvard, Vancouver, ISO, and other styles

39

Guo, Xianping, and Liuer Ye. "New discount and average optimality conditions for continuous-time Markov decision processes." Advances in Applied Probability 42, no. 4 (2010): 953–85. http://dx.doi.org/10.1239/aap/1293113146.

Full text

Abstract:

This paper deals with continuous-time Markov decision processes in Polish spaces, under the discounted and average cost criteria. All underlying Markov processes are determined by given transition rates which are allowed to be unbounded, and the costs are assumed to be bounded below. By introducing an occupation measure of a randomized Markov policy and analyzing properties of occupation measures, we first show that the family of all randomized stationary policies is ‘sufficient’ within the class of all randomized Markov policies. Then, under the semicontinuity and compactness conditions, we p

APA, Harvard, Vancouver, ISO, and other styles

40

Guo, Xianping, and Liuer Ye. "New discount and average optimality conditions for continuous-time Markov decision processes." Advances in Applied Probability 42, no. 04 (2010): 953–85. http://dx.doi.org/10.1017/s000186780000447x.

Full text

Abstract:

This paper deals with continuous-time Markov decision processes in Polish spaces, under the discounted and average cost criteria. All underlying Markov processes are determined by given transition rates which are allowed to be unbounded, and the costs are assumed to be bounded below. By introducing an occupation measure of a randomized Markov policy and analyzing properties of occupation measures, we first show that the family of all randomized stationary policies is ‘sufficient’ within the class of all randomized Markov policies. Then, under the semicontinuity and compactness conditions, we p

APA, Harvard, Vancouver, ISO, and other styles

41

Hu, Q. Y. "Nonstationary Continuous Time Markov Decision Processes in a Semi-Markov Environment with Discounted Criterion." Journal of Mathematical Analysis and Applications 194, no. 3 (1995): 640–59. http://dx.doi.org/10.1006/jmaa.1995.1322.

Full text

APA, Harvard, Vancouver, ISO, and other styles

42

HUANG, XiangXiang, and LiuEr YE. "A mean-variance optimization problem for continuous-time Markov decision processes." SCIENTIA SINICA Mathematica 44, no. 8 (2014): 883–98. http://dx.doi.org/10.1360/n012013-00117.

Full text

APA, Harvard, Vancouver, ISO, and other styles

43

Wei, Qingda, and Xian Chen. "Risk-sensitive average continuous-time Markov decision processes with unbounded rates." Optimization 68, no. 4 (2018): 773–800. http://dx.doi.org/10.1080/02331934.2018.1547382.

Full text

APA, Harvard, Vancouver, ISO, and other styles

44

Gao, Xuefeng, and Xun Yu Zhou. "Logarithmic Regret Bounds for Continuous-Time Average-Reward Markov Decision Processes." SIAM Journal on Control and Optimization 62, no. 5 (2024): 2529–56. http://dx.doi.org/10.1137/23m1584101.

Full text

APA, Harvard, Vancouver, ISO, and other styles

45

Guo, Xianping, and Lanlan Zhang. "Total reward criteria for unconstrained/constrained continuous-time Markov decision processes." Journal of Systems Science and Complexity 24, no. 3 (2011): 491–505. http://dx.doi.org/10.1007/s11424-011-8004-9.

Full text

APA, Harvard, Vancouver, ISO, and other styles

46

Zou, Xiaolong, and Yonghui Huang. "Verifiable conditions for average optimality of continuous-time Markov decision processes." Operations Research Letters 44, no. 6 (2016): 742–46. http://dx.doi.org/10.1016/j.orl.2016.09.007.

Full text

APA, Harvard, Vancouver, ISO, and other styles

47

Huo, Haifeng, Xiaolong Zou, and Xianping Guo. "The risk probability criterion for discounted continuous-time Markov decision processes." Discrete Event Dynamic Systems 27, no. 4 (2017): 675–99. http://dx.doi.org/10.1007/s10626-017-0257-6.

Full text

APA, Harvard, Vancouver, ISO, and other styles

48

Feinberg, Eugene A. "Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach." Mathematics of Operations Research 29, no. 3 (2004): 492–524. http://dx.doi.org/10.1287/moor.1040.0089.

Full text

APA, Harvard, Vancouver, ISO, and other styles

49

Guo, Xianping, and Zhong-Wei Liao. "Risk-Sensitive Discounted Continuous-Time Markov Decision Processes with Unbounded Rates." SIAM Journal on Control and Optimization 57, no. 6 (2019): 3857–83. http://dx.doi.org/10.1137/18m1222016.

Full text

APA, Harvard, Vancouver, ISO, and other styles

50

Liu, Qiuli, Hangsheng Tan, and Xianping Guo. "Denumerable continuous-time Markov decision processes with multiconstraints on average costs." International Journal of Systems Science 43, no. 3 (2012): 576–85. http://dx.doi.org/10.1080/00207721.2010.517868.

Full text

APA, Harvard, Vancouver, ISO, and other styles

We offer discounts on all premium plans for authors whose works are included in thematic literature selections. Contact us to get a unique promo code!