Academic literature on the topic 'Limiting state probabilities'

AbstractWe shall study continuous-time Markov chains on the nonnegative integers which are both irreducible and transient, and which exhibit discernible stationarity before drift to infinity “sets in”. We will show how this ‘quasi’ stationary behaviour can be modelled using a limiting conditional distribution: specifically, the limiting state probabilities conditional on not having left 0 for the last time. By way of a dual chain, obtained by killing the original process on last exit from 0, we invoke the theory of quasistationarity for absorbing Markov chains. We prove that the conditioned state probabilities of the original chain are equal to the state probabilities of its dual conditioned on non-absorption, thus allowing to establish the simultaneous existence and then equivalence, of their limiting conditional distributions. Although a limiting conditional distribution for the dual chain is always quasistationary distribution in the usual sense, a similar statement is not possible for the original chain.

AbstractThe article presents the three-state semi-Markov model of the process {W(t): t ≥ 0} of state transitions of a ship power plant machine, with the following interpretation of these states: s1 – state of full serviceability, s2 – state of partial serviceability, and s3 – state of unserviceability. These states are precisely defined for the ship main engine (ME). A hypothesis is proposed which explains the possibility of application of this model to examine models of real state transitions of ship power plant machines. Empirical data concerning ME were used for calculating limiting probabilities for the process {W(t): t ≥ 0}. The applicability of these probabilities in decision making with the assistance of the Bayesian statistical theory is demonstrated. The probabilities were calculated using a procedure included in the computational software MATHEMATICA, taking into consideration the fact that the random variables representing state transition times of the process {W(t): t ≥ 0} have gamma distributions. The usefulness of the Bayesian statistical theory in operational decision-making concerning ship power plants is shown using a decision dendrite which maps ME states and consequences of particular decisions, thus making it possible to choose between the following two decisions: d1 – first perform a relevant preventive service of the engine to restore its state and then perform the commissioned task within the time limit determined by the customer, and d2 – omit the preventive service and start performing the commissioned task.

In this paper we consider a fluid model of a single buffer in a random external environment modeled by an Ornstein-Uhlenbeck process. Such a model appears as the limiting case of the multiplexing model of Anick, Mitra, and Sondhi [2] in a heavy traffic environment. We use the change of measure technique to derive an exponential upper bound on the tail probabilities of the steady-state buffer content. We also establish an asymptotic upper and lower exponential bounds on the tail probabilities.

The fluctuations in stock prices produce a high risk that makes investors uncertain about their investment decisions. The present paper provides a methodology to forecast the long-term behavior of five randomly selected equities operating in the Malaysian construction sector. The method used in this study involves Markov chains as a stochastic analysis, assuming that the price changes have the proparty of Markov dependency with their transition probabilities. We identified a three-state Markov model (i.e., increase, stable, fall) and a two-state Markov model (i.e., increase and fall). The findings suggested that the chains had limiting distributions. The mean return time was computed for respective equities as well as to determine the average duration to return to a stock price increase. The analysis might aid investors in improving their investment knowledge, and they will be able to make better decisions when an equity portfolio possesses higher transition probabilities, higher limiting distribution, and lowest mean return time in response to a price increase. Finally, our investigations suggest that investors are more likely to invest in the GKent based on the three-state model, while VIZIONE seems to be a better investment choice based on a two-state model.

We use the notion of an invariant manifold to describe the long-term behaviour of absorbing continuous-time Markov processes with a denumerable infinity of states. We show that there exists an invariant manifold for the forward differential equations and we are able to describe the evolution of the state probabilities on this manifold. Our approach gives rise to a new method for calculating conditional limiting distributions, one which is also appropriate for dealing with processes whose transition probabilities satisfy a system of non-linear differential equations.

We use the notion of an invariant manifold to describe the long-term behaviour of absorbing continuous-time Markov processes with a denumerable infinity of states. We show that there exists an invariant manifold for the forward differential equations and we are able to describe the evolution of the state probabilities on this manifold. Our approach gives rise to a new method for calculating conditional limiting distributions, one which is also appropriate for dealing with processes whose transition probabilities satisfy a system of non-linear differential equations.

Background Electronic health record (EHR) data contain longitudinal patient information and standardized diagnostic codes. EHR data may be useful for estimating transition probabilities for state-transition models, but no guidelines exist on appropriate methods. We applied 3 potential methods to estimate transition probabilities from EHR data, using pediatric eating disorders (EDs) as a case study. Methods We obtained EHR data from PEDsnet, which includes 8 US children’s hospitals. Data included inpatient, outpatient, and emergency department visits for all patients with an ED. We mapped diagnoses to 3 ED health states: anorexia nervosa, bulimia nervosa, and other specified feeding or eating disorder. We estimated 1-y transition probabilities for males and females using 3 approaches: simple first-last proportions, a multistate Markov (MSM) model, and independent survival models. Results Transition probability estimates varied widely between approaches. The first-last proportion approach estimated higher probabilities of remaining in the same health state, while the MSM and independent survival approaches estimated higher probabilities of transitioning to a different health state. All estimates differed substantially from published literature. Limitations As a source of health state information, EHR data are incomplete and sometimes inaccurate. EHR data were especially challenging for EDs, limiting the estimation and interpretation of transition probabilities. Conclusions The 3 approaches produced very different transition probability estimates. Estimates varied considerably from published literature and were rescaled and calibrated for use in a microsimulation model. Estimation of transition probabilities from EHR data may be more promising for diseases that are well documented in the EHR. Furthermore, clinicians and health systems should work to improve documentation of ED in the EHR. Further research is needed on methods for using EHR data to inform transition probabilities.

We consider a classical population flow model in which individuals pass through n strata with certain state-dependent probabilities and at every time t = 0,1,2,…, there is a stochastic inflow of new individuals to every stratum. For a stationary inflow process we prove the convergence of the joint distribution of group sizes and derive the limiting Laplace transform.

We consider a classical population flow model in which individuals pass through n strata with certain state-dependent probabilities and at every time t = 0,1,2,…, there is a stochastic inflow of new individuals to every stratum. For a stationary inflow process we prove the convergence of the joint distribution of group sizes and derive the limiting Laplace transform.

Let (X, S) = {(Xn, Sn); n ≧0} be a Markov random walk with finite state space. For a ≦ 0 < b define the stopping times τ= inf {n:Sn > b} and T= inf{n:Sn∉(a, b)}. The diffusion approximations of a one-barrier probability P {τ < ∝ | Xo= i}, and a two-barrier probability P{ST ≧b | Xo = i} with correction terms are derived. Furthermore, to approximate the above ruin probabilities, the limiting distributions of overshoot for a driftless Markov random walk are involved.

Extracting useful knowledge from data sets is a key concept in modern information systems. Consequently, the need of efficient techniques to extract the desired knowledge has been growing over time. Machine learning is a research field dedicated to the development of techniques capable of enabling a machine to \"learn\" from data. Many techniques have been proposed so far, but there are still issues to be unveiled specially in interdisciplinary research. In this thesis, we explore the advantages of network data representation to develop machine learning techniques based on dynamical processes on networks. The network representation unifies the structure, dynamics and functions of the system it represents, and thus is capable of capturing the spatial, topological and functional relations of the data sets under analysis. We develop network-based techniques for the three machine learning paradigms: supervised, semi-supervised and unsupervised. The random walk dynamical process is used to characterize the access of unlabeled data to data classes, configuring a new heuristic we call ease of access in the supervised paradigm. We also propose a classification technique which combines the high-level view of the data, via network topological characterization, and the low-level relations, via similarity measures, in a general framework. Still in the supervised setting, the modularity and Katz centrality network measures are applied to classify multiple observation sets, and an evolving network construction method is applied to the dimensionality reduction problem. The semi-supervised paradigm is covered by extending the ease of access heuristic to the cases in which just a few labeled data samples and many unlabeled samples are available. A semi-supervised technique based on interacting forces is also proposed, for which we provide parameter heuristics and stability analysis via a Lyapunov function. Finally, an unsupervised network-based technique uses the concepts of pinning control and consensus time from dynamical processes to derive a similarity measure used to cluster data. The data is represented by a connected and sparse network in which nodes are dynamical elements. Simulations on benchmark data sets and comparisons to well-known machine learning techniques are provided for all proposed techniques. Advantages of network data representation and dynamical processes for machine learning are highlighted in all cases
A extração de conhecimento útil a partir de conjuntos de dados é um conceito chave em sistemas de informação modernos. Por conseguinte, a necessidade de técnicas eficientes para extrair o conhecimento desejado vem crescendo ao longo do tempo. Aprendizado de máquina é uma área de pesquisa dedicada ao desenvolvimento de técnicas capazes de permitir que uma máquina \"aprenda\" a partir de conjuntos de dados. Muitas técnicas já foram propostas, mas ainda há questões a serem reveladas especialmente em pesquisas interdisciplinares. Nesta tese, exploramos as vantagens da representação de dados em rede para desenvolver técnicas de aprendizado de máquina baseadas em processos dinâmicos em redes. A representação em rede unifica a estrutura, a dinâmica e as funções do sistema representado e, portanto, é capaz de capturar as relações espaciais, topológicas e funcionais dos conjuntos de dados sob análise. Desenvolvemos técnicas baseadas em rede para os três paradigmas de aprendizado de máquina: supervisionado, semissupervisionado e não supervisionado. O processo dinâmico de passeio aleatório é utilizado para caracterizar o acesso de dados não rotulados às classes de dados configurando uma nova heurística no paradigma supervisionado, a qual chamamos de facilidade de acesso. Também propomos uma técnica de classificação de dados que combina a visão de alto nível dos dados, por meio da caracterização topológica de rede, com relações de baixo nível, por meio de medidas de similaridade, em uma estrutura geral. Ainda no aprendizado supervisionado, as medidas de rede modularidade e centralidade Katz são aplicadas para classificar conjuntos de múltiplas observações, e um método de construção evolutiva de rede é aplicado ao problema de redução de dimensionalidade. O paradigma semissupervisionado é abordado por meio da extensão da heurística de facilidade de acesso para os casos em que apenas algumas amostras de dados rotuladas e muitas amostras não rotuladas estão disponíveis. É também proposta uma técnica semissupervisionada baseada em forças de interação, para a qual fornecemos heurísticas para selecionar parâmetros e uma análise de estabilidade mediante uma função de Lyapunov. Finalmente, uma técnica não supervisionada baseada em rede utiliza os conceitos de controle pontual e tempo de consenso de processos dinâmicos para derivar uma medida de similaridade usada para agrupar dados. Os dados são representados por uma rede conectada e esparsa na qual os vértices são elementos dinâmicos. Simulações com dados de referência e comparações com técnicas de aprendizado de máquina conhecidas são fornecidos para todas as técnicas propostas. As vantagens da representação de dados em rede e de processos dinâmicos para o aprendizado de máquina são evidenciadas em todos os casos

The diminishing potential for development of further reservoirs, coupled with environmental awareness about the negative environmental impacts of reservoir construction and operation, has not only necessitated the need for improved operation of reservoirs through better planning, but has also created an additional demand on reservoirs in the form of instream flow requirements to preserve the ecological integrity of the rivers. The combination of these factors has lead to a considerable interest in both private and government water resources engineering practice in the use of mathematical models for optimisation of reservoir operations.

The optimisation approaches which have been most commonly used for planning the operation of reservoirs are dynamic programming (DP), linear programming (LP) and non-linear programming. While all of these techniques are reasonably effective for optimisation of operation of single reservoirs, dynamic programs are by far the most frequently used partly because of the ease with which they can handle stochasticity of inflow regimes. However, while all the techniques, including dynamic programming techniques, are relatively easily applied to optimisation of single reservoirs, serious theoretical and computational issues arise when they are applied to the optimisation of the operations of multiple reservoir systems, particularly when stochastic issues related to inflow regimes or variation in demands are considered. For this reason, none of the above techniques have been able to be applied directly to the simultaneous optimisation of the operation of multiple reservoir systems. Instead, the optimisation processes have relied upon approximations such as decomposition of a system or joint simulation-optimisation approaches.

The research reported in this thesis proposes a new approach to optimisation of the operation of reservoir systems, particularly multiple reservoir systems. This approach enables improved levels of consideration of the stochasticity of the inflow process while also significantly reducing the computational requirement and permits a more detailed and accurate representation of the system within the optimisation process. The approach is based on consideration of the stochasticity of inflows through the concept of Limiting State Probabilities. These Limiting State Probabilities rely on an assumption of stationarity of monthly transition probability matrices, an assumption which is also commonly used in stochastic reservoir operation models and define a probability distribution of inflows which, for each time period, are independent of the flows in the previous month, but which implicitly incorporate the time period to time period correlations normally captured by Markov processes. The Limiting State Probability vectors for each time period are obtained by a process of multiplication of the transition probability matrices associated with the inflows in that time period and the time period immediately preceding it. These Limiting State Probability vectors are the same as the marginal probabilities of inflows derived from steady state solutions in stochastic dynamic programs. The ability of Limiting State Probability vectors to remove the explicit temporal correlations is derived from the close relationship of Limiting State Probabilities to the long term steady state conditions of optimal reservoir operation. The elimination of temporal correlation also enables the spatial correlation between the inflows to reservoirs at different locations to be considered implicitly rather than explicitly. The spatial correlation is able to be eliminated from explicit consideration in the inflows to the model because the removal of the time period to time period correlation means that the results of a deterministic optimisation of reservoir using an inflow sequence generated by and conforming to the Limiting State Probability are independent of the actual order of inflows in that inflow series. This non-dependence of the results of the optimisation on the order of inflows enables the Limiting State Probability generated inflow sequences to be used as input to each reservoir in a multiple reservoir system with a diminished need to consider spatial correlation of inflows explicitly. The approach is validated first by application to the optimisation of the operation of a single reservoir wherein it is shown that the same results, i.e., optimal operating policies, are obtained when Limiting State Probabilities rather than traditional transition probability matrices are used in the recursive equations of the stochastic dynamic program. Optimal operation of the same single reservoir was then performed by the deterministic modelling technique network linear programming using inflow sequences generated by Limiting State Probabilities. The results obtained from the optimisation technique were similar to those obtained by stochastic dynamic programming with some of the differences being due to use of discrete variables in stochastic dynamic programming and continuous variables in the network linear program. Use of the Limiting State Probability concept was then extended to simultaneous optimisation, using network linear programming, of a multiple reservoir system comprising six reservoirs and seventeen demand centres, plus instream flow requirements. The deterministic inflow inputs, i.e., inflow sequences to each reservoir required by network linear programming were generated on the basis of Limiting State Probabilities relevant to each reservoir. The results of the application of the NLP technique using the inflows generated by Limiting State Probabilities showed the approach to be a computationally tractable and effective means to improved level of consideration of stochasticity of inflows in the optimisation of the operation of multiple reservoir systems.

It was argued in Chapter 1 that various kinds of propensities make intuitive sense, but existing propensity theories do little to clarify their nature. We are now in a position to give precise definitions for several different kinds of propensities in terms of nomic probabilities. The characteristics of these propensities can then be deduced from the theory of nomic probability. The technique for defining propensities in terms of nomic probabilities is modeled on the definitions of objective and physical/epistemic definite probabilities proposed in Chapter 4. We often want to know how probable it is that P would be true if Q were true. This is a kind of counterfactual probability, and I will symbolize it as ┌prob(P/Q)┐. Counterfactual conditionals constitute the limiting case of counterfactual probability, in the sense that if (Q > P) obtains then prob(P/Q) = 1. prob(PlQ) is to be distinguished from prob(P/g) and PROB(P/(2), either of which can be regarded as an “indicative” probability that P is true if Q is true. Recall that (Q > P) obtains iff P obtains at every nearest world at which Q obtains. In other words, where M(0 is the set of nearest ^-worlds, (Q > P) obtains iff M(Q) ⊆ |P|. Analogously, prob(P/Q) can be regarded as a measure of the proportion of nearest g-worlds that are also P-worlds. This has the immediate result that if (Q > P) obtains then prob(PIQ) = 1 (but not conversely). This sort of heuristic description of counterfactual probability enables us to investigate its formal properties, but if the notion is to be of any real use we must do more than wave our hands and talk about an unspecified measure on M(Q). We are now in a position to accurately define counterfactual probability. Let CQ be the conjunction of all of the counterfactual consequences of Q, that is, the conjunction of all states of affairs R such that (Q > R) obtains.

Abstract A marine operation is a complex non-routine activity of limited duration carried out in offshore environment. Due to safety reasons, these operations are normally performed within specific sea state limits, which are derived from numerical modelling and analysis of hazardous events. In view of the uncertainties in the assessment of structural responses under stochastic environmental conditions, these limiting curves correspond to a target structural failure probability recommended in offshore standards (for example, 10−4 per operation as specified by DNV-GL). However, one of the main limitations is that these curves do not reflect site-specific safety assessment. The current paper presents a novel methodology for assessing the structural safety level of marine operations from a long-term perspective. The methodology includes estimation of extreme response distribution under all possible operational sea states (i.e. the operational domain under the limiting sea states) for a given offshore site and is compared to the response limit to obtain an average failure probability. A case study is also presented for a blade root mating process onto preassembled hub using a jack-up crane vessel and risk of impact between root and hub is considered critical. Global time-domain simulations are performed using multibody dynamics, and extreme value distributions for impact velocities are derived for different wind-wave conditions. The allowable impact velocity between the blade root and the hub is determined by an explicit finite element analysis of the damage at the blade root. Finally, the average failure probabilities considering the operational domain are obtained for four different European offshore sites and are compared to the target level of structural failure probability considered for the limiting sea states.