Tesis sobre el tema "Ergodic control"

The core of this thesis focuses on a number of different aspects of ergodic stochastic control in connection with backward stochastic differential equations (BSDEs for short). Chapter 1 serves as an introduction to the problem formulation in various contexts and states a number of results we will be using in the sequel. Chapter 2 deals with the so called weak formulation, where the control is represented as a change of measure. The optimal value and feedback control are obtained using a relatively recent object called ergodic BSDEs. In order to achieve this we establish the existence and uniqueness of solutions to these equations along the way. Chapter 3 is concerned with non zero-sum games where multiple players control the drift of a process, and their payoffs depend on its ergodic behaviour. We establish their connection with systems of Ergodic BSDEs, and prove the existence of a Nash equilibrium under general conditions. In Chapter 4 we show a novel duality between the existence of a solution to an infinite horizon adjoint BSDE and strong dissipativity of the forward process. Thus the link between ergodicity of the controlled process and the infinite horizon stochastic maximum principle is established. Finally, in Chapter 5 we provide conclusions, conjectures and directions for future research.

This thesis is concerned with nonlinear elliptic PDEs and system of PDEs arising in various problems of stochastic control, differential games, specifically Mean Field Games, and differential geometry. It is divided in three parts. The first part is focused on stochastic ergodic control problems where both the state and the control space is R^d. The interest is in giving conditions on the fixed drift, the cost function and the Lagrangian function that are sufficient for synthesizing an optimal control of feedback type. In order to obtain such conditions, an approach that combines the Lyapunov method and the approximation of the problem on bounded sets with reflection of the diffusions at the boundary is proposed. A general framework is developed first, and then particular cases are considered, which show how Lyapunov functions can be constructed from the solutions of the approximating problems. The second part is devoted to the study of Mean Field Games, a recent theory which aims at modeling and analyzing complex decision processes involving a very large number of indistinguishable rational agents. The attention is given to existence results for the multi- population MFG system of PDEs with homogeneous Neumann boundary conditions, that are obtained combining elliptic a-priori estimates and fixed point arguments. A model of segregation between human populations, inspired by ideas of T. Schelling is then proposed. The model, that fits into the theoretical framework developed in the thesis, is analyzed from the qualitative point of view using numerical finite-difference techniques. The phenomenon of segregation between the population densities arises in the numerical experiments on the particular mean field game model, assuming mild ethnocentric attitude of people as in the original model of Schelling. In the last part of the thesis some results on existence and uniqueness of solutions for the prescribed k-th principal curvature equation are presented. The Dirichlet problem for such a family of degenerate elliptic fully nonlinear partial differential equations is solved using the theory of Viscosity solutions, by implementing a version of the Perron method which involves semiconvex subsolutions; the restriction to this class of functions is sufficient for proving a Lipschitz estimate on the elliptic operator with respect to the gradient entry which is also required for obtaining the comparison principle. Existence and uniqueness are stated under general assumptions, and examples of data which satisfy the general hypotheses are provided.
Questa tesi ha come oggetto di studio EDP ellittiche nonlineari e sistemi di EDP che si presentano in problemi di controllo stocastico, giochi differenziali, in particolare Mean Field Games e geometria differenziale. I risultati contenuti si possono suddividere in tre parti. Nella prima parte si pone l'attenzione su problemi di controllo ergodico stocastico dove lo spazio degli stati e dei controlli coincide con l'intero Rd. L'interesse è posto sul formulare condizioni sul drift, il funzionale di costo e la Lagrangiana sufficienti a sintetizzare un controllo ottimo di tipo feedback. Al fine di ottenere tali condizioni, viene proposto un approccio che combina il metodo delle funzioni di Lyapunov e l'approssimazione del problema su domini limitati con riflessione delle traiettorie al bordo. Le tecniche vengono formulate in termini generali e successivamente sono presi in considerazione esempi specifici, che mostrano come opportune funzioni di Lyapunov possono essere costruite a partire dalle soluzioni dei problemi approssimanti. La seconda parte è incentrata sullo studio di Mean Fielda Games, una recente teoria che mira a elaborare modelli per analizzare processi di decisione in cui è coinvolto un grande numero di agenti indistinguibili. Sono ottenuti nella tesi alcuni risultati di esistenza di soluzioni per sistemi MFG a più popolazioni con condizioni al bordo omogenee di tipo Neumann, attraverso stime a-priori ellittiche e argomenti di punto fisso. Viene in seguito proposto un modello di segregazione tra popolazioni umane che prende ispirazione da alcune idee di T. Schelling. Tale modello si inserisce nel contesto teorico sviluppato nella tesi, e viene analizzato dal punto di vista qualitativo tramite tecniche numeriche alle differenze finite. Il fenomeno di segregazione tra popolazioni si riscontra negli esperimenti numerici svolti sul particolare modello mean field, assumendo l'ipotesi di moderata mentalità etnocentrica delle persone, similmente all’originale modello di Schelling. L'ultima parte della tesi riguarda alcuni risultati di esistenza e unicità di soluzioni per l’equazione di k-esima curvatura principale prescritta. Il problema di Dirichlet per tale famiglia di equazioni ellittiche degeneri nonlineari è risolto implementando la teoria delle soluzioni di Viscosità, applicando in particolare una versione del metodo di Perron basata su soluzioni semiconvesse; la restrizione a tale classe di funzioni risulta sufficiente per dimostrare una stima di tipo Lipschitz sull'operatore ellittico, essenziale per ottenere un principio di confronto. Esistenza e unicità di soluzioni sono formulate in termini generali; vengono forniti infine esempi in cui condizioni particolari sui dati soddisfano tali ipotesi.

The work in this thesis concerns the analysis of first-order mean field game (MFG) systems with control of acceleration and the study of the long time-average behavior of control systems of sub-Riemannian type. More precisely, in the first part we begin by studying the well-posedness of the MFG system associated with a control problem with linear state equation. In particular, via a relaxed approach, we prove the existence and the uniqueness of mild solutions and we also study their regularity. Then, we focus on the MFG system with control of the acceleration, a particular case of the one above, and we investigate the long time-average behavior of solutions showing the convergence to the critical constant. Here, as for the previous analysis, the main issues are the lack of strict convexity and coercivity of the Hamiltonian with respect to the momentum variable. Indeed, for instance, when studying the asymptotic behavior of the control system this lead us to a non existence result of continuous viscosity solutions to the ergodic Hamilton-Jacobi equation. Consequently, it does not allowed us to the define the ergodic MFG system as one would expect. We conclude this first part establishing a connection between the MFG system with control of acceleration and the classical one. To do so, we study the singular perturbation problem for MFG system of acceleration, that is, we analyze the behavior of solutions to the system when the acceleration cost goes to zero. Again, we solve the problem by using variation techniques due to the problems arising from the lack of strict convexity and coercivity of the Hamiltonian with respect to the momentum variable. In the second part, we concentrate the attention to drift-less affine control systems (sub-Riemannian type). Differently from the case of acceleration, we prove that there exists a critical constant and the ergodic Hamilton-Jacobi equation associated with such a constant has continuous viscosity solutions. This is possible appealing to the properties of the sub-Riemannian geometry on the state space. Still using the properties of this geometry we finally define the Lax-Oleinink semigroup and we prove the existence of a fixed point of such semigroup. We conclude this part, and thus this thesis, extending the celebrated Aubry-Mather Theory to the case of sub-Riemannian control system. We first show a variational representation formula for the critical constant and from this we define the Aubry set. By using a dynamical approach we study the analytical and topological properties of such sets as, for instance, horizontal differentiability of the critical solution at any points lying in such a set.

For robotic systems to continue to move towards ubiquity, robots need to be more autonomous. More autonomy dictates that robots need to be able to make better decisions. Control theory and machine learning are fields of robotics that focus on the decision making process. However, each of these fields implements decision making at different levels of abstraction and at different time scales. Control theory defines low-level decisions at high rates, while machine learning defines high-level decision at low rates. The objective of this research is to integrate tools from both machine leaning and control theory to solve higher dimensional, complex problems, and to optimize the decision making process. Throughout this research, multiple algorithms were created that use concepts from both control theory and machine learning, which provide new tools for robots to make better decisions. One algorithm enables a robot to learn how to optimally explore an unknown space, and autonomously decide when to explore for new information or exploit its current information. Another algorithm enables a robot to learn how to locomote with complex dynamics. These algorithms are evaluated both in simulation and on real robots. The results and analysis of these experiments are presented, which demonstrate the utility of the algorithms introduced in this work. Additionally, a new notion of “learnability” is introduced to define and determine when a given dynamical system has the ability to gain knowledge to optimize a given objective function.

In this thesis we treat the first singular perturbation problem of a stochastic model with unbounded and controlled fast variables with success. Our methods are based on the theory of viscosity solutions, homogenisation of fully nonlinear PDEs and a careful analysis of the associated ergodic stochastic control problem in the whole space R^m. The text is divided in two parts. In the first chapter, we investigate the existence and uniqueness as well as a suitable stability of the solution to the associated ergodic problem that are crucial to characterize the effective Hamiltonian of the limit (effective) Cauchy problem in Chapter II of this thesis. The main achievement obtained in this part is a purely analytical proof for the uniqueness of solution to such ergodic problem. Since the state space of the problem is not compact, in general there are infinitely many solutions to the ergodic problem. However, if one restrict the class of solutions to the set of bounded-below functions, then it is known that uniqueness holds up to an additive constant. The existing proof relies on some probabilistic techniques employing the invariant probability measure for the associated stochastic process. Here we give a new proof, purely analytic, based on the strong maximum principle. We believe that our results can be interesting and useful for researchers in the PDE community. In the second chapter, we introduce our singular perturbation model of a stochastic control problem and we prove our main result: the convergence of the value function $V^\epsilon$ associated to the problem to the solution of the limiting equation. More precisely, we prove that the functions \underline{V} (t,x):=\liminf_{(\epsilon,t',x') \to (0,t,x)} \inf_{y \in \mathbb{R}^m} V^\epsilon (t',x',y) and \bar{V} (t,x) :=(\sup_{R} \bar{V}_R)^* (t,x) \text{ (upper semi-continuous envelope of $\sup_{R} \bar{V}_R$ )} where $\bar{V}_{R} (t,x):=\limsup_{(\epsilon, t',x') \to (0,t,x)} \sup_{y \in B_R (0)} V^\epsilon (t',x',y)$, are, respectively, a super and a subsolution of the effective Cauchy problem. As a corollary of this result, $V^\epsilon$ converges to the unique solution $V$ of the effective equation provided the equation admits the comparison principle for discontinuous viscosity solutions. The justification of this convergence is not trivial at all. It especially involves some regularity issues and a careful treatment of viscosity techniques and stochastic analysis. This result has never been obtained before.
In questa tesi viene trattato con successo il primo problema di perturbazione singolare di un modello stocastico con variabili veloci controllate e non limitate. I metodi si basano sulla teoria delle soluzioni di viscosità, omogeinizzazione dei PDE completamente non lineari, e su un'attenta analisi del problema stocastico ergodico associato, valido nell'intero spazio R^m. Il testo è diviso in due parti. Nel primo capitolo, saranno studiate l'esistenza, l'unicità e alcune proprietà di stabilità della soluzione del problema ergodico, riferito sopra, che sono essenziali per caratterizzare il Hamiltoniano effettivo che appare in un Problema di Cauchy "limite", che sarà descritto nel capitolo II di questa tesi. Il principale contributo, presentato in questa parte, è una prova puramente analitica dell'unicità della soluzione di questo problema ergodico. Siccome lo stato dello spazio del problema non è compatto, in generale ci sono un numero infinito di soluzioni a questo problema. Tuttavia, se uno limitasse la classe di soluzioni all'insieme di funzioni limitate inferiormente, allora è noto che l'unicità sarà mantenuta a meno di una costante. La prova esistente si basa su alcune tecniche probabilistiche che impiegano la misura di probabilità invariante per l'associato processo stocastico. Qua verrà data una nuova prova, puramente analitica, basata sul principio del massimo. Si ritiene che il risultato potrà essere interessante ed utile per i ricercatori che lavorano all'interno della comunità di ricerca delle Equazioni Differenziali alle derivate Parziali (PDE). Nel secondo capitolo, sarà introdotto un modello di perturbazione singolare di un problema di controllo stocastico, e provato il risultato principale: la convergenza della funzione valore $V^\epsilon$, associata al nostro problema, per soluzione dell'equazione limite. Più precisamente, sarà provato che le funzioni: \underline{V} (t,x):=\liminf_{(\epsilon,t',x') \to (0,t,x)} \inf_{y \in \mathbb{R}^m} V^\epsilon (t',x',y) e \bar{V} (t,x) :=(\sup_{R} \bar{V}_R)^* (t,x) \text{ (upper semi-continuous envelope of $\sup_{R} \bar{V}_R$ )} dove $\bar{V}_{R} (t,x):=\limsup_{(\epsilon, t',x') \to (0,t,x)} \sup_{y \in B_R (0)} V^\epsilon (t',x',y)$, sono, rispettivamente, una super soluzione e una sottosoluzione del problema effettivo di Cauchy. Come corollario di questo risultato, $V^\epsilon$ converge all'unica soluzione V della equazione effettiva se l'equazione limite permette il principio di comparazione per le soluzioni di viscosità discontinue. La motivazione di questa convergenza non è ovvia del tutto. Coinvolge specialmente alcuni problemi di regolarità e un trattamento attento delle tecniche di viscosità e di analisi stocastica. Questo risultato è nuovo e non è mai stato ottenuto, prima d'ora, nella letteratura Matematica.

We report progress made on an analytic investigation of low-frequency cardiorespiratory variability in humans. The work is based on an existing physiological model of chemically-mediated blood-gas control via the central and peripheral chemoreceptors, that of Grodins, Buell & Bart (1967). Scaling and simplification of the Grodins model yields a rich variety of dynamical subsets; the thesis focusses on the dynamics obtained under the normoxic assumption (i.e., when oxygen is decoupled from the system). In general, the method of asymptotic reduction yields submodels that validate or invalidate numerous (and more heuristic) extant efforts in the literature. Some of the physiologically-relevant behaviour obtained here has therefore been reported before, but a large number of features are reported for the first time. A particular novelty is the explicit demonstration of cardiorespiratory coupling via chemosensory control. The physiology and literature reviewed in Chapters 1 and 2 set the stage for the investigation. Chapter 3 scales and simplifies the Grodins model; Chapters 4, 5, 6 consider carbon dioxide dynamics at the central chemoreceptor. Chapter 7 begins analysis of the dynamics mediated by the peripheral receptor. Essentially all of the dynamical behaviour is due to the effect of time delays occurring within the conservation relations (which are ordinary differential equations). The pathophysiology highlighted by the analysis is considerable, and includes central nervous system disorders, heart failure, metabolic diseases, lung disorders, vascular pathologies, physiological changes during sleep, and ascent to high altitude. Chapter 8 concludes the thesis with a summary of achievements and directions for further work.

We study dynamical systems in the complex plane under the effect of constant noise. We show for a wide class of polynomial equations that the ergodic property is valid in the associated stochastic perturbation if and only if the noise added is in the direction transversal to all unstable trajectories of the deterministic system. This has the interpretation that noise in the "right" direction prevents the process from being unstable: a fundamental, but not well-understood, geometric principle which seems to underlie many other similar equations. The result is proven by using Lyapunov functions and geometric control theory.

L'objectif principal de cette thèse est de comprendre les interactions entre les agents financiers et le carnet d'ordres. Elle se compose de six chapitres inter-connectés qui peuvent toutefois être lus indépendamment.Nous considérons dans le premier chapitre le problème de contrôle d'un agent cherchant à prendre en compte la liquidité disponible dans le carnet d'ordres afin d'optimiser le placement d'un ordre unitaire. Notre stratégie permet de réduire le risque de sélection adverse. Néanmoins, la valeur ajoutée de cette approche est affaiblie en présence de temps de latence: prédire les mouvements futurs des prix est peu utile si le temps de réaction des agents est lent.Dans le chapitre suivant, nous étendons notre étude à un problème d'exécution plus général où les agents traitent des quantités non unitaires afin de limiter leur impact sur le prix. Notre tactique permet d'obtenir de meilleurs résultats que les stratégies d'exécution classiques.Dans le troisième chapitre, on s'inspire de l'approche précédente pour résoudre cette fois des problèmes de market making plutôt que des problèmes d'exécution. Ceci nous permet de proposer des stratégies pertinentes compatibles avec les actions typiques des market makers. Ensuite, nous modélisons les comportements des traders haute fréquence directionnels et des brokers institutionnels dans le but de simuler un marché où nos trois types d'agents interagissent de manière optimale les uns avec les autres.Nous proposons dans le quatrième chapitre un modèle d'agents où la dynamique des flux dépend non seulement de l'état du carnet d'ordres mais aussi de l'historique du marché. Pour ce faire, nous utilisons des généralisations des processus de Hawkes non linéaires. Dans ce cadre, nous sommes en mesure de calculer en fonction de flux individuels plusieurs indicateurs pertinents. Il est notamment possible de classer les market makers en fonction de leur contribution à la volatilité.Pour résoudre les problèmes de contrôle soulevés dans la première partie de la thèse, nous avons développé des schémas numériques. Une telle approche est possible lorsque la dynamique du modèle est connue. Lorsque l'environnement est inconnu, on utilise généralement les algorithmes itératifs stochastiques. Dans le cinquième chapitre, nous proposons une méthode permettant d'accélérer la convergence de tels algorithmes.Les approches considérées dans les chapitres précédents sont adaptées pour des marchés liquides utilisant le mécanisme du carnet d'ordres. Cependant, cette méthodologie n'est plus nécessairement pertinente pour des marchés régis par des règles de fonctionnement spécifiques. Pour répondre à cette problématique, nous proposons, dans un premier temps, d'étudier le comportement des prix sur le marché très particulier de l'électricité
This thesis aims at understanding the interactions between the market participants and the order book. It consists of six connected chapters which can however be read independently.In the first chapter, we tackle the control problem of an agent who wish to exploit the order book liquidity to optimise the placement of a unit limit order. We show that our optimal tactic reduces the adverse selection risk. Nonetheless, the added value of taking into account order book liquidity is eroded by latency: being able to predict future price moves is less profitable if agents reaction time is large.In the next chapter, we extend our study to more general execution problems where agents handle non-unit quantities to mitigate their price impact. We show that our optimal tactic enables us to outperform significantly standard execution strategies.The third chapter adapts our previous approach to solve market making issues. This enables us to propose relevant strategies which are consistent with typical market makers behaviours. After that, we model the behaviours of directional high frequency traders and institutional brokers in order to simulate an order book driven market with our three classes of agents interacting optimally with each others.We introduce in the fourth chapter an agent-based model where the dynamics of the flow depend not only on the order book state but also on the history of the market. For this, we use generalisations of non-linear Hawkes processes. In this setting, we are able to compute several relevant microstructural indicators in terms of the individual flows. It is notably possible to rank market makers according to their own contribution to volatility.To solve the control problems appearing in the first part of the thesis, we develop numerical schemes. This is possible when the dynamic of the model is known. To tackle control problems in an unknown environment, it is common to use stochastic iterative algorithms. In the fifth chapter, we propose a method that accelerates the convergence of such algorithms.The approaches built in the previous chapters are appropriate for liquid markets that use an order book mechanism. However our methodologies may not be suitable for exchanges with very specific operating rules. To investigate this issue, as a first step, we study the price behaviour of the very particular intra-day electricity market

Les jeux stochastiques à somme nulle possèdent une structure récursive qui s'exprime dans leur opérateur de programmation dynamique, appelé opérateur de Shapley. Ce dernier permet d'étudier le comportement asymptotique de la moyenne des paiements par unité de temps. En particulier, le paiement moyen existe et ne dépend pas de l'état initial si l'équation ergodique - une équation non-linéaire aux valeurs propres faisant intervenir l'opérateur de Shapley - admet une solution. Comprendre sous quelles conditions cette équation admet une solution est un problème central de la théorie de Perron-Frobenius non-linéaire, et constitue le principal thème d'étude de cette thèse. Diverses classes connues d'opérateur de Shapley peuvent être caractérisées par des propriétés basées entièrement sur la relation d'ordre ou la structure métrique de l'espace. Nous étendons tout d'abord cette caractérisation aux opérateurs de Shapley "sans paiements", qui proviennent de jeux sans paiements instantanés. Pour cela, nous établissons une expression sous forme minimax des fonctions homogènes de degré un et non-expansives par rapport à une norme faible de Minkowski. Nous nous intéressons ensuite au problème de savoir si l'équation ergodique a une solution pour toute perturbation additive des paiements, problème qui étend la notion d'ergodicité des chaînes de Markov. Quand les paiements sont bornés, cette propriété d'"ergodicité" est caractérisée par l'unicité, à une constante additive près, du point fixe d'un opérateur de Shapley sans paiement. Nous donnons une solution combinatoire s'exprimant au moyen d'hypergraphes à ce problème, ainsi qu'à des problèmes voisins d'existence de points fixes. Puis, nous en déduisons des résultats de complexité. En utilisant la théorie des opérateurs accrétifs, nous généralisons ensuite la condition d'hypergraphes à tous types d'opérateurs de Shapley, y compris ceux provenant de jeux dont les paiements ne sont pas bornés. Dans un troisième temps, nous considérons le problème de l'unicité, à une constante additive près, du vecteur propre. Nous montrons d'abord que l'unicité a lieu pour une perturbation générique des paiements. Puis, dans le cadre des jeux à information parfaite avec un nombre fini d'actions, nous précisons la nature géométrique de l'ensemble des perturbations où se produit l'unicité. Nous en déduisons un schéma de perturbations qui permet de résoudre les instances dégénérées pour l'itération sur les politiques
Zero-sum stochastic games have a recursive structure encompassed in their dynamic programming operator, so-called Shapley operator. The latter is a useful tool to study the asymptotic behavior of the average payoff per time unit. Particularly, the mean payoff exists and is independent of the initial state as soon as the ergodic equation - a nonlinear eigenvalue equation involving the Shapley operator - has a solution. The solvability of the latter equation in finite dimension is a central question in nonlinear Perron-Frobenius theory, and the main focus of the present thesis. Several known classes of Shapley operators can be characterized by properties based entirely on the order structure or the metric structure of the space. We first extend this characterization to "payment-free" Shapley operators, that is, operators arising from games without stage payments. This is derived from a general minimax formula for functions homogeneous of degree one and nonexpansive with respect to a given weak Minkowski norm. Next, we address the problem of the solvability of the ergodic equation for all additive perturbations of the payment function. This problem extends the notion of ergodicity for finite Markov chains. With bounded payment function, this "ergodicity" property is characterized by the uniqueness, up to the addition by a constant, of the fixed point of a payment-free Shapley operator. We give a combinatorial solution in terms of hypergraphs to this problem, as well as other related problems of fixed-point existence, and we infer complexity results. Then, we use the theory of accretive operators to generalize the hypergraph condition to all Shapley operators, including ones for which the payment function is not bounded. Finally, we consider the problem of uniqueness, up to the addition by a constant, of the nonlinear eigenvector. We first show that uniqueness holds for a generic additive perturbation of the payments. Then, in the framework of perfect information and finite action spaces, we provide an additional geometric description of the perturbations for which uniqueness occurs. As an application, we obtain a perturbation scheme allowing one to solve degenerate instances of stochastic games by policy iteration

This thesis is concerned with methodologies for the accurate quantitative modelling of molecular biological systems. The first part is devoted to the chemical Langevin equation (CLE), a stochastic differential equation driven by a multidimensional Wiener process. The CLE is an approximation to the standard discrete Markov jump process model of chemical reaction kinetics. It is valid in the regime where molecular populations are abundant enough to assume their concentrations change continuously, but stochastic fluctuations still play a major role. We observe that the CLE is not a single equation, but a family of equations with shared finite-dimensional distributions. On the theoretical side, we prove that as many Wiener processes are sufficient to formulate the CLE as there are independent variables in the equation, which is just the rank of the stoichiometric matrix. On the practical side, we show that in the case where there are m_1 pairs of reversible reactions and m_2 irreversible reactions, there is another, simple formulation of the CLE with only m_1+m_2 Wiener processes, whereas the standard approach uses 2m_1+m_2. Considerable computational savings are achieved with this latter formulation. A flaw of the CLE model is identified: trajectories may leave the nonnegative orthant with positive probability. The second part addresses the challenge when alternative, structurally different ordinary differential equation models of similar complexity fit the available experimental data equally well. We review optimal experiment design methods for choosing the initial state and structural changes on the biological system to maximally discriminate between the outputs of rival models in terms of L_2-distance. We determine the optimal stimulus (input) profile for externally excitable systems. The numerical implementation relies on sum of squares decompositions and is demonstrated on two rival models of signal processing in starving Dictyostelium amoebae. Such experiments accelerate the perfection of our understanding of biochemical mechanisms.

The demand for automated decision making, learning and inference in uncertain, risk sensitive and dynamically changing situations presents a challenge: to design computational approaches that promise to be widely deployable and flexible to adapt on the one hand, while offering reliable guarantees on safety on the other. The tension between these desiderata has created a gap that, in spite of intensive research and contributions made from a wide range of communities, remains to be filled. This represents an intriguing challenge that provided motivation for much of the work presented in this thesis. With these desiderata in mind, this thesis makes a number of contributions towards the development of algorithms for automated decision-making and inference under uncertainty. To facilitate inference over unobserved effects of actions, we develop machine learning approaches that are suitable for the construction of models over dynamical laws that provide uncertainty bounds around their predictions. As an example application for conservative decision-making, we apply our learning and inference methods to control in uncertain dynamical systems. Owing to the uncertainty bounds, we can derive performance guarantees of the resulting learning-based controllers. Furthermore, our simulations demonstrate that the resulting decision-making algorithms are effective in learning and controlling under uncertain dynamics and can outperform alternative methods. Another set of contributions is made in multi-agent decision-making which we cast in the general framework of optimisation with interaction constraints. The constraints necessitate coordination, for which we develop several methods. As a particularly challenging application domain, our exposition focusses on collision avoidance. Here we consider coordination both in discrete-time and continuous-time dynamical systems. In the continuous-time case, inference is required to ensure that decisions are made that avoid collisions with adjustably high certainty even when computation is inevitably finite. In both discrete-time and finite-time settings, we introduce conservative decision-making. That is, even with finite computation, a coordination outcome is guaranteed to satisfy collision-avoidance constraints with adjustably high confidence relative to the current uncertain model. Our methods are illustrated in simulations in the context of collision avoidance in graphs, multi-commodity flow problems, distributed stochastic model-predictive control, as well as in collision-prediction and avoidance in stochastic differential systems. Finally, we provide an example of how to combine some of our different methods into a multi-agent predictive controller that coordinates learning agents with uncertain beliefs over their dynamics. Utilising the guarantees established for our learning algorithms, the resulting mechanism can provide collision avoidance guarantees relative to the a posteriori epistemic beliefs over the agents' dynamics.

Controlled linear stochastic evolution equations driven by Lévy processes are studied in the Hilbert space setting. The control operator may be unbounded which makes the results obtained in the abstract setting applicable to parabolic SPDEs with boundary or point control. The first part contains some preliminary technical results, notably a version of Itô formula which is applicable to weak/mild solutions of controlled equations. In the second part, the ergodic control problem is solved: The feedback form of the optimal control and the formula for the optimal cost are found. The control problem is solved in the mean-value sense and, under selective conditions, in the pathwise sense. As examples, various parabolic type controlled SPDEs are studied. 1

We partially solve the adaptive ergodic stochastic optimal control problem where the driving process is a fractional Brownian motion with Hurst parameter H > 1/2. A formula is provided for an optimal feedback control given a strongly consistent estimator of the parameters of the controlled system is avail- able. There are some special conditions imposed on the estimator which means the results are not completely general. They apply, for example, in the case where the estimator is independent of the driving fractional Brownian motion. In the course of the thesis, construction of stochastic integrals of suitable determinis- tic functions with respect to fractional Brownian motion with Hurst parameter H > 1/2 over the unbounded positive real half-line is presented as well. 1