Chaîne de Markov | EPFL Graph Search

vignette|Exemple élémentaire de chaîne de Markov, à deux états A et E. Les flèches indiquent les probabilités de transition d'un état à un autre. En mathématiques, une chaîne de Markov est un processus de Markov à temps discret, ou à temps continu et à espace d'états discret. Un processus de Markov est un processus stochastique possédant la propriété de Markov : l'information utile pour la prédiction du futur est entièrement contenue dans l'état présent du processus et n'est pas dépendante des états antérieurs (le système n'a pas de « mémoire »). Les processus de Markov portent le nom de leur inventeur, Andreï Markov. Un processus de Markov à temps discret est une séquence X_0, X_1, X_2, X_3,\ \dots de variables aléatoires à valeurs dans l’espace des états, qu'on notera E dans la suite. La valeur X_n est l'état du processus à l'instant n. Les applications où l'espace d'états E est

Rabi oscillation in a quantum cavity: Markovian and non-Markovian dynamics

Pierre-Olivier Guimond, Alexandre Roulet

We investigate the Rabi oscillation of an atom placed inside a quantum cavity where each mirror is formed by a chain of atoms trapped near a one-dimensional waveguide. This proposal was studied previously with the use of Markov approximation, where the delay due to the finite travel time of light between the two cavity mirrors is neglected. We show that Rabi oscillation analogous to that obtained with high-finesse classical cavities is achieved only when this travel time is much larger than the time scale that characterizes the superradiant response of the mirrors. Therefore, the delay must be taken into account and the dynamics of the problem is inherently non-Markovian. Parameters of interest such as the Rabi frequency and the cavity loss rate due to photon leakage through the mirrors are obtained.

Amer Physical Soc2016

Statistical Analysis of Chao’s Immune System Simulation

Eric Winnington

Modeling the immune system (IS) means putting together a set of assumptions about its components (cells and organs) and their interactions. Simulations of a model show joint behavior of the components, which for complex realistic models is often impossible to find analytically. Simulations allow us to experiment on how initial concentrations and properties of the immune cells and viruses impact the IS behavior, and gain better quantitative and qualitative insight into how the IS works and why different behavior patterns occur. A simulation, once it has been created, must be reviewed both statistically and analytically as well as validated from the biological point of view. We analyzed Chao’s immune system simulation [1][2] from a statistical and analytical point. We explicited both the Markov chain which was simulated and the underlying process on which Chao’s stage-structured approach was built. Furthermore, we established a test protocol for timestep validation which Chao’s simulator passed. We evaluated Chao’s simulator’s dependence on the random number generator, which was shown to be negligible. Finally, we evaluated the simulator output and our major result is the discovery of a secondary response to a primary infection, an occurrence is not shown in Chao’s dissertation. A tertiary response to the infection is never possible due to the size of the secondary response caused by memory cells.

2005

HIERARCHICAL MARKOV CHAIN MONTE CARLO METHODS FOR BAYESIAN INVERSE PROBLEMS

Juan Pablo Madrigal Cianci

This thesis is devoted to the construction, analysis, and implementation of two types of hierarchical Markov Chain Monte Carlo (MCMC) methods for the solution of large-scale Bayesian Inverse Problems (BIP).The first hierarchical method we present is based on the idea of parallel tempering and is well-suited for BIP whose underlying posterior measure is multi-modal or concentrates around a lower-dimensional, non-linear manifold. In particular, we present two generalizations of the Parallel Tempering algorithm in the context of discrete-time Markov chain Monte Carlo methods for Bayesian inverse problems. These generalizations use state-dependent swapping rates and are inspired by the so-called continuous-time Infinite Swapping algorithm presented in Plattner et al. [J Chem Phys 135(13):134111, 2011]. We present a thorough analysis of the convergence of our proposed methods and show that they are reversible and geometrically ergodic. Numerical experiments conducted over an array of BIP show that our proposed algorithms significantly improve sampling efficiency over competing methodologies. Our second hierarchical method is based on multi-level MCMC (ML-MCMC) techniques. In this setting, instead of sampling directly from a sufficiently accurate (and computationally expensive) posterior measure, one introduces a sequence of accuracy levels for the solution of the underlying computational model, which induces a hierarchy of posterior measures with increasing accuracy and cost to sample from. The key point of this algorithm is to construct highly coupled Markov chains together with the standard Multi-level Monte Carlo argument to obtain a better cost-tolerance complexity than a single-level MCMC algorithm. We present two types of multi-level MCMC algorithms which can be thought of as an extension of the ideas presented in Dodwell, et al. [SIAM-ASA J. Uncertain. Quantif (2015): 1075-1108]. Our first ML-MCMC method extends said ideas to a setting where a wider class of Independent Metropolis-Hastings (IMH) proposals are considered. We provide a thorough theoretical analysis and provide sufficient conditions on the proposals and the family of posteriors so that there exists a unique invariant probability measure for the coupled chains generated by our method, and the convergence to it is uniformly ergodic. We also generalize the cost-tolerance theorem of Dodwell et al., to our setting, and propose a self-tuning continuation-type ML-MCMC algorithm. Our second ML-MCMC method presents an algorithm that admits state-dependent proposals by using a maximal coupling approach. This is desirable, from a methodological perspective, whenever it is difficult to construct suitable IMH proposals, or when the empirical measure resulting from samples from the posterior at the previous level does not satisfy the assumptions required for convergence of the ML-MCMC method. We present a theoretical analysis of the method at hand and show that this new method has an invariant probability measure and converges to it with geometric ergodicity. We also extend the cost-tolerance theorem of Dodwell et. al. to this algorithm, albeit with quite restrictive assumptions. We illustrate both of the proposed ML-MCMC methodologies on several numerical examples.

EPFL2022

HIERARCHICAL MARKOV CHAIN MONTE CARLO METHODS FOR BAYESIAN INVERSE PROBLEMS

Juan Pablo Madrigal Cianci

EPFL2022