Processus de décision markovien partiellement observable

Résumé

En théorie de la décision et de la théorie des probabilités, un processus de décision markovien partiellement observable (POMDP pour partially observable Markov decision process) est une généralisation d'un processus de décision markoviens (MDP pour Markov decision process). Comme dans un MDP, l'effet des actions est incertain mais, contrairement à ce qui se passe pour un MDP, l'agent n'a qu'une information partielle de l'état courant. Les POMDP sont des modèles de Markov cachés (HMM pour hidden Markov model) particuliers, dans lesquels on dispose d'actions probabilistes. Le tableau suivant montre la place des POMDP dans la famille des processus de décision : Les modèles de cette famille sont, entre autres, utilisés en intelligence artificielle pour le contrôle de systèmes complexes comme des agents intelligents. Définition formelle Un POMDP est un tuple { S, A, T, R, \Omega, O }, où :

S est un ensemble fini non vide des états possibles

Source officielle

https://fr.wikipedia.org/wiki/Processus_de_d%C3%A9cision_markovien_partiellement_observable

À propos de ce résultat

Cette page est générée automatiquement et peut contenir des informations qui ne sont pas correctes, complètes, à jour ou pertinentes par rapport à votre recherche. Il en va de même pour toutes les autres pages de ce site. Veillez à vérifier les informations auprès des sources officielles de l'EPFL.

Publications associées (25)

Publications associées

Chargement

Personnes associées (3)

, ,

Personnes associées

Chargement

Unités associées (1)

Unités associées

Chargement

Concepts associés (1)

Concepts associés

Chargement

Cours associés (6)

Cours associés

Chargement

Séances de cours associées (9)

Séances de cours associées

Chargement

Measurement Scheduling for Soil Moisture Sensing: From Physical Models to Optimal Control

Ke Li, Aditya Mahajan, David Shuman

In this paper, we consider the problem of monitoring soil moisture evolution using a wireless network of in situ sensors. Continuously sampling moisture levels with these sensors incurs high-maintenance and energy consumption costs, which are particularly undesirable for wireless networks. Our main hypothesis is that a sparser set of measurements can meet the monitoring objectives in an energy-efficient manner. The underlying idea is that we can trade off some inaccuracy in estimating soil moisture evolution for a significant reduction in energy consumption. We investigate how to dynamically schedule the sensor measurements so as to balance this tradeoff. Unlike many prior studies on sensor scheduling that make generic assumptions on the statistics of the observed phenomenon, we obtain statistics of soil moisture evolution from a physical model. We formulate the optimal measurement scheduling and estimation problem as a partially observable Markov decision problem (POMDP). We then utilize special features of the problem to approximate the POMDP by a computationally simpler finite-state Markov decision problem (MDP). The result is a scalable, implementable technology that we have tested and validated numerically and in the field.

2010

Chargement

, ,

Finding sources of airborne chemicals with mobile sensing systems finds applications across safety, security, environmental monitoring, and medical domains. In this paper, we present an algorithm based on Source Term Estimation for odor source localization that is coupled with a navigation method based on partially observable Markov decision processes. We propose a novel strategy to balance exploration and exploitation in navigation. Moreover, we study two variants of the algorithm, one exploiting a global and the other one a local framework. The method was evaluated through high-fidelity simulations and in a wind tunnel emulating a quasi-laminar air flow in a controlled environment, in particular by systematically investigating the impact of multiple algorithmic and environmental parameters (wind speed and source release rate) on the overall performance. The outcome of the experiments showed that the algorithm is robust to different environmental conditions in the global framework, but, in the local framework, it is only successful in relatively high wind speeds. In the local framework, on the other hand, the algorithm is less demanding in terms of energy consumption as it does not require any absolute positioning information from the environment and the robot travels less distance compared to the global framework.

2019

Chargement

An Algorithm for Odor Source Localization based on Source Term Estimation

Ali Marjovi, Alcherio Martinoli, Faezeh Rahbar

Finding sources of airborne chemicals with mobile sensing systems finds applications across the security, safety, domestic, medical, and environmental domains. In this paper, we present an algorithm based on source term estimation for odor source localization that is coupled with a navigation method based on partially observable Markov decision processes. We propose an innovative strategy to balance exploration and exploitation in navigation. The method has been evaluated systematically through high-fidelity simulations and in a wind tunnel emulating realistic and repeatable conditions. The impact of multiple algorithmic and environmental parameters has been studied in the experiments.

2019

Chargement