Hamilton–Jacobi–Bellman equation

Résumé

The Hamilton-Jacobi-Bellman (HJB) equation is a nonlinear partial differential equation that provides necessary and sufficient conditions for optimality of a control with respect to a loss function. Its solution is the value function of the optimal control problem which, once known, can be used to obtain the optimal control by taking the maximizer (or minimizer) of the Hamiltonian involved in the HJB equation. The equation is a result of the theory of dynamic programming which was pioneered in the 1950s by Richard Bellman and coworkers. The connection to the Hamilton–Jacobi equation from classical physics was first drawn by Rudolf Kálmán. In discrete-time problems, the analogous difference equation is usually referred to as the Bellman equation. While classical variational problems, such as the brachistochrone problem, can be solved using the Hamilton–Jacobi–Bellman equation, the method can be applied to a broader spectrum of problems. Further it can be generalized to stochastic sys

Source officielle

https://en.wikipedia.org/wiki/Hamilton%E2%80%93Jacobi%E2%80%93Bellman_equation

À propos de ce résultat

Cette page est générée automatiquement et peut contenir des informations qui ne sont pas correctes, complètes, à jour ou pertinentes par rapport à votre recherche. Il en va de même pour toutes les autres pages de ce site. Veillez à vérifier les informations auprès des sources officielles de l'EPFL.

Publications associées (5)

Publications associées

Chargement

Personnes associées

Chargement

Unités associées

Chargement

Concepts associés (6)

Concepts associés

Chargement

Cours associés (3)

Cours associés

Chargement

Séances de cours associées (6)

Séances de cours associées

Chargement

Symmetry Reduction Of Brownian Motion And Quantum Calogero-Moser Models

Simon Hochgerner

Let Q be a Riemannian G-manifold. This paper is concerned with the symmetry reduction of Brownian motion in Q and ramifications thereof in a Hamiltonian context. Specializing to the case of polar actions, we discuss various versions of the stochastic Hamilton-Jacobi equation associated to the symmetry reduction of Brownian motion and observe some similarities to the Schrodinger equation of the quantum-free particle reduction as described by Feher and Pusztai [10]. As an application we use this reduction scheme to derive examples of quantum Calogero-Moser systems from a stochastic setting.

World Scientific Publ Co Pte Ltd2013

Chargement

We study the convergence of Markov Decision Processes made of a large number of objects to optimization problems on ordinary differential equations (ODE). We show that the optimal reward of such a Markov Decision Process, satisfying a Bellman equation, converges to the solution of a continuous Hamilton-Jacobi-Bellman (HJB) equation based on the mean field approximation of the Markov Decision Process. We give bounds on the difference of the rewards, and a constructive algorithm for deriving an approximating solution to the Markov Decision Process from a solution of the HJB equations. We illustrate the method on three examples pertaining respectively to investment strategies, population dynamics control and scheduling in queues are developed. They are used to illustrate and justify the construction of the controlled ODE and to show the gain obtained by solving a continuous HJB equation rather than a large discrete Bellman equation.

2010

Chargement

Mean Field for Markov Decision Processes: from Discrete to Continuous Optimization

Nicolas Gabriel Gast, Jean-Yves Le Boudec

We study the convergence of Markov decision processes, composed of a large number of objects, to optimization problems on ordinary differential equations. We show that the optimal reward of such a Markov decision process, which satisfies a Bellman equation, converges to the solution of a continuous Hamilton-Jacobi-Bellman (HJB) equation based on the mean field approximation of the Markov decision process. We give bounds on the difference of the rewards and an algorithm for deriving an approximating solution to the Markov decision process from a solution of the HJB equations. We illustrate the method on three examples pertaining, respectively, to investment strategies, population dynamics control and scheduling in queues. They are used to illustrate and justify the construction of the controlled ODE and to show the advantage of solving a continuous HJB equation rather than a large discrete Bellman equation.

Institute of Electrical and Electronics Engineers2012

Chargement