Stochastic control

Stochastic control or stochastic optimal control is a sub field of control theory that deals with the existence of uncertainty either in observations or in the noise that drives the evolution of the system. The system designer assumes, in a Bayesian probability-driven fashion, that random noise with known probability distribution affects the evolution and observation of the state variables. Stochastic control aims to design the time path of the controlled variables that performs the desired control task with minimum cost, somehow defined, despite the presence of this noise. The context may be either discrete time or continuous time. Certainty equivalence An extremely well-studied formulation in stochastic control is that of linear quadratic Gaussian control. Here the model is linear, the objective function is the expected value of a quadratic form, and the disturbances are purely additive. A basic result for discrete-time centralized systems with only additive uncertainty

Stochastic Control and Free Boundary Problems for Sailboat Trajectory Optimization

Laura Vinckenbosch

The topic of this thesis is the study of several stochastic control problems motivated by sailing races. The goal is to minimize the travel time between two locations, by selecting the fastest route in face of randomly changing weather conditions, such as wind direction. When a sailboat is travelling upwind, the key is to decide when to tack. Since this maneuver slows down the yacht, it is natural to model this time lost by a "tacking penalty" which places the problem in the context of optimal stochastic control problems with switching costs. An objective of this work is to propose and to study mathematical models that capture some of the features of a sailing race, but which remain amenable to an explicit rigorous solution that can be proved to be optimal. We consider three different models in which the wind direction is described by a stochastic process. In the first model, we consider a wind that changes randomly only once. In the second model, the wind oscillates between two possible directions according to a continuous-time Markov chain. We exhibit a free boundary problem for the value function involving hyperbolic partial differential equations of Klein-Gordon type. The last model considers the wind direction as a Brownian motion. We prove the existence of a finite value function and exhibit a free boundary problem involving parabolic partial differential equations with non-constant coefficients. In these three models, the optimal solution consists of a partition of the state space into a region where it is optimal to tack immediately and a region where it is optimal to continue on the current tack. The boundaries between these regions are given by one or more "switching curves" and in the cases where we have been able to exhibit them, the optimality of the solution is established by a verification theorem based on the martingale method. We also solve two other control problems in which a player tries to minimize or maximize the exit time from an interval of a Brownian particle by controlling its drift and subject to a switching penalty. In each problem, the value function is written as the solution of a second order ordinary differential equations problem whose unknown boundaries are found by applying the principle of smooth fit. For both problems, we exhibit a candidate strategy as a function of the switching cost and we prove its optimality as well as its generic uniqueness.

EPFL2012

Optimal expulsion and optimal confinement of a Brownian particle with a switching cost

Robert Dalang, Laura Vinckenbosch

We solve two stochastic control problems in which a player tries to minimize or maximize the exit time from an interval of a Brownian particle, by controlling its drift. The player can change from one drift to another but is subject to a switching cost. In each problem, the value function is written as the solution of a free boundary problem involving second order ordinary differential equations, in which the unknown boundaries are found by applying the principle of smooth fit. For both problems, we compute the value function, we exhibit the optimal strategy and we prove its generic uniqueness. (C) 2014 Elsevier B.V. All rights reserved.

Elsevier Science Bv2014

Stochastic Control and Free Boundary Problems for Sailboat Trajectory Optimization

Laura Vinckenbosch

EPFL2012