Hitting time | EPFL Graph Search

In the study of stochastic processes in mathematics, a hitting time (or first hit time) is the first time at which a given process "hits" a given subset of the state space. Exit times and return times are also examples of hitting times. Definitions Let T be an ordered index set such as the natural numbers, \N, the non-negative real numbers, [0, +∞), or a subset of these; elements t \in T can be thought of as "times". Given a probability space (Ω, Σ, Pr) and a measurable state space S, let X :\Omega \times T \to S be a stochastic process, and let A be a measurable subset of the state space S. Then the first hit time \tau_A : \Omega \to [0, +\infty] is the random variable defined by :\tau_A (\omega) := \inf { t \in T \mid X_t (\omega) \in A }. The first exit time (from A) is defined to be the first hit time for S \ A, the complement of A in S.

Stochastic Control and Free Boundary Problems for Sailboat Trajectory Optimization

Laura Vinckenbosch

The topic of this thesis is the study of several stochastic control problems motivated by sailing races. The goal is to minimize the travel time between two locations, by selecting the fastest route in face of randomly changing weather conditions, such as wind direction. When a sailboat is travelling upwind, the key is to decide when to tack. Since this maneuver slows down the yacht, it is natural to model this time lost by a "tacking penalty" which places the problem in the context of optimal stochastic control problems with switching costs. An objective of this work is to propose and to study mathematical models that capture some of the features of a sailing race, but which remain amenable to an explicit rigorous solution that can be proved to be optimal. We consider three different models in which the wind direction is described by a stochastic process. In the first model, we consider a wind that changes randomly only once. In the second model, the wind oscillates between two possible directions according to a continuous-time Markov chain. We exhibit a free boundary problem for the value function involving hyperbolic partial differential equations of Klein-Gordon type. The last model considers the wind direction as a Brownian motion. We prove the existence of a finite value function and exhibit a free boundary problem involving parabolic partial differential equations with non-constant coefficients. In these three models, the optimal solution consists of a partition of the state space into a region where it is optimal to tack immediately and a region where it is optimal to continue on the current tack. The boundaries between these regions are given by one or more "switching curves" and in the cases where we have been able to exhibit them, the optimality of the solution is established by a verification theorem based on the martingale method. We also solve two other control problems in which a player tries to minimize or maximize the exit time from an interval of a Brownian particle by controlling its drift and subject to a switching penalty. In each problem, the value function is written as the solution of a second order ordinary differential equations problem whose unknown boundaries are found by applying the principle of smooth fit. For both problems, we exhibit a candidate strategy as a function of the switching cost and we prove its optimality as well as its generic uniqueness.

EPFL2012

Optimal expulsion and optimal confinement of a Brownian particle with a switching cost

Robert Dalang, Laura Vinckenbosch

We solve two stochastic control problems in which a player tries to minimize or maximize the exit time from an interval of a Brownian particle, by controlling its drift. The player can change from one drift to another but is subject to a switching cost. In each problem, the value function is written as the solution of a free boundary problem involving second order ordinary differential equations, in which the unknown boundaries are found by applying the principle of smooth fit. For both problems, we compute the value function, we exhibit the optimal strategy and we prove its generic uniqueness. (C) 2014 Elsevier B.V. All rights reserved.

Elsevier Science Bv2014

Stochastic Control and Free Boundary Problems for Sailboat Trajectory Optimization

Laura Vinckenbosch

EPFL2012