PDE-constrained optimal control problems with uncertain parameters using SAGA

We consider an optimal control problem for an elliptic partial differential equation (PDE) with random coefficients. The control function is a deterministic, distributed forcing term that minimizes an expected quadratic regularized loss functional. We consider a Finite Element discretization of the underlying PDEs and a Gaussian-type quadrature formula to approximate the expected loss. For the computation of the approximate optimal control, we propose a generalization of the SAGA algorithm [Defazio-Bach-Lacoste Julien, 2014], a type of Stochastic Gradient algorithm with a fixed-length memory term, which computes at each iteration the gradient of the loss functional in only one quadrature point, randomly chosen from a possibly non uniform distribution. We provide a full error and complexity analysis of the proposed numerical scheme. In particular we compare the complexity of the generalized SAGA algorithm, with that of the Stochastic Gradient (SG) and the Full Gradient (FG) algorithms, applied to the same discretized optimal control problem. We show that SAGA converges exponentially in the number of iterations as for a FG algorithm and has a similar asymptotic computational complexity. Moreover, it features good pre asymptotic properties, as shown by our numerical experiments, which makes it appealing in a limited budget context.