Continuation Methods For Riemannian Optimization

Numerical continuation in the context of optimization can be used to mitigate convergence issues due to a poor initial guess. In this work, we extend this idea to Riemannian optimization problems, that is, the minimization of a target function on a Riemannian manifold. For this purpose, a suitable homotopy is constructed between the original problem and a problem that admits an easy solution. We develop and analyze a path-following numerical continuation algorithm on manifolds for solving the resulting parameter-dependent equation. To illustrate our developments, we consider two typical classical applications of Riemannian optimization: the computation of the Karcher mean and low-rank matrix completion. We demonstrate that numerical continuation can yield improvements for challenging instances of both problems.

Optimal control of geodesics in Riemannian manifolds

Roland Rozsnyo

The aim of this dissertation is to solve numerically the following problem, denoted by P : given a Riemannian manifold and two points a and b belonging to that manifold, find a tangent vector T at a, such that expa(T) = b, assuming that T exists. This problem is set under an optimal control formulation, which requires the definition of an objective function and a space of control, the choice of a method for the calculation of the descent direction of that function in the space of control and the use of an optimization algorithm to find its minimum, which corresponds to the solution of the original problem by construction. Several techniques are necessary to be put together, coming from the fields of geometry, numerical analysis and optimization. The first part will concern a recalling of the mathematical context in which this formulation takes place. The general principles of optimal control will also be given. In the second part, we will present an intrinsic formulation of the optimal control problem associated to P, based on Jacobi fields, which will play the role of the so called adjoint state. This derivation leads to necessary optimality conditions. We will illustrate explicitly that formulation by treating the specific case of Riemannian manifolds with constant sectional curvature. Then, we will derive the optimal control problem in coordinates, not only to check the intrinsic formulation but also to reveal how it is hidden behind the expressions in coordinates. Their use reveals some quantities whose interpretation may be given this way. Moreover, we will show that more possibilities exist to chose the cost function and the control space in coordinates. In a second step, an alternative approach will consider the Hamiltonian formulation of geodesics. This is an incursion into symplectic geometry. We will then reformulate the Riemannian optimal control problem in its Hamiltonian version. In the third part, the numerical methods used for solving P will be presented. The discretization imposes the definition of new discrete optimal control problems. The technique shows that the discrete adjoint state equation strongly depends on the numerical scheme used to solve the direct problem. We will give a collection of numerical computations in the specific case of parametric piece of surfaces, where the surface can be defined by one or several Bézier patches, each one corresponding to a chart, which is representative of a Riemannian manifold. We will compare the different numerical approaches. The last but one part will be devoted to the interesting application of wooden roof building, where the structure is made of wooden boards, with geodesic trajectories on the designed piece of surface. The Geos (Geodesic solver) software has been developed for that purpose. After having introduced some specific numerical methods used in the code, we present the Geos application interface (AI) developed as a tool for the conception of such a roof. We then show an existing wooden structure built according to that mean. Finally, we will summarize the results of our research and discuss future possible prospects.

EPFL2007

A Dual Modifier-Adaptation Approach for Real-Time Optimization

Dominique Bonvin, Benoît Chachuat, Alejandro Gabriel Marchetti

For good performance in practice, real-time optimization schemes need to be able to deal with the inevitable plant-model mismatch problem. Unlike the two-step schemes combining parameter estimation and optimization, the modifier-adaptation approach does not require the model parameters to be estimated on-line. Instead, it uses information regarding the constraints and selected gradients to improve the plant operation. The dual modifier-adaptation approach presented in this paper drives the process towards optimality, while paying attention to the accuracy of the estimated gradients. The gradients are estimated from successive operating points generated by the optimization algorithm. The novelty lies in the development of an upper bound on the norm of the gradient errors, which is used as a constraint when determining the next operating point. The proposed approach is demonstrated via numerical simulation for both an unconstrained and a constrained problem.

Elsevier2010

Low-Rank Matrix Completion By Riemannian Optimization

The matrix completion problem consists of finding or approximating a low-rank matrix based on a few samples of this matrix. We propose a new algorithm for matrix completion that minimizes the least-square distance on the sampling set over the Riemannian manifold of fixed-rank matrices. The algorithm is an adaptation of classical nonlinear conjugate gradients, developed within the framework of retraction-based optimization on manifolds. We describe all the necessary objects from differential geometry necessary to perform optimization over this low-rank matrix manifold, seen as a submanifold embedded in the space of matrices. In particular, we describe how metric projection can be used as retraction and how vector transport lets us obtain the conjugate search directions. Finally, we prove convergence of a regularized version of our algorithm under the assumption that the restricted isometry property holds for incoherent matrices throughout the iterations. The numerical experiments indicate that our approach scales very well for large-scale problems and compares favorably with the state-of-the-art, while outperforming most existing solvers.

Siam Publications2013

Optimal control of geodesics in Riemannian manifolds

Roland Rozsnyo

EPFL2007