Sample Complexity of Linear Quadratic Gaussian (LQG) Control for Output Feedback Systems

This paper studies a class of partially observed Linear Quadratic Gaussian (LQG) problems with unknown dynamics. We establish an end-to-end sample complexity bound on learning a robust LQG controller for open-loop stable plants. This is achieved using a robust synthesis procedure, where we first estimate a model from a single input-output trajectory of finite length, identify an H-infinity bound on the estimation error, and then design a robust controller using the estimated model and its quantified uncertainty. Our synthesis procedure leverages a recent control tool called Input-Output Parameterization (IOP) that enables robust controller design using convex optimization. For open-loop stable systems, we prove that the LQG performance degrades linearly with respect to the model estimation error using the proposed synthesis procedure. Despite the hidden states in the LQG problem, the achieved scaling matches previous results on learning Linear Quadratic Regulator (LQR) controllers with full state observations.

Robust Adaptive Decision Making: Bayesian Optimization and Beyond

Ilija Bogunovic

The central task in many interactive machine learning systems can be formalized as the sequential optimization of a black-box function. Bayesian optimization (BO) is a powerful model-based framework for \emph{adaptive} experimentation, where the primary goal is the optimization of the black-box function via sequentially chosen decisions. In many real-world tasks, it is essential for the decisions to be \emph{robust} against, e.g., adversarial failures and perturbations, dynamic and time-varying phenomena, a mismatch between simulations and reality, etc. Under such requirements, the standard methods and BO algorithms become inadequate. In this dissertation, we consider four research directions with the goal of enhancing robust and adaptive decision making in BO and associated problems. First, we study the related problem of level-set estimation (LSE) with Gaussian Processes (GPs). While in BO the goal is to find a maximizer of the unknown function, in LSE one seeks to find all "sufficiently good" solutions. We propose an efficient confidence-bound based algorithm that treats BO and LSE in a unified fashion. It is effective in settings that are non-trivial to incorporate into existing algorithms, including cases with pointwise costs, heteroscedastic noise, and multi-fidelity setting. Our main result is a general regret guarantee that covers these aspects. Next, we consider GP optimization with robustness requirement: An adversary may perturb the returned design, and so we seek to find a robust maximizer in the case this occurs. This requirement is motivated by, e.g., settings where the functions during optimization and implementation stages are different. We propose a novel robust confidence-bound based algorithm. The rigorous regret guarantees for this algorithm are established and complemented with an algorithm-independent lower bound. We experimentally demonstrate that our robust approach consistently succeeds in finding a robust maximizer while standard BO methods fail. We then investigate the problem of GP optimization in which the reward function varies with time. The setting is motivated by many practical applications in which the function to be optimized is not static. We model the unknown reward function via a GP whose evolution obeys a simple Markov model. Two confidence-bound based algorithms with the ability to "forget" about old data are proposed. We obtain regret bounds for these algorithms that jointly depend on the time horizon and the rate at which the function varies. Finally, we consider the maximization of a set function subject to a cardinality constraint

k

in the case a number of items

\tau

from the returned set may be removed. One notable application is in batch BO where we need to select experiments to run, but some of them can fail. Our focus is on the worst-case adversarial setting, and we consider both \emph{submodular} (i.e., satisfies a natural notion of diminishing returns) and \emph{non-submodular} objectives. We propose robust algorithms that achieve constant-factor approximation guarantees. In the submodular case, the result on the maximum number of allowed removals is improved to

\tau = o(k)

in comparison to the previously known

\tau=o(\sqrt{k})

. In the non-submodular case, we obtain new guarantees in the support selection and batch BO tasks. We empirically demonstrate the robust performance of our algorithms in these, as well as, in data summarization and influence maximization tasks.

EPFL2019

Robust gain-scheduled blending control of raw-mix quality in cement industries

Michael Amrhein, Vinicius De Oliveira, Alireza Karimi

This paper addresses the control of the blending process in cement industries. This process can be modeled by a nonlinear multivariable system with large parametric uncertainty. Using a specific transformation, a linear parameter varying (LPV) model with set-points as scheduling parameters is developed. Moreover, the model uncertainty originated from the stochastic variation of the composition of the input materials is represented as a polytopic multimodel uncertainty. Then a multivariable gain-scheduled robust controller is designed by convex optimization to control the quality of the raw mix in the blending process. The control performance is illustrated by simulation and compared with a robust controller based on a nominal model.

Ieee Service Center, 445 Hoes Lane, Po Box 1331, Piscataway, Nj 08855-1331 Usa2011

A Data-driven Approach to Robust Control of Multivariable Systems by Convex Optimization

Christoph Manuel Kammer, Alireza Karimi

The frequency-domain data of a multivariable system in different operating points is used to design a robust controller with respect to the measurement noise and multimodel uncertainty. The controller is fully parametrized in terms of matrix polynomial functions and can be formulated as a centralized, decentralized or distributed controller. All standard performance specifications like H2, H∞ and loop shaping are considered in a unified framework for continuous- and discrete-time systems. The control problem is formulated as a convex-concave optimization problem and then convexified by linearization of the concave part around an initial controller. The performance criterion converges monotonically to a local optimum or a saddle point in an iterative algorithm. The effectiveness of the method is compared with fixed-structure controller design methods based on non-smooth optimization via multiple simulation examples.

2017

Robust Adaptive Decision Making: Bayesian Optimization and Beyond

Ilija Bogunovic

k

in the case a number of items

\tau

\tau = o(k)

in comparison to the previously known

\tau=o(\sqrt{k})

EPFL2019