**Are you an EPFL student looking for a semester project?**

Work with us on data science and visualisation projects, and deploy your project as an app on top of GraphSearch.

Publication# A time-domain feedback analysis of adaptive gradient algorithms via the small gain theorem

Abstract

This paper provides a time-domain feedback analysis of gradient-based adaptive schemes with emphasis on stability and robustness issues. It is shown that an intrinsic feedback structure, mapping the noise sequence and the initial weight guess to the a priori estimation errors and the final weight estimate, can be associated with such schemes. The feedback configuration is motivated via energy arguments and is shown to consist of two major blocks: a time-variant lossless (i.e., energy preserving) feedforward path and a time-variant feedback path. The configuration is further shown to lend itself rather immediately to analysis via a so-called small gain theorem; thus leading to stability conditions that require the contractivity of certain operators.

Official source

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related concepts (32)

Related publications (39)

Scheme (mathematics)

In mathematics, a scheme is a mathematical structure that enlarges the notion of algebraic variety in several ways, such as taking account of multiplicities (the equations x = 0 and x2 = 0 define the same algebraic variety but different schemes) and allowing "varieties" defined over any commutative ring (for example, Fermat curves are defined over the integers). Scheme theory was introduced by Alexander Grothendieck in 1960 in his treatise "Éléments de géométrie algébrique"; one of its aims was developing the formalism needed to solve deep problems of algebraic geometry, such as the Weil conjectures (the last of which was proved by Pierre Deligne).

Hilbert scheme

In algebraic geometry, a branch of mathematics, a Hilbert scheme is a scheme that is the parameter space for the closed subschemes of some projective space (or a more general projective scheme), refining the Chow variety. The Hilbert scheme is a disjoint union of projective subschemes corresponding to Hilbert polynomials. The basic theory of Hilbert schemes was developed by . Hironaka's example shows that non-projective varieties need not have Hilbert schemes.

Group scheme

In mathematics, a group scheme is a type of object from algebraic geometry equipped with a composition law. Group schemes arise naturally as symmetries of schemes, and they generalize algebraic groups, in the sense that all algebraic groups have group scheme structure, but group schemes are not necessarily connected, smooth, or defined over a field. This extra generality allows one to study richer infinitesimal structures, and this can help one to understand and answer questions of arithmetic significance.

Sebastian Urban Stich, Konstantin Mishchenko

We consider distributed optimization over several devices, each sending incremental model updates to a central server. This setting is considered, for instance, in federated learning. Various schemes have been designed to compress the model updates in orde ...

Fabio Nobile, Yoshihito Kazashi, Eva Vidlicková

We consider the Dynamical Low Rank (DLR) approximation of random parabolic equations and propose a class of fully discrete numerical schemes. Similarly to the continuous DLR approximation, our schemes are shown to satisfy a discrete variational formulation ...

Nikolaos Geroliminis, Raphael Ali Francis Lamotte

Several works published over the last two decades have shown for a stylized set-up with homogeneous users that metering-based priority (MBP) schemes may generate Pareto improving departure time adjustments similar to those induced by congestion pricing, bu ...