Cover: A Cluster-Based Variance Reduced Method For Online Learning

In this paper, we develop a stochastic-gradient learning algorithm for situations involving streaming data that arise from an underlying clustered structure. In such settings, the variance of gradient noise can be decomposed into the in-cluster variance sigma(2)(in) in plus the between-cluster variance sigma(2)(bet). We develop a cluster-based online variance-reduced method (COVER) to eliminate sigma(2)(bet) and improve the MSD performance of stochastic- gradient descent (SGD) to the order of O(sigma(2)(in)). We establish the convergence property of COVER and derive a tight closed-form mean-square deviation (MSD) performance expression. Our simulations illustrate the improved performance of COVER in terms of steady- state performance.

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Cover: A Cluster-Based Variance Reduced Method For Online Learning

Graph Chatbot

Chat with Graph Search

Imitation Learning in Discounted Linear MDPs without exploration assumptions

On the Double Descent of Random Features Models Trained with SGD

Emergency adaptation to distance learning: no magic bullet, but where there’s a will, there’s a way

Imitation Learning in Discounted Linear MDPs without exploration assumptions

Emergency adaptation to distance learning: no magic bullet, but where there’s a will, there’s a way

On the Double Descent of Random Features Models Trained with SGD