Publication

On the Double Descent of Random Features Models Trained with SGD

Related publications (36)

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Mirrored Langevin Dynamics

Volkan Cevher, Paul Thierry Yves Rolland, Ya-Ping Hsieh, Ali Kavis

We consider the problem of sampling from constrained distributions, which has posed significant challenges to both non-asymptotic analysis and algorithmic design. We propose a unified framework, which is inspired by the classical mirror descent, to derive ...

2018

, , ,

In empirical risk optimization, it has been observed that gradient descent implementations that rely on random reshuffling of the data achieve better performance than implementations that rely on sampling the data randomly and independently of each other. ...

2017

Safe Adaptive Importance Sampling

Martin Jaggi, Sebastian Urban Stich, Anant Raj

Importance sampling has become an indispensable strategy to speed up optimization algorithms for large-scale applications. Improved adaptive variants - using importance values defined by the complete gradient information which changes during optimization - ...

2017

Sampling at unknown locations, with an application in surface retrieval

Martin Vetterli, Benjamin Bejar Haro, Adam James Scholefield, Michalina Wanda Pacholska

We consider the problem of sampling at unknown locations. We prove that, in this setting, if we take arbitrarily many samples of a polynomial or real bandlimited signal, it is possible to find another function in the same class, arbitrarily far away from t ...

2017

Approximate Steepest Coordinate Descent

Martin Jaggi, Sebastian Urban Stich, Anant Raj

We propose a new selection rule for the coordinate selection in coordinate descent methods for huge-scale optimization. The efficiency of this novel scheme is provably better than the efficiency of uniformly random selection, and can reach the efficiency o ...

PMLR2017

Stochastic gradient descent with finite samples sizes

Ali H. Sayed, Stefan Vlaski, Bicheng Ying, Kun Yuan

The minimization of empirical risks over finite sample sizes is an important problem in large-scale machine learning. A variety of algorithms has been proposed in the literature to alleviate the computational burden per iteration at the expense of converge ...

IEEE2016

Attributing Aliasing Artifacts in Dirty Intensity Fields to their Parent Source

Matthieu Martin Jean-André Simeoni

Estimating intensity fields of stochastic phenomenons is of crucial interest in many scientific applications. Typical experimental setups involve an acquisition system, that subsequently filters and samples the probed intensity field. This equivalently def ...

2016

Theory of representation learning in cortical neural networks

Carlos Stein Naves de Brito

Our brain continuously self-organizes to construct and maintain an internal representation of the world based on the information arriving through sensory stimuli. Remarkably, cortical areas related to different sensory modalities appear to share the same f ...

EPFL2016

Compressive Spectral Clustering

Pierre Vandergheynst, Rémi Gribonval, Gilles Puy, Nicolas Tremblay

Spectral clustering has become a popular technique due to its high performance in many contexts. It comprises three main steps: create a similarity graph between N objects to cluster, compute the first k eigenvectors of its Laplacian matrix to define a fea ...

2016

Restricted Boltzmann Machines (RBMs) are widely used as building blocks for deep learning models. Learning typically proceeds by using stochastic gradient descent, and the gradients are estimated with sampling methods. However, the gradient estimation is a ...

2015