Publication

On the Double Descent of Random Features Models Trained with SGD

Publications associées (36)

Graph Chatbot

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

Connectez-vous pour utiliser Chat avec Graph Search

Mirrored Langevin Dynamics

Volkan Cevher, Paul Thierry Yves Rolland, Ya-Ping Hsieh, Ali Kavis

We consider the problem of sampling from constrained distributions, which has posed significant challenges to both non-asymptotic analysis and algorithmic design. We propose a unified framework, which is inspired by the classical mirror descent, to derive ...

2018

On the performance of random reshuffling in stochastic learning

Ali H. Sayed, Stefan Vlaski, Bicheng Ying, Kun Yuan

In empirical risk optimization, it has been observed that gradient descent implementations that rely on random reshuffling of the data achieve better performance than implementations that rely on sampling the data randomly and independently of each other. ...

2017

Safe Adaptive Importance Sampling

Martin Jaggi, Sebastian Urban Stich, Anant Raj

Importance sampling has become an indispensable strategy to speed up optimization algorithms for large-scale applications. Improved adaptive variants - using importance values defined by the complete gradient information which changes during optimization - ...

2017

Sampling at unknown locations, with an application in surface retrieval

Martin Vetterli, Benjamin Bejar Haro, Adam James Scholefield, Michalina Wanda Pacholska

We consider the problem of sampling at unknown locations. We prove that, in this setting, if we take arbitrarily many samples of a polynomial or real bandlimited signal, it is possible to find another function in the same class, arbitrarily far away from t ...

2017

Approximate Steepest Coordinate Descent

Martin Jaggi, Sebastian Urban Stich, Anant Raj

We propose a new selection rule for the coordinate selection in coordinate descent methods for huge-scale optimization. The efficiency of this novel scheme is provably better than the efficiency of uniformly random selection, and can reach the efficiency o ...

PMLR2017

, , ,

The minimization of empirical risks over finite sample sizes is an important problem in large-scale machine learning. A variety of algorithms has been proposed in the literature to alleviate the computational burden per iteration at the expense of converge ...

IEEE2016

Attributing Aliasing Artifacts in Dirty Intensity Fields to their Parent Source

Matthieu Martin Jean-André Simeoni

Estimating intensity fields of stochastic phenomenons is of crucial interest in many scientific applications. Typical experimental setups involve an acquisition system, that subsequently filters and samples the probed intensity field. This equivalently def ...

2016

Theory of representation learning in cortical neural networks

Carlos Stein Naves de Brito

Our brain continuously self-organizes to construct and maintain an internal representation of the world based on the information arriving through sensory stimuli. Remarkably, cortical areas related to different sensory modalities appear to share the same f ...

EPFL2016

Compressive Spectral Clustering

Pierre Vandergheynst, Rémi Gribonval, Gilles Puy, Nicolas Tremblay

Spectral clustering has become a popular technique due to its high performance in many contexts. It comprises three main steps: create a similarity graph between N objects to cluster, compute the first k eigenvectors of its Laplacian matrix to define a fea ...

2016

Stochastic Spectral Descent for Restricted Boltzmann Machines.

Volkan Cevher

Restricted Boltzmann Machines (RBMs) are widely used as building blocks for deep learning models. Learning typically proceeds by using stochastic gradient descent, and the gradients are estimated with sampling methods. However, the gradient estimation is a ...

2015