Publication

On the Double Descent of Random Features Models Trained with SGD

Related publications (36)

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

Volkan Cevher, Alp Yurtsever, Maria-Luiza Vladarean

We propose a stochastic conditional gradient method (CGM) for minimizing convex finitesum objectives formed as a sum of smooth and non-smooth terms. Existing CGM variants for this template either suffer from slow convergence rates, or require carefully inc ...

2022

Countless signal processing applications include the reconstruction of signals from few indirect linear measurements. The design of effective measurement operators is typically constrained by the underlying hardware and physics, posing a challenging and of ...

2022

We formulate gradient-based Markov chain Monte Carlo (MCMC) sampling as optimization on the space of probability measures, with Kullback-Leibler (KL) divergence as the objective functional. We show that an under-damped form of the Langevin algorithm perfor ...

INT STATISTICAL INST2021

Distributed Momentum for Byzantine-resilient Stochastic Gradient Descent

Rachid Guerraoui, El Mahdi El Mhamdi, Sébastien Louis Alexandre Rouault

Byzantine-resilient Stochastic Gradient Descent (SGD) aims at shielding model training from Byzantine faults, be they ill-labeled training datapoints, exploited software/hardware vulnerabilities, or malicious worker nodes in a distributed setting. Two rece ...

2021

Collaborative Sampling in Generative Adversarial Networks

Alexandre Massoud Alahi, Yuejiang Liu, Parth Ashit Kothari

The standard practice in Generative Adversarial Networks (GANs) discards the discriminator during sampling. However, this sampling method loses valuable information learned by the discriminator regarding the data distribution. In this work, we propose a co ...

2020

Multi-armed Bandits in Action

Farnood Salehi

Making decisions is part and parcel of being human. Among a set of actions, we want to choose the one that has the highest reward. But the uncertainty of the outcome prevents us from always making the right decision. Making decisions under uncertainty can ...

EPFL2020

Fast Mean Estimation with Sub-Gaussian Rates

Nicolas Henri Bernard Flammarion

We present an improved analysis of the Euler-Maruyama discretization of the Langevin diffusion. Our analysis does not require global contractivity, and yields polynomial dependence on the time horizon. Compared to existing approaches, we make an additional ...

2019

Rethinking Sampling in Parallel MRI: A Data-Driven Approach

Volkan Cevher, Baran Gözcü, Thomas Sanchez

In the last decade, Compressive Sensing (CS) has emerged as the most promising, model-driven approach to accelerate MRI scans. CS relies on the key sparsity assumption and proposes random sampling for data acquisition. The practical CS approaches in MRI em ...

IEEE2019

Cover: A Cluster-Based Variance Reduced Method For Online Learning

Ali H. Sayed, Bicheng Ying, Kun Yuan

In this paper, we develop a stochastic-gradient learning algorithm for situations involving streaming data that arise from an underlying clustered structure. In such settings, the variance of gradient noise can be decomposed into the in-cluster variance si ...

IEEE2019

Mirrored Langevin Dynamics

Volkan Cevher, Paul Thierry Yves Rolland, Ya-Ping Hsieh, Ali Kavis

We consider the problem of sampling from constrained distributions, which has posed significant challenges to both non-asymptotic analysis and algorithmic design. We propose a unified framework, which is inspired by the classical mirror descent, to derive ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2018