Publications associées à Empirical risk minimization

Asymptotic Errors for Teacher-Student Convex Generalized Linear Models (Or: How to Prove Kabashima’s Replica Formula)

There has been a recent surge of interest in the study of asymptotic reconstruction performance in various cases of generalized linear estimation problems in the teacher-student setting, especially for the case of i.i.d standard normal matrices. Here, we g ...

2022

Byzantine Machine Learning Made Easy By Resilient Averaging of Momentums

Rachid Guerraoui, Nirupam Gupta, John Stephan, Sadegh Farhadkhani, Rafaël Benjamin Pinot

Byzantine resilience emerged as a prominent topic within the distributed machine learning community. Essentially, the goal is to enhance distributed optimization algorithms, such as distributed SGD, in a way that guarantees convergence despite the presence ...

PMLR2022

Local plasticity rules can learn deep representations using self-supervised contrastive predictions

Wulfram Gerstner, Bernd Albert Illing, Jean Ventura

Learning in the brain is poorly understood and learning rules that respect biological constraints, yet yield deep hierarchical representations, are still unknown. Here, we propose a learning rule that takes inspiration from neuroscience and recent advances ...

2021

NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization

Dan Alistarh, Fartash Faghri

As the size and complexity of models and datasets grow, so does the need for communication-efficient variants of stochastic gradient descent that can be deployed to perform parallel model training. One popular communication-compression method for data-para ...

MICROTOME PUBL2021

Disparity Between Batches as a Signal for Early Stopping

Patrick Thiran, Mahsa Forouzesh

We propose a metric for evaluating the generalization ability of deep neural networks trained with mini-batch gradient descent. Our metric, called gradient disparity, is the l2 norm distance between the gradient vectors of two mini-batches drawn from the t ...

Springer2021

Byzantine Fault-Tolerant Distributed Machine Learning with Norm-Based Comparative Gradient Elimination

Nirupam Gupta, Shuo Liu

This paper considers the Byzantine fault-tolerance problem in distributed stochastic gradient descent (D-SGD) method - a popular algorithm for distributed multi-agent machine learning. In this problem, each agent samples data points independently from a ce ...

IEEE COMPUTER SOC2021

Extrapolation for Large-batch Training in Deep Learning

Martin Jaggi, Sebastian Urban Stich, Tao Lin, Lingjing Kong

Deep learning networks are typically trained by Stochastic Gradient Descent (SGD) methods that iteratively improve the model parameters by estimating a gradient on a very small fraction of the training data. A major roadblock faced when increasing the batc ...

2020

Risk prediction error signaling: A two-component response?

Michael Herzog, Ophélie Gladys Favrod, Kerstin Preuschoff, Marc Michael Lauffs, Sophie Alberta Catherine Geoghan

Organisms use rewards to navigate and adapt to (uncertain) environments. Error-based learning about rewards issupported by the dopaminergic system, which is thought to signal reward prediction errors to make adjustmentsto past predictions. More recently, t ...

2020

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

Empirical risk minimization

Asymptotic Errors for Teacher-Student Convex Generalized Linear Models (Or: How to Prove Kabashima’s Replica Formula)

Byzantine Machine Learning Made Easy By Resilient Averaging of Momentums

Local plasticity rules can learn deep representations using self-supervised contrastive predictions

NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization

Disparity Between Batches as a Signal for Early Stopping

Byzantine Fault-Tolerant Distributed Machine Learning with Norm-Based Comparative Gradient Elimination

Chaining Meets Chain Rule: Multilevel Entropic Regularization and Training of Neural Networks

Wasserstein Distributionally Robust Learning

Extrapolation for Large-batch Training in Deep Learning

Risk prediction error signaling: A two-component response?

Graph Chatbot

Chattez avec Graph Search

NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization

Local plasticity rules can learn deep representations using self-supervised contrastive predictions

Disparity Between Batches as a Signal for Early Stopping

Asymptotic Errors for Teacher-Student Convex Generalized Linear Models (Or: How to Prove Kabashima’s Replica Formula)

Byzantine Fault-Tolerant Distributed Machine Learning with Norm-Based Comparative Gradient Elimination

Byzantine Machine Learning Made Easy By Resilient Averaging of Momentums

Wasserstein Distributionally Robust Learning

Chaining Meets Chain Rule: Multilevel Entropic Regularization and Training of Neural Networks

Extrapolation for Large-batch Training in Deep Learning

Risk prediction error signaling: A two-component response?