Publications related to Byzantine machine learning

On the Generalization of Stochastic Gradient Descent with Momentum

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...

Brookline2024

Universal and adaptive methods for robust stochastic optimization

Ali Kavis

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

EPFL2023

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Negar Kiyavash, Saber Salehkaleybar

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

Volkan Cevher, Alp Yurtsever, Maria-Luiza Vladarean

We propose a stochastic conditional gradient method (CGM) for minimizing convex finitesum objectives formed as a sum of smooth and non-smooth terms. Existing CGM variants for this template either suffer from slow convergence rates, or require carefully inc ...

2022

Using Machine Learning to Predict Mortality for COVID-19 Patients on Day 0 in the ICU

Nahal Mansouri, Sahand Jamal Rahi

Rationale: Given the expanding number of COVID-19 cases and the potential for new waves of infection, there is an urgent need for early prediction of the severity of the disease in intensive care unit (ICU) patients to optimize treatment strategies.Objecti ...

FRONTIERS MEDIA SA2022

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Florent Gérard Krzakala, Lenka Zdeborová, Ludovic Théo Stephan, Bruno Loureiro

Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achieve global convergence under gradient descent. The picture can be radically different for narrow net-works, which tend to get stuck in badly-generalizing loca ...

2022

Distributed Learning in Non-Convex Environments-Part I: Agreement at a Linear Rate

Ali H. Sayed, Stefan Vlaski

Driven by the need to solve increasingly complex optimization problems in signal processing and machine learning, there has been increasing interest in understanding the behavior of gradient-descent algorithms in non-convex environments. Most available wor ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2021

Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithms

Volkan Cevher, Junhong Lin

We study generalization properties of distributed algorithms in the setting of nonparametric regression over a reproducing kernel Hilbert space (RKHS). We first investigate distributed stochastic gradient methods (SGM), with mini-batches and multi-passes o ...

2020

Multi-armed Bandits in Action

Farnood Salehi

Making decisions is part and parcel of being human. Among a set of actions, we want to choose the one that has the highest reward. But the uncertainty of the outcome prevents us from always making the right decision. Making decisions under uncertainty can ...

EPFL2020

Byzantine tolerant gradient descent for distributed machine learning with adversaries

Rachid Guerraoui, El Mahdi El Mhamdi, Julien Michel Stainer, Peva François Blanchard

The present application concerns a computer-implemented method for training a machine learning model in a distributed fashion, using Stochastic Gradient Descent, SGD, wherein the method is performed by a first computer in a distributed computing environmen ...

2019

Byzantine machine learning

Graph Chatbot

Chat with Graph Search

On the Generalization of Stochastic Gradient Descent with Momentum

Universal and adaptive methods for robust stochastic optimization

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

Using Machine Learning to Predict Mortality for COVID-19 Patients on Day 0 in the ICU

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Distributed Learning in Non-Convex Environments-Part I: Agreement at a Linear Rate

Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithms

Multi-armed Bandits in Action

Byzantine tolerant gradient descent for distributed machine learning with adversaries

Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization

Using Machine Learning to Predict Mortality for COVID-19 Patients on Day 0 in the ICU

On the Generalization of Stochastic Gradient Descent with Momentum

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Universal and adaptive methods for robust stochastic optimization

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Distributed Learning in Non-Convex Environments-Part I: Agreement at a Linear Rate

Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithms

Multi-armed Bandits in Action

Byzantine tolerant gradient descent for distributed machine learning with adversaries