Publication

Distributed Momentum for Byzantine-resilient Stochastic Gradient Descent

Related publications (32)

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Not All Samples Are Created Equal: Deep Learning with Importance Sampling

François Fleuret, Angelos Katharopoulos

Deep neural network training spends most of the computation on examples that are properly handled, and could be ignored. We propose to mitigate this phenomenon with a principled importance sampling scheme that focuses computation on "informative" examples ...

Idiap2018

Coordinate-Descent Diffusion Learning by Networked Agents

Ali H. Sayed, Bicheng Ying, Chengcheng Wang

This paper examines the mean-square error performance of diffusion stochastic algorithms under a generalized coordinate-descent scheme. In this setting, the adaptation step by each agent is limited to a random subset of the coordinates of its stochastic gr ...

Ieee-Inst Electrical Electronics Engineers Inc2018

Blowing in the Wind

Marta Martinez-Camara

Every day tons of pollutants are emitted into the atmosphere all around the world. These pollutants are altering the equilibrium of our planet, causing profound changes in its climate, increasing global temperatures, and raising the sea level. The need to ...

EPFL2017

Excess-Risk of Distributed Stochastic Learners

Ali H. Sayed

This paper studies the learning ability of consensus and diffusion distributed learners from continuous streams of data arising from different but related statistical distributions. Four distinctive features for diffusion learners are revealed in relation ...

IEEE2016

Stochastic gradient descent with finite samples sizes

Ali H. Sayed, Stefan Vlaski, Bicheng Ying, Kun Yuan

The minimization of empirical risks over finite sample sizes is an important problem in large-scale machine learning. A variety of algorithms has been proposed in the literature to alleviate the computational burden per iteration at the expense of converge ...

IEEE2016

On the influence of momentum acceleration on online learning

Ali H. Sayed, Bicheng Ying, Kun Yuan

This paper examines the convergence rate and mean-square-error performance of momentum stochastic gradient methods in the constant step-size and slow adaptation regime. The results establish that momentum methods are equivalent to the standard stochastic g ...

IEEE2016

Automated Vulnerability Discovery in Distributed Systems

Rachid Guerraoui, George Candea, Radu Banabic

In this paper we present a technique for automatically assessing the amount of damage a small number of participant nodes can inﬂict on the overall performance of a large distributed system. We propose a feedback-driven tool that synthesizes malicious node ...

2011

Unbiased Model Combinations for Adaptive Filtering

Ali H. Sayed

In this paper, we consider model combination methods for adaptive filtering that perform unbiased estimation. In this widely studied framework, two adaptive filters are run in parallel, each producing unbiased estimates of an underlying linear model. The o ...

IEEE2010

Nonideal sampling and regularized interpolation of noisy data

Sathish Ramani

Conventional sampling (Shannon's sampling formulation and its approximation-theoretic counterparts) and interpolation theories provide effective solutions to the problem of reconstructing a signal from its samples, but they are primarily restricted to the ...

EPFL2009

Security Aspects of Inter-Vehicle Communications

Jean-Pierre Hubaux, Maxime Raya

Inter-Vehicular Communications (IVC) are a cornerstone of the future intelligent transportation systems. A crucial enabling component of IVC is its security and privacy. Indeed, as vehicular computing systems become interconnected, there will be new venues ...

2005