Publications related to Robust Collaborative Learning with Linear Gradient Overhead

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Distributed learning is the key for enabling training of modern large-scale machine learning models, through parallelising the learning process. Collaborative learning is essential for learning from privacy-sensitive data that is distributed across various ...

EPFL2024

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Nicolas Henri Bernard Flammarion, Hristo Georgiev Papazov, Scott William Pesme

In this work, we investigate the effect of momentum on the optimisation trajectory of gradient descent. We leverage a continuous-time approach in the analysis of momentum gradient descent with step size

\gamma

and momentum parameter

\beta

that allows u ...

2024

Universal and adaptive methods for robust stochastic optimization

Ali Kavis

Within the context of contemporary machine learning problems, efficiency of optimization process depends on the properties of the model and the nature of the data available, which poses a significant problem as the complexity of either increases ad infinit ...

EPFL2023

Distributionally Robust Linear Quadratic Control

Daniel Kuhn, Bahar Taskesen, Cagil Kocyigit

Linear-Quadratic-Gaussian (LQG) control is a fundamental control paradigm that is studied in various fields such as engineering, computer science, economics, and neuroscience. It involves controlling a system with linear dynamics and imperfect observations ...

2023

Data-driven Methods for Control: from Linear to Lifting

Yingzhao Lian

The progress towards intelligent systems and digitalization relies heavily on the use of automation technology. However, the growing diversity of control objects presents significant challenges for traditional control approaches, as they are highly depende ...

EPFL2023

Creating Trust by Abolishing Hierarchies

Mathias Josef Payer, Edouard Bugnion, Evangelos Marios Kogias, Adrien Ghosn, Charly Nicolas Lucien Castes, Neelu Shivprakash Kalani, Yuchen Qian

Software is going through a trust crisis. Privileged code is no longer trusted and processes insufficiently protect user code from unverified libraries. While usually treated separately, confidential computing and program compartmentalization are both symp ...

Association for Computing Machinery2023

Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity

Nicolas Henri Bernard Flammarion, Scott William Pesme, Loucas Pillaud-Vivien

Understanding the implicit bias of training algorithms is of crucial importance in order to explain the success of overparametrised neural networks. In this paper, we study the dynamics of stochastic gradient descent over diagonal linear networks through i ...

2021

Extrapolation for Large-batch Training in Deep Learning

Martin Jaggi, Sebastian Urban Stich, Tao Lin, Lingjing Kong

Deep learning networks are typically trained by Stochastic Gradient Descent (SGD) methods that iteratively improve the model parameters by estimating a gradient on a very small fraction of the training data. A major roadblock faced when increasing the batc ...

2020

Chirok: a post-refinement tool to analyse absolute structure

Philip Pattison

Chirok is software for a post-refinement test of the absolute structure. The software allows a user to calculate a distribution of the measure of chirality based on intensity quotients and linked to the Flack parameter. The distribution is fitted by a set ...

2020

HotelRec: a Novel Very Large-Scale Hotel Recommendation Dataset

Boi Faltings, Diego Matteo Antognini

Today, recommender systems are an inevitable part of everyone's daily digital routine and are present on most internet platforms. State-of-the-art deep learning-based models require a large number of data to achieve their best performance. Many datasets fu ...

EUROPEAN LANGUAGE RESOURCES ASSOC-ELRA2020

Robust Collaborative Learning with Linear Gradient Overhead

Graph Chatbot

Chat with Graph Search

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Universal and adaptive methods for robust stochastic optimization

Distributionally Robust Linear Quadratic Control

Data-driven Methods for Control: from Linear to Lifting

Creating Trust by Abolishing Hierarchies

Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity

Extrapolation for Large-batch Training in Deep Learning

Chirok: a post-refinement tool to analyse absolute structure

HotelRec: a Novel Very Large-Scale Hotel Recommendation Dataset

Data-driven Methods for Control: from Linear to Lifting

Distributionally Robust Linear Quadratic Control

Optimization Algorithms for Decentralized, Distributed and Collaborative Machine Learning

Universal and adaptive methods for robust stochastic optimization

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Creating Trust by Abolishing Hierarchies

Chirok: a post-refinement tool to analyse absolute structure

Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity

Extrapolation for Large-batch Training in Deep Learning

HotelRec: a Novel Very Large-Scale Hotel Recommendation Dataset