Publications related to Second-Order Guarantees In Federated Learning

Novel Ordering-based Approaches for Causal Structure Learning in the Presence of Unobserved Variables

We propose ordering-based approaches for learning the maximal ancestral graph (MAG) of a structural equation model (SEM) up to its Markov equivalence class (MEC) in the presence of unobserved variables. Existing ordering-based methods in the literature rec ...

Association for the Advancement of Artificial Intelligence (AAAI)2023

Communication-efficient distributed training of machine learning models

Thijs Vogels

In this thesis, we explore techniques for addressing the communication bottleneck in data-parallel distributed training of deep learning models. We investigate algorithms that either reduce the size of the messages that are exchanged between workers, or th ...

EPFL2023

Breaking the Curse of Dimensionality in Deep Neural Networks by Learning Invariant Representations

Leonardo Petrini

Artificial intelligence, particularly the subfield of machine learning, has seen a paradigm shift towards data-driven models that learn from and adapt to data. This has resulted in unprecedented advancements in various domains such as natural language proc ...

EPFL2023

End-to-End Learning for Stochastic Optimization: A Bayesian Perspective

Daniel Kuhn, Yves Rychener, Tobias Sutter

We develop a principled approach to end-to-end learning in stochastic optimization. First, we show that the standard end-to-end learning algorithm admits a Bayesian interpretation and trains a posterior Bayes action map. Building on the insights of this an ...

2023

Byzantine Machine Learning: A Primer

Rachid Guerraoui, Nirupam Gupta, Rafaël Benjamin Pinot

The problem of Byzantine resilience in distributed machine learning, a.k.a., Byzantine machine learning, consists in designing distributed algorithms that can train an accurate model despite the presence of Byzantine nodes, i.e., nodes with corrupt data or ...

2023

The statistical complexity of early-stopped mirror descent

Tomas Vaskevicius, Varun Kanade

Recently there has been a surge of interest in understanding implicit regularization properties of iterative gradient-based optimization algorithms. In this paper, we study the statistical guarantees on the excess risk achieved by early-stopped unconstrain ...

Oxford2023

Aiming beyond slight increases in accuracy

Daniel Probst

Owing to the diminishing returns of deep learning and the focus on model accuracy, machine learning for chemistry might become an endeavour exclusive to well-funded institutions and industry. Extending the focus to model efficiency and interpretability wil ...

NATURE PORTFOLIO2023

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Patrick Thiran, Negar Kiyavash, Saber Salehkaleybar

We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with

1\le\alpha\le2

which holds in a wide range of applications in machine learning and signal processing. This conditio ...

NeurIPS2022

Predicting in Uncertain Environments: Methods for Robust Machine Learning

Paul Thierry Yves Rolland

One of the main goal of Artificial Intelligence is to develop models capable of providing valuable predictions in real-world environments. In particular, Machine Learning (ML) seeks to design such models by learning from examples coming from this same envi ...

EPFL2022

Optimization Over Banach Spaces: A Unified View on Supervised Learning and Inverse Problems

Shayan Aziznejad

In this thesis, we reveal that supervised learning and inverse problems share similar mathematical foundations. Consequently, we are able to present a unified variational view of these tasks that we formulate as optimization problems posed over infinite-di ...

EPFL2022

Second-Order Guarantees In Federated Learning

Graph Chatbot

Chat with Graph Search

Novel Ordering-based Approaches for Causal Structure Learning in the Presence of Unobserved Variables

Communication-efficient distributed training of machine learning models

Breaking the Curse of Dimensionality in Deep Neural Networks by Learning Invariant Representations

End-to-End Learning for Stochastic Optimization: A Bayesian Perspective

Byzantine Machine Learning: A Primer

The statistical complexity of early-stopped mirror descent

Aiming beyond slight increases in accuracy

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Predicting in Uncertain Environments: Methods for Robust Machine Learning

Optimization Over Banach Spaces: A Unified View on Supervised Learning and Inverse Problems

Communication-efficient distributed training of machine learning models

Optimization Over Banach Spaces: A Unified View on Supervised Learning and Inverse Problems

Breaking the Curse of Dimensionality in Deep Neural Networks by Learning Invariant Representations

Novel Ordering-based Approaches for Causal Structure Learning in the Presence of Unobserved Variables

End-to-End Learning for Stochastic Optimization: A Bayesian Perspective

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Predicting in Uncertain Environments: Methods for Robust Machine Learning

Aiming beyond slight increases in accuracy

The statistical complexity of early-stopped mirror descent

Byzantine Machine Learning: A Primer