Publications related to Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification

Byzantine Fault-Tolerance in Federated Local SGD Under 2f-Redundancy

In this article, we study the problem of Byzantine fault-tolerance in a federated optimization setting, where there is a group of agents communicating with a centralized coordinator. We allow up to

f

Byzantine-faulty agents, which may not follow a prescr ...

Piscataway2023

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Florent Gérard Krzakala, Lenka Zdeborová, Ludovic Théo Stephan, Bruno Loureiro

Despite the non-convex optimization landscape, over-parametrized shallow networks are able to achieve global convergence under gradient descent. The picture can be radically different for narrow net-works, which tend to get stuck in badly-generalizing loca ...

2022

Towards Understanding Sharpness-Aware Minimization

Nicolas Henri Bernard Flammarion, Maksym Andriushchenko

Sharpness-Aware Minimization (SAM) is a recent training method that relies on worst-case weight perturbations which significantly improves generalization in various settings. We argue that the existing justifications for the success of SAM which are based ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

On the robustness of randomized classifiers to adversarial examples

Rafaël Benjamin Pinot

This paper investigates the theory of robustness against adversarial attacks. We focus on randomized classifiers (i.e. classifiers that output random variables) and provide a thorough analysis of their behavior through the lens of statistical learning theo ...

SPRINGER2022

Stop Wasting my FLOPS: Improving the Efficiency of Deep Learning Models

Angelos Katharopoulos

Deep neural networks have completely revolutionized the field of machinelearning by achieving state-of-the-art results on various tasks ranging fromcomputer vision to protein folding. However, their application is hindered bytheir large computational and m ...

EPFL2022

Towards Verifiable, Generalizable and Efficient Robust Deep Neural Networks.

Chen Liu

In the last decade, deep neural networks have achieved tremendous success in many fields of machine learning.However, they are shown vulnerable against adversarial attacks: well-designed, yet imperceptible, perturbations can make the state-of-the-art deep ...

EPFL2022

System and method for privacy-preserving distributed training of neural network models on distributed datasets

Jean-Pierre Hubaux, Juan Ramón Troncoso-Pastoriza, Jean-Philippe Léonard Bossuat, Apostolos Pyrgelis, David Jules Froelicher, Sinem Sav

A computer-implemented method and a distributed computer system (100) for privacy- preserving distributed training of a global neural network model on distributed datasets (DS1 to DSn). The system has a plurality of data providers (DP1 to DPn) being commun ...

2022

Advances In Morphological Neural Networks: Training, Pruning And Enforcing Shape Constraints

Nikolaos Dimitriadis

In this paper, we study an emerging class of neural networks, the Morphological Neural networks, from some modern perspectives. Our approach utilizes ideas from tropical geometry and mathematical morphology. First, we state the training of a binary morphol ...

IEEE2021

Gradient-based Methods for Deep Model Interpretability

Suraj Srinivas

In this dissertation, we propose gradient-based methods for characterizing model behaviour for the purposes of knowledge transfer and post-hoc model interpretation. Broadly, gradients capture the variation of some output feature of the model upon unit vari ...

EPFL2021

Neural Tangent Kernel: Convergence and Generalization in Neural Networks (Invited Paper)

Clément Hongler, Franck Raymond Gabriel, Arthur Jacot

The Neural Tangent Kernel is a new way to understand the gradient descent in deep neural networks, connecting them with kernel methods. In this talk, I'll introduce this formalism and give a number of results on the Neural Tangent Kernel and explain how th ...

ASSOC COMPUTING MACHINERY2021

Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification

Graph Chatbot

Chat with Graph Search

Byzantine Fault-Tolerance in Federated Local SGD Under 2f-Redundancy

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Towards Understanding Sharpness-Aware Minimization

On the robustness of randomized classifiers to adversarial examples

Stop Wasting my FLOPS: Improving the Efficiency of Deep Learning Models

Towards Verifiable, Generalizable and Efficient Robust Deep Neural Networks.

System and method for privacy-preserving distributed training of neural network models on distributed datasets

Advances In Morphological Neural Networks: Training, Pruning And Enforcing Shape Constraints

Gradient-based Methods for Deep Model Interpretability

Neural Tangent Kernel: Convergence and Generalization in Neural Networks (Invited Paper)

Byzantine Fault-Tolerance in Federated Local SGD Under 2f-Redundancy

Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks

Towards Understanding Sharpness-Aware Minimization

Stop Wasting my FLOPS: Improving the Efficiency of Deep Learning Models

On the robustness of randomized classifiers to adversarial examples

Towards Verifiable, Generalizable and Efficient Robust Deep Neural Networks.

Advances In Morphological Neural Networks: Training, Pruning And Enforcing Shape Constraints

Gradient-based Methods for Deep Model Interpretability

Neural Tangent Kernel: Convergence and Generalization in Neural Networks (Invited Paper)

System and method for privacy-preserving distributed training of neural network models on distributed datasets