Publications by Maksym Andriushchenko

Understanding generalization and robustness in modern deep learning

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

EPFL2024

Sparse-RS: A Versatile Framework for Query-Efficient Sparse Black-Box Adversarial Attacks

Nicolas Henri Bernard Flammarion, Maksym Andriushchenko, Francesco Croce

We propose a versatile framework based on random search, Sparse-RS, for score-based sparse targeted and untargeted attacks in the black-box setting. Sparse-RS does not rely on substitute models and achieves state-of-the-art success rate and query efficienc ...

ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE2022

Towards Understanding Sharpness-Aware Minimization

Nicolas Henri Bernard Flammarion, Maksym Andriushchenko

Sharpness-Aware Minimization (SAM) is a recent training method that relies on worst-case weight perturbations which significantly improves generalization in various settings. We argue that the existing justifications for the success of SAM which are based ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

On the effectiveness of adversarial training against common corruptions

Nicolas Henri Bernard Flammarion, Maksym Andriushchenko, Klim Kireev

The literature on robustness towards common corruptions shows no consensus on whether adversarial training can improve the performance in this setting. First, we show that, when used with an appropriately selected perturbation radius, ℓp adversarial traini ...

2021

On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines

Maksym Andriushchenko

Fine-tuning pre-trained transformer-based language models such as BERT has become a common practice dominating leaderboards across various NLP benchmarks. Despite the strong empirical performance of fine-tuned models, fine-tuning is an unstable process: tr ...

2021

Understanding and Improving Fast Adversarial Training

Nicolas Henri Bernard Flammarion, Maksym Andriushchenko

A recent line of work focused on making adversarial training computationally efficient for deep learning models. In particular, Wong et al. (2020) showed that ℓ∞-adversarial training with fast gradient sign method (FGSM) can fail due to a phenomenon called ...

2020

Square Attack: a query-efficient black-box adversarial attack via random search

Nicolas Henri Bernard Flammarion, Maksym Andriushchenko, Francesco Croce

We propose the Square Attack, a new score-based black-box

l_2

and

l_\infty

adversarial attack that does not rely on local gradient information and thus is not affected by gradient masking. The Square Attack is based on a randomized search scheme where ...

2020

Sparse-RS: a versatile framework for query-efficient sparse black-box adversarial attacks

Nicolas Henri Bernard Flammarion, Maksym Andriushchenko, Francesco Croce

A large body of research has focused on adversarial attacks which require to modify all input features with small l2- or l∞-norms. In this paper we instead focus on query-efficient sparse attacks in the black-box setting. Our versatile framework, Sparse-RS ...

2020

RobustBench: a standardized adversarial robustness benchmark

Nicolas Henri Bernard Flammarion, Maksym Andriushchenko, Francesco Croce

Evaluation of adversarial robustness is often error-prone leading to overestimation of the true robustness of models. While adaptive attacks designed for a particular defense are a way out of this, there are only approximate guidelines on how to perform th ...

2020

Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem

Maksym Andriushchenko

Classifiers used in the wild, in particular for safety-critical systems, should not only have good generalization properties but also should know when they don't know, in particular make low confidence predictions far away from the training data. We show t ...

2019

Maksym Andriushchenko

Graph Chatbot

Chat with Graph Search

Understanding generalization and robustness in modern deep learning

Sparse-RS: A Versatile Framework for Query-Efficient Sparse Black-Box Adversarial Attacks

Towards Understanding Sharpness-Aware Minimization

On the effectiveness of adversarial training against common corruptions

On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines

Understanding and Improving Fast Adversarial Training

Square Attack: a query-efficient black-box adversarial attack via random search

Sparse-RS: a versatile framework for query-efficient sparse black-box adversarial attacks

RobustBench: a standardized adversarial robustness benchmark

Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem

On the effectiveness of adversarial training against common corruptions

Towards Understanding Sharpness-Aware Minimization

On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines

Square Attack: a query-efficient black-box adversarial attack via random search

Sparse-RS: a versatile framework for query-efficient sparse black-box adversarial attacks

RobustBench: a standardized adversarial robustness benchmark

Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem

Understanding generalization and robustness in modern deep learning

Sparse-RS: A Versatile Framework for Query-Efficient Sparse Black-Box Adversarial Attacks

Understanding and Improving Fast Adversarial Training