Chen Liu | EPFL Graph Search

I am a PhD candidate at the School of Computer and Communication Sciences (IC), École Polytechnique Fédérale de Lausanne (EPFL), supervised by Prof. Sabine Süsstrunk and co-supervised by Dr. Mathieu Salzmann. Previously, I obtained my Master Degree from École Polytechnique Fédérale de Lausanne (EPFL) in 2017 and Bachelor Degree from Tsinghua University in 2015, both in Computer Science.

Towards Stable and Efficient Adversarial Training against $l_1$ Bounded Adversarial Attacks

Yulun Jiang, Chen Liu, Mathieu Salzmann, Sabine Süsstrunk

We address the problem of stably and efficiently training a deep neural network robust to adversarial perturbations bounded by an

l_1

norm. We demonstrate that achieving robustness against

l_1

-bounded perturbations is more challenging than in the

l_2

l_\infty

cases, because adversarial training against

l_1

-bounded perturbations is more likely to suffer from catastrophic overfitting and yield training instabilities. Our analysis links these issues to the coordinate descent strategy used in existing methods. We address this by introducing Fast-EG-

l_1

, an efficient adversarial training algorithm based on Euclidean geometry and free of coordinate descent. Fast-EG-

l_1

comes with no additional memory costs and no extra hyper-parameters to tune. Our experimental results on various datasets demonstrate that Fast-EG-

l_1

yields the best and most stable robustness against

l_1

-bounded adversarial attacks among the methods of comparable computational complexity. Code and the checkpoints are available at https://github.com/IVRL/FastAdvL.

2023

Robust Binary Models by Pruning Randomly-initialized Networks

Chen Liu, Mathieu Salzmann, Sabine Süsstrunk, Ziqi Zhao

Robustness to adversarial attacks was shown to require a larger model capacity, and thus a larger memory footprint. In this paper, we introduce an approach to obtain robust yet compact models by pruning randomly-initialized binary networks. Unlike adversarial training, which learns the model parameters, we initialize the model parameters as either +1 or −1, keep them fixed, and find a subnetwork structure that is robust to attacks. Our method confirms the Strong Lottery Ticket Hypothesis in the presence of adversarial attacks, and extends this to binary networks. Furthermore, it yields more compact networks with competitive performance than existing works by 1) adaptively pruning different network layers; 2) exploiting an effective binary initialization scheme; 3) incorporating a last batch normalization layer to improve training stability. Our experiments demonstrate that our approach not only always outperforms the state-of-the-art robust binary networks, but also can achieve accuracy better than full-precision ones on some datasets. Finally, we show the structured patterns of our pruned binary networks.

2022

Towards Verifiable, Generalizable and Efficient Robust Deep Neural Networks.

Chen Liu

In the last decade, deep neural networks have achieved tremendous success in many fields of machine learning.However, they are shown vulnerable against adversarial attacks: well-designed, yet imperceptible, perturbations can make the state-of-the-art deep neural networks output incorrect results.Understanding adversarial attacks and designing algorithms to make deep neural networks robust against these attacks are key steps to building reliable artificial intelligence in real-life applications.In this thesis, we will first formulate the robust learning problem.Based on the notations of empirical robustness and verified robustness, we design new algorithms to achieve both of these types of robustness.Specifically, we investigate the robust learning problem from the optimization perspectives.Compared with classic empirical risk minimization, we show the slow convergence and large generalization gap in robust learning.Our theoretical and numerical analysis indicates that these challenges arise, respectively, from non-smooth loss landscapes and model's fitting hard adversarial instances.Our insights shed some light on designing algorithms for mitigating these challenges.Robust learning has other challenges, such as large model capacity requirements and high computational complexity.To solve the model capacity issue, we combine robust learning with model compression.We design an algorithm to obtain sparse and binary neural networks and make it robust.To decrease the computational complexity, we accelerate the existing adversarial training algorithm and preserve its performance stability.In addition to making models robust, our research provides other benefits.Our methods demonstrate that robust models, compared with non-robust ones, usually utilize input features in a way more similar to the way human beings use them, hence the robust models are more interpretable.To obtain verified robustness, our methods indicate the geometric similarity of the decision boundaries near data points.Our approaches towards reliable artificial intelligence can not only render deep neural networks more robust in safety-critical applications but also make us better aware of how they work.

EPFL2022