Publication

On the effectiveness of adversarial training against common corruptions

Abstract

The literature on robustness towards common corruptions shows no consensus on whether adversarial training can improve the performance in this setting. First, we show that, when used with an appropriately selected perturbation radius, ℓp adversarial training can serve as a strong baseline against common corruptions. Then we explain why adversarial training performs better than data augmentation with simple Gaussian noise which has been observed to be a meaningful baseline on common corruptions. Related to this, we identify the σ-overfitting phenomenon when Gaussian augmentation overfits to a particular standard deviation used for training which has a significant detrimental effect on common corruption accuracy. We discuss how to alleviate this problem and then how to further enhance ℓp adversarial training by introducing an efficient relaxation of adversarial training with learned perceptual image patch similarity as the distance metric. Through experiments on CIFAR-10 and ImageNet-100, we show that our approach does not only improve the ℓp adversarial training baseline but also has cumulative gains with data augmentation methods such as AugMix, ANT, and SIN leading to state-of-the-art performance on common corruptions. The code of our experiments is publicly available at https://github.com/tml-epfl/adv-training-corruptions.

About this result
This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.
Related concepts (33)
Corruption
Corruption is a form of dishonesty or a criminal offense which is undertaken by a person or an organization which is entrusted in a position of authority, in order to acquire illicit benefits or abuse power for one's personal gain. Corruption may involve many activities which include bribery, influence peddling and the embezzlement and it may also involve practices which are legal in many countries. Political corruption occurs when an office-holder or other governmental employee acts with an official capacity for personal gain.
Standard deviation
In statistics, the standard deviation is a measure of the amount of variation or dispersion of a set of values. A low standard deviation indicates that the values tend to be close to the mean (also called the expected value) of the set, while a high standard deviation indicates that the values are spread out over a wider range. Standard deviation may be abbreviated SD, and is most commonly represented in mathematical texts and equations by the lower case Greek letter σ (sigma), for the population standard deviation, or the Latin letter s, for the sample standard deviation.
Average absolute deviation
The average absolute deviation (AAD) of a data set is the average of the absolute deviations from a central point. It is a summary statistic of statistical dispersion or variability. In the general form, the central point can be a mean, median, mode, or the result of any other measure of central tendency or any reference value related to the given data set. AAD includes the mean absolute deviation and the median absolute deviation (both abbreviated as MAD). Several measures of statistical dispersion are defined in terms of the absolute deviation.
Show more
Related publications (32)

RMSSD Is More Sensitive to Artifacts Than Frequency-Domain Parameters: Implication in Athletes? Monitoring

Jean-Marc Vesin, Grégoire Millet, Sasan Yazdani

Easy-to-use and accurate heart rate variability (HRV) assessments are essential in athletes??? follow-up, but artifacts may lead to erroneous analysis. Artifact detection and correction are the purpose of extensive literature and implemented in dedicated a ...
JOURNAL SPORTS SCIENCE & MEDICINE2022

Shannon entropy and degree of polarization of a speckle pattern

Abhijit Roy

The dependence of the Shannon entropy (SE) of a speckle pattern on the degree of polarization (DoP) of the pattern is investigated both experimentally and numerically. The superposition of two uncorrelated speckle patterns with polarization diversity is ut ...
OPTICAL SOC AMER2021

Study of B -> ppi pi

Olivier Schneider, Mingkui Wang, Chao Wang, Tagir Aushev, Sun Hee Kim, Donghyun Kim, Xiao Wang, Lei Li

Using a data sample of 772 x 10(6) B (B) over bar pairs collected on the Upsilon(4S) resonance with the Belle detector at the KEKB asymmetric energy e(+)e(-) collider, we report the observation of B-0 -> p (p) over bar pi(+)pi(-) and the first observation ...
AMER PHYSICAL SOC2020
Show more