Publication
Redundant features can hurt robustness to distribution shift
Pascal Frossard, Seyed Mohsen Moosavi Dezfooli, Guillermo Ortiz Jimenez, Apostolos Modas
2020
Conference paper
2020
Conference paper
Abstract
In this work, we borrow tools from the field of adversarial robustness, and propose a new framework that permits to relate dataset features to the distance of samples to the decision boundary. Using this framework we identify the subspace of features used by CNNs to classify largescale vision benchmarks, and reveal some intriguing aspects of their robustness to distributions shift. Specifically, by manipulating the frequency content in CIFAR-10 we show that the existence of redundant features on a dataset can harm the networks’ robustness to distribution shifts. We demonstrate that completely erasing the redundant information from the training set can efficiently solve this problem. This paper is a short version of (Ortiz-Jimenez et al., 2020).
Official source
About this result
This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.
Related concepts (16)
Stable distribution
In probability theory, a distribution is said to be stable if a linear combination of two independent random variables with this distribution has the same distribution, up to location and scale parameters. A random variable is said to be stable if its distribution is stable. The stable distribution family is also sometimes referred to as the Lévy alpha-stable distribution, after Paul Lévy, the first mathematician to have studied it. Of the four parameters defining the family, most attention has been focused on the stability parameter, (see panel).
Lévy distribution
In probability theory and statistics, the Lévy distribution, named after Paul Lévy, is a continuous probability distribution for a non-negative random variable. In spectroscopy, this distribution, with frequency as the dependent variable, is known as a van der Waals profile. It is a special case of the inverse-gamma distribution. It is a stable distribution. The probability density function of the Lévy distribution over the domain is where is the location parameter and is the scale parameter.
Hyperbolic secant distribution
In probability theory and statistics, the hyperbolic secant distribution is a continuous probability distribution whose probability density function and characteristic function are proportional to the hyperbolic secant function. The hyperbolic secant function is equivalent to the reciprocal hyperbolic cosine, and thus this distribution is also called the inverse-cosh distribution. Generalisation of the distribution gives rise to the Meixner distribution, also known as the Natural Exponential Family - Generalised Hyperbolic Secant or NEF-GHS distribution.
Related publications (32)
Making Computer Vision Models Robust and Adaptive
Visual perception is indispensable for many real-world applications. However, perception models deployed in the real world will encounter numerous and unpredictable distribution shifts, for example, changes in geographic locations, motion blur, and adverse ...
EPFL2023Learning Robust and Adaptive Representations: from Interactions, for Interactions
Interactions are ubiquitous in our world, spanning from social interactions between human individuals to physical interactions between robots and objects to mechanistic interactions among different components of an intelligent system. Despite their prevale ...
EPFL2023Federated Learning under Covariate Shifts with Generalization Guarantees
Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Thomas Michaelsen Pethick
This paper addresses intra-client and inter-client covariate shifts in federated learning (FL) with a focus on the overall generalization performance. To handle covariate shifts, we formulate a new global model training paradigm and propose Federated Impor ...
2023Related MOOCs (4)
Advanced statistical physics
We explore statistical physics in both classical and open quantum systems. Additionally, we will cover probabilistic data analysis that is extremely useful in many applications.
Advanced statistical physics
We explore statistical physics in both classical and open quantum systems. Additionally, we will cover probabilistic data analysis that is extremely useful in many applications.
Selected Topics on Discrete Choice
Discrete choice models are used extensively in many disciplines where it is important to predict human behavior at a disaggregate level. This course is a follow up of the online course “Introduction t