Publications related to A jamming transition from under- to over-parametrization affects generalization in deep learning

Three-dimensional tomography of red blood cells using deep learning

Demetri Psaltis, Joowon Lim, Ahmed Ayoub

We accurately reconstruct three-dimensional (3-D) refractive index (RI) distributions from highly ill-posed two-dimensional (2-D) measurements using a deep neural network (DNN). Strong distortions are introduced on reconstructions obtained by the Wolf tran ...

2020

Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions

Lenka Zdeborová

p>We study the dynamics of optimization and the generalization properties of one-hidden layer neural networks with quadratic activation function in the overparametrized regime where the layer width m is larger than the input dimension d. We conside ...

Curran Associates, Inc.2020

Supplementary Material - AL2: Progressive Activation Loss for Learning General Representations in Classification Neural Networks

Sabine Süsstrunk, Majed El Helou, Frederike Dümbgen

In this supplementary material, we present the details of the neural network architecture and training settings used in all our experiments. This holds for all experiments presented in the main paper as well as in this supplementary material. We also show ...

2020

Crowding and the Architecture of the Visual System

Adrien Christophe Doerig

Classically, vision is seen as a cascade of local, feedforward computations. This framework has been tremendously successful, inspiring a wide range of ground-breaking findings in neuroscience and computer vision. Recently, feedforward Convolutional Neural ...

EPFL2020

Neural Anisotropy Directions

Pascal Frossard, Seyed Mohsen Moosavi Dezfooli, Guillermo Ortiz Jimenez, Apostolos Modas

In this work, we analyze the role of the network architecture in shaping the inductive bias of deep classifiers. To that end, we start by focusing on a very simple problem, i.e., classifying a class of linearly separable distributions, and show that, depen ...

2020

Square Attack: a query-efficient black-box adversarial attack via random search

Nicolas Henri Bernard Flammarion, Maksym Andriushchenko, Francesco Croce

We propose the Square Attack, a new score-based black-box

l_2

and

l_\infty

adversarial attack that does not rely on local gradient information and thus is not affected by gradient masking. The Square Attack is based on a randomized search scheme where ...

2020

Shape Reconstruction by Learning Differentiable Surface Representations

Pascal Fua, Mathieu Salzmann, Shaifali Parashar, Jan Bednarík, Erhan Gündogdu

Generative models that produce point clouds have emerged as a powerful tool to represent 3D surfaces, and the best current ones rely on learning an ensemble of parametric representations. Unfortunately, they offer no control over the deformations of the su ...

2020

Biologically plausible deep learning – but how far can we go with shallow networks?

Wulfram Gerstner, Johanni Michael Brea, Bernd Albert Illing

Training deep neural networks with the error backpropagation algorithm is considered implausible from a biological perspective. Numerous recent publications suggest elaborate models for biologically plausible variants of deep learning, typically defining s ...

2019

Evaluating and Interpreting Deep Convolutional Neural Networks via Non-negative Matrix Factorization

Edo Collins

With ever greater computational resources and more accessible software, deep neural networks have become ubiquitous across industry and academia. Their remarkable ability to generalize to new samples defies the conventional view, which holds that complex, ...

EPFL2019

Pretraining boosts out-of-domain robustness for pose estimation

Alexander Mathis, Mackenzie Mathis, Matthias Bethge, Mert Yüksekgönül

Deep neural networks are highly effective tools for human and animal pose estimation. However, robustness to out-of-domain data remains a challenge. Here, we probe the transfer and generalization ability for pose estimation with two architecture classes (M ...

2019

A jamming transition from under- to over-parametrization affects generalization in deep learning

Graph Chatbot

Chat with Graph Search

Three-dimensional tomography of red blood cells using deep learning

Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions

Supplementary Material - AL2: Progressive Activation Loss for Learning General Representations in Classification Neural Networks

Crowding and the Architecture of the Visual System

Neural Anisotropy Directions

Square Attack: a query-efficient black-box adversarial attack via random search

Shape Reconstruction by Learning Differentiable Surface Representations

Biologically plausible deep learning – but how far can we go with shallow networks?

Evaluating and Interpreting Deep Convolutional Neural Networks via Non-negative Matrix Factorization

Pretraining boosts out-of-domain robustness for pose estimation

Three-dimensional tomography of red blood cells using deep learning

Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions

Neural Anisotropy Directions

Crowding and the Architecture of the Visual System

Square Attack: a query-efficient black-box adversarial attack via random search

Supplementary Material - AL2: Progressive Activation Loss for Learning General Representations in Classification Neural Networks

Shape Reconstruction by Learning Differentiable Surface Representations

Pretraining boosts out-of-domain robustness for pose estimation

Biologically plausible deep learning – but how far can we go with shallow networks?

Evaluating and Interpreting Deep Convolutional Neural Networks via Non-negative Matrix Factorization