Publication

Subquadratic Overparameterization for Shallow Neural Networks

Publications associées (33)

Graph Chatbot

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

Connectez-vous pour utiliser Chat avec Graph Search

Fixing the problems of deep neural networks will require better training data and learning algorithms

Martin Schrimpf, Adrien Christophe Doerig, Matthias Bethge, Jianghao Liu, Kuntal Ghosh

Bowers et al. argue that deep neural networks (DNNs) are poor models of biological vision because they often learn to rival human accuracy by relying on strategies that differ markedly from those of humans. We show that this problem is worsening as DNNs ar ...

Cambridge2023

Polynomial-time universality and limitations of deep learning

Emmanuel Abbé

The goal of this paper is to characterize function distributions that general neural networks trained by descent algorithms (GD/SGD), can or cannot learn in polytime. The results are: (1) The paradigm of general neural networks trained by SGD is poly-time ...

WILEY2023

Theory of Deep Learning: Neural Tangent Kernel and Beyond

Arthur Ulysse Jacot-Guillarmod

In the recent years, Deep Neural Networks (DNNs) have managed to succeed at tasks that previously appeared impossible, such as human-level object recognition, text synthesis, translation, playing games and many more. In spite of these major achievements, o ...

EPFL2022

Stop Wasting my FLOPS: Improving the Efficiency of Deep Learning Models

Angelos Katharopoulos

Deep neural networks have completely revolutionized the field of machinelearning by achieving state-of-the-art results on various tasks ranging fromcomputer vision to protein folding. However, their application is hindered bytheir large computational and m ...

EPFL2022

On the robustness of randomized classifiers to adversarial examples

Rafaël Benjamin Pinot

This paper investigates the theory of robustness against adversarial attacks. We focus on randomized classifiers (i.e. classifiers that output random variables) and provide a thorough analysis of their behavior through the lens of statistical learning theo ...

SPRINGER2022

Robust Binary Models by Pruning Randomly-initialized Networks

Sabine Süsstrunk, Mathieu Salzmann, Chen Liu, Ziqi Zhao

Robustness to adversarial attacks was shown to require a larger model capacity, and thus a larger memory footprint. In this paper, we introduce an approach to obtain robust yet compact models by pruning randomly-initialized binary networks. Unlike adversar ...

2022

ADAGRAD Avoids Saddle Points

Kimon Antonakopoulos, Xiao Wang

Adaptive first-order methods in optimization are prominent in machine learning and data science owing to their ability to automatically adapt to the landscape of the function being optimized. However, their convergence guarantees are typically stated in te ...

2022

Biologically plausible unsupervised learning in shallow and deep neural networks

Bernd Albert Illing

The way our brain learns to disentangle complex signals into unambiguous concepts is fascinating but remains largely unknown. There is evidence, however, that hierarchical neural representations play a key role in the cortex. This thesis investigates biolo ...

EPFL2021

The relationship between simulated ion cyclotron emission (ICE) signals s and the corresponding 1D velocity distribution function f(upsilon(perpendicular to)) of the fast ions triggering the ICE is modeled using a two-layer deep neural network. The network ...

AIP Publishing2021

Neural networks (NNs) have been very successful in a variety of tasks ranging from machine translation to image classification. Despite their success, the reasons for their performance are still not well-understood. This thesis explores two main themes: lo ...

EPFL2021