Publications associées à The committee machine: Computational to statistical gaps in learning a two-layers neural network

Disentangling feature and lazy training in deep neural networks

Matthieu Wyart, Mario Geiger, Stefano Spigler, Arthur Jacot

Two distinct limits for deep learning have been derived as the network width h -> infinity, depending on how the weights of the last layer scale with h. In the neural tangent Kernel (NTK) limit, the dynamics becomes linear in the weights and is described b ...

IOP PUBLISHING LTD2020

Introducing reinforcement learning to the energy system design process

Jean-Louis Scartezzini, Amarasinghage Tharindu Dasun Perera, Vahid Moussavi Nik

Design optimization of distributed energy systems has become an interest of a wider group of researchers due the capability of these systems to integrate non-dispatchable renewable energy technologies such as solar PV and wind. White box models, using line ...

2020

Supply-Power-Constrained Cable Capacity Maximization Using Multi-Layer Neural Networks

Erixhen Sula

We experimentally solve the problem of maximizing capacity under a total supply power constraint in a massively parallel submarine cable context, i.e., for a spatially uncoupled system in which fiber Kerr nonlinearity is not a dominant limitation. By using ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2020

AL2: Progressive Activation Loss for Learning General Representations in Classification Neural Networks

Sabine Süsstrunk, Majed El Helou, Frederike Dümbgen

The large capacity of neural networks enables them to learn complex functions. To avoid overfitting, networks however require a lot of training data that can be expensive and time-consuming to collect. A common practical approach to attenuate overfitting i ...

IEEE2020

The Unstoppable Rise of Computational Linguistics in Deep Learning

James Henderson

In this paper, we trace the history of neural networks applied to natural language understanding tasks, and identify key contributions which the nature of language has made to the development of neural network architectures. We focus on the importance of v ...

Association for Computational Linguistics2020

Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimization

Florent Gérard Krzakala, Lenka Zdeborová

We consider a commonly studied supervised classification of a synthetic dataset whose labels are generated by feeding a one-layer non-linear neural network with random iid inputs. We study the generalization performances of standard classifiers in the high ...

Curran Associates, Inc.2020

Pedestrian Intention Prediction: A Multi-task Perspective

Alexandre Massoud Alahi, Saeed Saadatnejad

In order to be globally deployed, autonomous cars must guarantee the safety of pedestrians. This is the reason why forecasting pedestrians' intentions sufficiently in advance is one of the most critical and challenging tasks for autonomous vehicles. This w ...

2020

The committee machine: computational to statistical gaps in learning a two-layers neural network

Nicolas Macris, Florent Gérard Krzakala, Lenka Zdeborová, Jean François Emmanuel Barbier

Heuristic tools from statistical physics have been used in the past to locate the phase transitions and compute the optimal learning and generalization errors in the teacher-student scenario in multi-layer neural networks. In this paper, we provide a rigor ...

IOP PUBLISHING LTD2019

Biologically plausible deep learning – but how far can we go with shallow networks?

Wulfram Gerstner, Johanni Michael Brea, Bernd Albert Illing

Training deep neural networks with the error backpropagation algorithm is considered implausible from a biological perspective. Numerous recent publications suggest elaborate models for biologically plausible variants of deep learning, typically defining s ...

2019

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

The committee machine: Computational to statistical gaps in learning a two-layers neural network

Disentangling feature and lazy training in deep neural networks

Introducing reinforcement learning to the energy system design process

Supply-Power-Constrained Cable Capacity Maximization Using Multi-Layer Neural Networks

AL2: Progressive Activation Loss for Learning General Representations in Classification Neural Networks

The Unstoppable Rise of Computational Linguistics in Deep Learning

Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimization

Pedestrian Intention Prediction: A Multi-task Perspective

Detection of Bad PMU Data using Machine Learning Techniques

The committee machine: computational to statistical gaps in learning a two-layers neural network

Biologically plausible deep learning – but how far can we go with shallow networks?

Graph Chatbot

Chattez avec Graph Search

Disentangling feature and lazy training in deep neural networks

Biologically plausible deep learning – but how far can we go with shallow networks?

The committee machine: computational to statistical gaps in learning a two-layers neural network

Detection of Bad PMU Data using Machine Learning Techniques

Pedestrian Intention Prediction: A Multi-task Perspective

Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimization

The Unstoppable Rise of Computational Linguistics in Deep Learning

AL2: Progressive Activation Loss for Learning General Representations in Classification Neural Networks

Supply-Power-Constrained Cable Capacity Maximization Using Multi-Layer Neural Networks

Introducing reinforcement learning to the energy system design process