Publications related to Multilayer perceptron

Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention

François Fleuret, Nikolaos Pappas, Angelos Katharopoulos, Apoorv Vyas

Transformers achieve remarkable performance in several tasks but due to their quadratic complexity, with respect to the input’s length, they are prohibitively slow for very long sequences. To address this limitation, we express the self-attention as a line ...

Idiap2020

Disentangling feature and lazy training in deep neural networks

Matthieu Wyart, Mario Geiger, Stefano Spigler, Arthur Jacot

Two distinct limits for deep learning have been derived as the network width h -> infinity, depending on how the weights of the last layer scale with h. In the neural tangent Kernel (NTK) limit, the dynamics becomes linear in the weights and is described b ...

IOP PUBLISHING LTD2020

Second-order guarantees in centralized, federated and decentralized nonconvex optimization

Ali H. Sayed, Stefan Vlaski

Rapid advances in data collection and processing capabilities have allowed for the use of increasingly complex models that give rise to nonconvex optimization problems. These formulations, however, can be arbitrarily difficult to solve in general, in the s ...

INT PRESS BOSTON, INC2020

XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera

Pascal Fua, Helge Jochen Rhodin, Gerard Pons Moll

We present a real-time approach for multi-person 3D motion capture at over 30 fps using a single RGB camera. It operates successfully in generic scenes which may contain occlusions by objects and by other people. Our method operates in subsequent stages. T ...

ASSOC COMPUTING MACHINERY2020

A Measure of Dynamical Efficiency for Multimodal Interconnected Urban Systems

Nikolaos Geroliminis, Leonardo Bellocchi

Complex network theory describes network performance mostly based on topological characteristics, like betweenness centrality. This work integrates concepts from complex networks and traffic engineering. We propose a new measure for spatial networks and, i ...

2020

Scaling description of generalization with number of parameters in deep learning

Matthieu Wyart, Clément Hongler, Mario Geiger, Stefano Spigler, Levent Dogus Sagun, Franck Raymond Gabriel, Arthur Jacot

Supervised deep learning involves the training of neural networks with a large number N of parameters. For large enough N, in the so-called over-parametrized regime, one can essentially fit the training data points. Sparsitybased arguments would suggest th ...

IOP PUBLISHING LTD2020

Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures

Florent Gérard Krzakala, Julien Marcel Daniel Emmanuel Launay

Despite being the workhorse of deep learning, the backpropagation algorithm is no panacea. It enforces sequential layer updates, thus preventing efficient parallelization of the training process. Furthermore, its biological plausibility is being challenged ...

Curran Associates, Inc.2020

Physico-chemical properties prediction of hydrochar in macroalgae Sargassum horneri hydrothermal carbonisation

Mohsen Talebkeikhah, Farzaneh Talebkeikhah

This study evaluates and compares several machine learning methods on the effects of different parameters in the hydrothermal carbonisation (HTC) process of macroalgae Sargassum horneri. Reaction temperature, residence time, biomass particle size, the amou ...

TAYLOR & FRANCIS LTD2019

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Multilayer perceptron

Adaptive Mitigation: Identification of the Dynamic Drivers of Effective Policy during the COVID-19 Pandemic

Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention

Disentangling feature and lazy training in deep neural networks

Second-order guarantees in centralized, federated and decentralized nonconvex optimization

XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera

A Measure of Dynamical Efficiency for Multimodal Interconnected Urban Systems

Scaling description of generalization with number of parameters in deep learning

Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures

Physico-chemical properties prediction of hydrochar in macroalgae Sargassum horneri hydrothermal carbonisation

Enhanced MR Image Classification Using Hybrid Statistical and Wavelets Features

Graph Chatbot

Chat with Graph Search

Enhanced MR Image Classification Using Hybrid Statistical and Wavelets Features

XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera

Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention

Second-order guarantees in centralized, federated and decentralized nonconvex optimization

A Measure of Dynamical Efficiency for Multimodal Interconnected Urban Systems

Physico-chemical properties prediction of hydrochar in macroalgae Sargassum horneri hydrothermal carbonisation

Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures

Adaptive Mitigation: Identification of the Dynamic Drivers of Effective Policy during the COVID-19 Pandemic

Scaling description of generalization with number of parameters in deep learning

Disentangling feature and lazy training in deep neural networks