Publications associées à Overcoming Multi-model Forgetting

Global information processing in feedforward deep networks

Michael Herzog, Ben Henrik Lönnqvist, Adrien Christophe Doerig, Alban Bornet

While deep neural networks are state-of-the-art models of many parts of the human visual system, here we show that they fail to process global information in a humanlike manner. First, using visual crowding as a probe into global visual information process ...

2022

Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Zhenyu Zhu, Yongtao Wu

Neural tangent kernel (NTK) is a powerful tool to analyze training dynamics of neural networks and their generalization bounds. The study on NTK has been devoted to typical neural network architectures, but it is incomplete for neural networks with Hadamar ...

2022

RayTran: 3D Pose Estimation and Shape Reconstruction of Multiple Objects from Videos with Ray-Traced Transformers

Michal Jan Tyszkiewicz, Vittorio Ferrari, Kevis-Kokitsi Maninis

We propose a transformer-based neural network architecture for multi-object 3D reconstruction from RGB videos. It relies on two alternative ways to represent its knowledge: as a global 3D grid of features and an array of view-specific 2D grids. We progress ...

Springer2022

Generalization of an Encoder-Decoder LSTM model for flood prediction in ungauged catchments

Olga Fink, Yi Zhang, Peter Molnar, Silvan Ragettli

Flood prediction in ungauged catchments is usually conducted by hydrological models that are parameterized based on nearby and similar gauged catchments. As an alternative to this process-based modelling, deep learning (DL) models have demonstrated their a ...

ELSEVIER2022

An Analysis of Super-Net Heuristics in Weight-Sharing NAS

Mathieu Salzmann, Kaicheng Yu

Weight sharing promises to make neural architecture search (NAS) tractable even on commodity hardware. Existing methods in this space rely on a diverse set of heuristics to design and train the shared-weight backbone network, a.k.a. the super-net. Since he ...

IEEE COMPUTER SOC2022

Towards Robust Vision Transformer

Shaokai Ye, Yuan He

Recent advances on Vision Transformer (ViT) and its improved variants have shown that self-attention-based networks surpass traditional Convolutional Neural Networks (CNNs) in most vision tasks. However, existing ViTs focus on the standard accuracy and com ...

IEEE COMPUTER SOC2022

Constrained Language Models for Interactive Poem Generation

Andrei Popescu-Belis, Gabriel Luthier, Aris Xanthos, Simon Mattei

This paper describes a system for interactive poem generation, which combines neural language models (LMs) for poem generation with explicit constraints that can be set by users on form, topic, emotion, and rhyming scheme. LMs cannot learn such constraints ...

EUROPEAN LANGUAGE RESOURCES ASSOC-ELRA2022

Pretraining boosts out-of-domain robustness for pose estimation

Alexander Mathis, Mackenzie Mathis, Steffen Schneider, Matthias Bethge, Thomas Ray Biasi, Mert Yüksekgönül

Neural networks are highly effective tools for pose estimation. However, as in other computer vision tasks, robustness to out-of-domain data remains a challenge, especially for small training sets that are common for real-world applications. Here, we probe ...

IEEE COMPUTER SOC2021

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

Overcoming Multi-model Forgetting

Global information processing in feedforward deep networks

Stop Wasting my FLOPS: Improving the Efficiency of Deep Learning Models

Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study

RayTran: 3D Pose Estimation and Shape Reconstruction of Multiple Objects from Videos with Ray-Traced Transformers

Generalization of an Encoder-Decoder LSTM model for flood prediction in ungauged catchments

An Analysis of Super-Net Heuristics in Weight-Sharing NAS

Towards Robust Vision Transformer

Constrained Language Models for Interactive Poem Generation

From Human-Designed Convolutional Neural Networks Towards Robust Neural Architecture Search

Pretraining boosts out-of-domain robustness for pose estimation

Graph Chatbot

Chattez avec Graph Search

Stop Wasting my FLOPS: Improving the Efficiency of Deep Learning Models

Global information processing in feedforward deep networks

Generalization of an Encoder-Decoder LSTM model for flood prediction in ungauged catchments

RayTran: 3D Pose Estimation and Shape Reconstruction of Multiple Objects from Videos with Ray-Traced Transformers

Constrained Language Models for Interactive Poem Generation

Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study

Towards Robust Vision Transformer

An Analysis of Super-Net Heuristics in Weight-Sharing NAS

From Human-Designed Convolutional Neural Networks Towards Robust Neural Architecture Search

Pretraining boosts out-of-domain robustness for pose estimation