Publications associées à Vision Transformer Adapters for Generalizable Multitask Learning

Modeling Structured Data in Attention-based Models

Natural language processing has experienced significant improvements with the development of Transformer-based models, which employ self-attention mechanism and pre-training strategies. However, these models still present several obstacles. A notable issue ...

EPFL2023

One-shot Garbage Collection for In-memory OLTP through Temporality-aware Version Storage

Anastasia Ailamaki, Syed Mohammad Aunn Raza, Periklis Chrysogelos, Angelos Christos Anadiotis

Most modern in-memory online transaction processing (OLTP) engines rely on multi-version concurrency control (MVCC) to provide data consistency guarantees in the presence of conflicting data accesses. MVCC improves concurrency by generating a new version o ...

ACM2023

Linear Complexity Self-Attention With 3rd Order Polynomials

Grigorios Chrysos, Filippos Kokkinos

Self-attention mechanisms and non-local blocks have become crucial building blocks for state-of-the-art neural architectures thanks to their unparalleled ability in capturing long-range dependencies in the input. However their cost is quadratic with the nu ...

Ieee Computer Soc2023

Transformer Models for Vision

Jean-Baptiste Francis Marie Juliette Cordonnier

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins. All these successful methods depend on a gradient-based learning algorithm to train a model on massive ...

EPFL2023

Dense Image-based Predictions for Comics Analysis

Deblina Bhattacharjee

Dense image-based prediction methods have advanced tremendously in recent years. Their remarkable development has been possible due to the ample availability of real-world imagery. While these methods work well on photographs, their abilities do not genera ...

EPFL2023

Concurrency control for database theorists

Christoph Koch, Bas Ketsman

The aim of this paper is to serve as a lightweight introduction to concurrency control for database theorists through a uniform presentation of the work on robustness against Multiversion Read Committed and Snapshot Isolation. ...

ASSOC COMPUTING MACHINERY2022

Remote sensing visual question answering with a self-attention multi-modal encoder

Devis Tuia, Joao Magalhaes

Visual Question Answering (VQA) on remote sensing imagery can help non-expert users in extracting information from Earth observation data. Current approaches follow a neural encoder-decoder design, combining convolutional and recurrent encoders together wi ...

2022

Efficient Transformer-Based Speech Recognition

Apoorv Vyas

Training deep neural network based Automatic Speech Recognition (ASR) models often requires thousands of hours of transcribed data, limiting their use to only a few languages. Moreover, current state-of-the-art acoustic models are based on the Transformer ...

EPFL2022

STATNet: Spatial-temporal attention in the traffic prediction

Alexandre Massoud Alahi, Seyed Mohamad Moghadas, Amin Gheibi

Recent traffic flow prediction methods are lacking abilities to determine predictive features. Thus, they will propagate the error in the next timestamps. In this paper, first, we assess the role of spatial and temporal features on the traffic speed predic ...

2022

On the Relationship between Self-Attention and Convolutional Layers

Martin Jaggi, Jean-Baptiste Francis Marie Juliette Cordonnier, Andreas Loukas

Recent trends of incorporating attention mechanisms in vision have led re- searchers to reconsider the supremacy of convolutional layers as a primary build- ing block. Beyond helping CNNs to handle long-range dependencies, Ramachandran et al. (2019) showed ...

2020