Publications associées à Transformeur

A machine learning architecture for including wave breaking in envelope-type wave models

Wave breaking is a complex physical process about which open questions remain. For some applications, it is critical to include breaking effects in phase-resolved envelope-based wave models such as the non-linear Schr & ouml;dinger. A promising approach is ...

Pergamon-Elsevier Science Ltd2024

Transformer Models for Vision

Jean-Baptiste Francis Marie Juliette Cordonnier

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins.All these successful methods depend on a gradient-based learning algorithm to train a model on massive a ...

EPFL2023

Vision Transformer Adapters for Generalizable Multitask Learning

Sabine Süsstrunk, Mathieu Salzmann, Deblina Bhattacharjee

We introduce the first multitasking vision transformer adapters that learn generalizable task affinities which can be applied to novel tasks and domains. Integrated into an off-the-shelf vision transformer backbone, our adapters can simultaneously solve mu ...

2023

Source-Free Open-Set Domain Adaptation for Histopathological Images via Distilling Self-Supervised Vision Transformer

Jean-Philippe Thiran, Guillaume Marc Georges Vray, Devavrat Tomar

There is a strong incentive to develop computational pathology models to i) ease the burden of tissue typology annotation from whole slide histological images; ii) transfer knowledge, e.g., tissue class separability from the withheld source domain to the d ...

2023

Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models

Karl Aberer, Rémi Philippe Lebret, Mohammadreza Banaei

Recent transformer language models achieve outstanding results in many natural language processing (NLP) tasks. However, their enormous size often makes them impractical on memory-constrained devices, requiring practitioners to compress them to smaller net ...

Assoc Computational Linguistics-Acl2023

An attention-based deep learning approach for the classification of subjective cognitive decline and mild cognitive impairment using resting-state EEG

Silvestro Micera, Michael Lassi

Objective. This study aims to design and implement the first deep learning (DL) model to classify subjects in the prodromic states of Alzheimer's disease (AD) based on resting-state electroencephalographic (EEG) signals. Approach. EEG recordings of 17 heal ...

IOP Publishing Ltd2023

CLIP the Gap: A Single Domain Generalization Approach for Object Detection

Mathieu Salzmann, Martin Pierre Engilberge, Vidit Vidit

Single Domain Generalization (SDG) tackles the problem of training a model on a single source domain so that it generalizes to any unseen target domain. While this has been well studied for image classification, the literature on SDG object detection remai ...

Los Alamitos2023

A multi-modal pre-training transformer for universal transfer learning in metal-organic frameworks

Berend Smit, Hyunsoo Park

Metal-organic frameworks (MOFs) are a class of crystalline porous materials that exhibit a vast chemical space owing to their tunable molecular building blocks with diverse topologies. An unlimited number of MOFs can, in principle, be synthesized. Machine ...

NATURE PORTFOLIO2023

Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learning

Alexandre Massoud Alahi, Yuejiang Liu

Recent years have seen a surge of interest in learning high-level causal representations from low-level image pairs under interventions. Yet, existing efforts are largely limited to simple synthetic settings that are far away from real-world problems. In t ...

2023

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

Transformeur

A machine learning architecture for including wave breaking in envelope-type wave models

Transformer Models for Vision

Vision Transformer Adapters for Generalizable Multitask Learning

Source-Free Open-Set Domain Adaptation for Histopathological Images via Distilling Self-Supervised Vision Transformer

The autoregressive neural network architecture of the Boltzmann distribution of pairwise interacting spins systems

Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models

An attention-based deep learning approach for the classification of subjective cognitive decline and mild cognitive impairment using resting-state EEG

CLIP the Gap: A Single Domain Generalization Approach for Object Detection

A multi-modal pre-training transformer for universal transfer learning in metal-organic frameworks

Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learning

Graph Chatbot

Chattez avec Graph Search

A multi-modal pre-training transformer for universal transfer learning in metal-organic frameworks

An attention-based deep learning approach for the classification of subjective cognitive decline and mild cognitive impairment using resting-state EEG

The autoregressive neural network architecture of the Boltzmann distribution of pairwise interacting spins systems

Source-Free Open-Set Domain Adaptation for Histopathological Images via Distilling Self-Supervised Vision Transformer

Vision Transformer Adapters for Generalizable Multitask Learning

Transformer Models for Vision

A machine learning architecture for including wave breaking in envelope-type wave models

CLIP the Gap: A Single Domain Generalization Approach for Object Detection

Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learning

Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models