Publication

Stop Pre-Training: Adapt Visual-Language Models to Unseen Languages

Publications associées (35)

Graph Chatbot

Chattez avec Graph Search

Posez n’importe quelle question sur les cours, conférences, exercices, recherches, actualités, etc. de l’EPFL ou essayez les exemples de questions ci-dessous.

AVERTISSEMENT : Le chatbot Graph n'est pas programmé pour fournir des réponses explicites ou catégoriques à vos questions. Il transforme plutôt vos questions en demandes API qui sont distribuées aux différents services informatiques officiellement administrés par l'EPFL. Son but est uniquement de collecter et de recommander des références pertinentes à des contenus que vous pouvez explorer pour vous aider à répondre à vos questions.

Connectez-vous pour utiliser Chat avec Graph Search

Direction is what you need: Improving Word Embedding Compression in Large Language Models

Karl Aberer, Rémi Philippe Lebret, Mohammadreza Banaei

The adoption of Transformer-based models in natural language processing (NLP) has led to great success using a massive number of parameters. However, due to deployment constraints in edge devices, there has been a rising interest in the compression of thes ...

ASSOC COMPUTATIONAL LINGUISTICS-ACL2021

Fidelity Estimation Improves Noisy-Image Classification with Pretrained Networks

Sabine Süsstrunk, Majed El Helou, Deblina Bhattacharjee, Xiaoyu Lin

Image classification has significantly improved using deep learning. This is mainly due to convolutional neural networks (CNNs) that are capable of learning rich feature extractors from large datasets. However, most deep learning classification methods are ...

2021

On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines

Maksym Andriushchenko

Fine-tuning pre-trained transformer-based language models such as BERT has become a common practice dominating leaderboards across various NLP benchmarks. Despite the strong empirical performance of fine-tuned models, fine-tuning is an unstable process: tr ...

2021

, , , ,

As the labeling cost for different modules in task-oriented dialog (ToD) systems is expensive, a major challenge is to train different modules with the least amount of labeled data. Recently, large-scale pre-trained language models, have shown promising re ...

ASSOC COMPUTATIONAL LINGUISTICS-ACL2021

The Unstoppable Rise of Computational Linguistics in Deep Learning

James Henderson

In this paper, we trace the history of neural networks applied to natural language understanding tasks, and identify key contributions which the nature of language has made to the development of neural network architectures. We focus on the importance of v ...

Association for Computational Linguistics2020

, ,

Large datasets on natural language inference are a potentially valuable resource for inducing semantic representations of natural language sentences. But in many such models the embeddings computed by the sentence encoder goes through an MLP-based interact ...

Idiap2019

, , , , ,

We identify a phenomenon, which we refer to as multi-model forgetting, that occurs when sequentially training multiple deep networks with partially-shared parameters; the performance of previously-trained models degrades as one optimizes a subsequent one, ...

JMLR2019

Natural language processing techniques are dependent upon punctuation to work well. When their input is taken from speech recognition, it is necessary to reconstruct the punctuation; in particular sentence boundaries. We define a range of features from low ...

2019

Idiap2019

How to Tell Ancient Signs Apart? Recognizing and Visualizing Maya Glyphs with CNNs

Daniel Gatica-Perez, Jean-Marc Odobez, Gülcan Can

Thanks to the digital preservation of cultural heritage materials, multimedia tools (e.g., based on automatic visual processing) considerably ease the work of scholars in the humanities and help them to perform quantitative analysis of their data. In this ...

2018