Publication

Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models

Related publications (54)

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Transformer-Based Multi-lingual Sentence Embeddings

In this thesis, we present a transformers-based multi-lingual embedding model to represent sentences in different languages in a common space. To do so, our system uses the structure of a simplified transformer with a shared byte-pair encoding vocabulary f ...

2019

Learning Word Vectors for 157 Languages

Prakhar Gupta, Edouard Grave

Distributed word representations, or word vectors, have recently been applied to many tasks in natural language processing, leading to state-of-the-art performance. A key ingredient to the successful application of these representations is to train them on ...

2018

Structured Sequence Modeling with Graph Convolutional Recurrent Networks

Pierre Vandergheynst, Xavier Bresson, Michaël Defferrard, Youngjoo Seo

This paper introduces Graph Convolutional Recurrent Network (GCRN), a deep learning model able to predict structured sequences of data. Precisely, GCRN is a generalization of classical recurrent neural networks (RNN) to data structured by an arbitrary grap ...

2017

Word Sequence Modeling using Deep Learning

Joël Yvon Roland Legrand

For a long time, natural language processing (NLP) has relied on generative models with task specific and manually engineered features. Recently, there has been a resurgence of interest for neural networks in the machine learning community, obtaining state ...

EPFL2016

Building Word Embeddings for Solving Natural Language Processing

Rémi Philippe Lebret

Word embedding is a feature learning technique which aims at mapping words from a vocabulary into vectors of real numbers in a low-dimensional space. By leveraging large corpora of unlabeled text, such continuous space representations can be computed for c ...

École Polytechnique Fédérale de Lausanne2016

Word Embeddings for Natural Language Processing

Rémi Philippe Lebret

EPFL2016

Twitter Sentiment Analysis (Almost) from Scratch

Rémi Philippe Lebret, Ronan Collobert, Pedro Henrique Oliveira Pinheiro

A popular application in Natural Language Processing (NLP) is the Sentiment Analysis (SA), i.e., the task of extracting contextual polarity from a given text. The social network Twitter provides an immense amount of text (called tweets) generated by users ...

Idiap2016

Simple Image Description Generator via a Linear Phrase-based Model

Rémi Philippe Lebret, Ronan Collobert, Pedro Henrique Oliveira Pinheiro

Generating a novel textual description of an image is an interesting problem that connects computer vision and natural language processing. In this paper, we present a simple model that is able to generate descriptive sentences given a sample image. This m ...

Idiap2015

Phrase-based Image Captioning

Rémi Philippe Lebret, Ronan Collobert, Pedro Henrique Oliveira Pinheiro

2015

Phrase-based Image Captioning

Rémi Philippe Lebret, Ronan Collobert, Pedro Henrique Oliveira Pinheiro

JMLR2015