Publication

Is Deep Learning Really Necessary for Word Embeddings?

Related publications (43)

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Learning computationally efficient static word and sentence representations

Prakhar Gupta

Most of the Natural Language Processing (NLP) algorithms involve use of distributed vector representations of linguistic units (primarily words and sentences) also known as embeddings in one way or another. These embeddings come in two flavours namely, sta ...

EPFL2021

Further results on latent discourse models and word embeddings

Youssef Allouah

We discuss some properties of generative models for word embeddings. Namely, (Arora et al., 2016) proposed a latent discourse model implying the concentration of the partition function of the word vectors. This concentration phenomenon led to an asymptotic ...

MICROTOME PUBL2021

Self-Supervised Neural Topic Modeling

Martin Jaggi

Topic models are useful tools for analyzing and interpreting the main underlying themes of large corpora of text. Most topic models rely on word co-occurrence for computing a topic, i.e., a weighted set of words that together represent a high-level semanti ...

Assoc Computational Linguistics-Acl2021

The Unstoppable Rise of Computational Linguistics in Deep Learning

James Henderson

In this paper, we trace the history of neural networks applied to natural language understanding tasks, and identify key contributions which the nature of language has made to the development of neural network architectures. We focus on the importance of v ...

Association for Computational Linguistics2020

Concept Discovery for The Interpretation of Landscape Scenicness

Devis Tuia

In this paper, we study how to extract visual concepts to understand landscape scenicness. Using visual feature representations from a Convolutional Neural Network (CNN), we learn a number of Concept Activation Vectors (CAV) aligned with semantic concepts ...

2020

Learning Entailment-Based Sentence Embeddings from Natural Language Inference

James Henderson, Rabeeh Karimi Mahabadi, Jan Frederik Jonas Florian Mai

Large datasets on natural language inference are a potentially valuable resource for inducing semantic representations of natural language sentences. But in many such models the embeddings computed by the sentence encoder goes through an MLP-based interact ...

Idiap2019

Better Word Embeddings by Disentangling Contextual n-Gram Information

Martin Jaggi, Matteo Pagliardini, Prakhar Gupta

Pre-trained word vectors are ubiquitous in Natural Language Processing applications. In this paper, we show how training word embeddings jointly with bigram and even trigram embeddings, results in improved unigram embeddings. We claim that training word em ...

2019

Implicit discourse relation classification with syntax-aware contextualized word representations

James Henderson

Automatically identifying implicit discourse relations requires an in-depth semantic understanding of the text fragments involved in such relations. While early work investigated the usefulness of different classes of input features, current state-of-the-a ...

2019

Overcoming Multi-model Forgetting

Mathieu Salzmann, Anthony Christopher Davison, Martin Jaggi, Yassine Benyahia, Claudiu-Cristian Musat, Kaicheng Yu

We identify a phenomenon, which we refer to as multi-model forgetting, that occurs when sequentially training multiple deep networks with partially-shared parameters; the performance of previously-trained models degrades as one optimizes a subsequent one, ...

JMLR2019

On the Effect of Word Order on Cross-lingual Sentiment Analysis

Current state-of-the-art models for sentiment analysis make use of word order either explicitly by pre-training on a language modeling objective or implicitly by using recurrent neural networks (RNNS) or convolutional networks (CNNS). This is a problem for ...

2019