Publication

CBOW Is Not All You Need: Combining CBOW with the Compositional Matrix Space Model

Related publications (32)

Graph Chatbot

Chat with Graph Search

Ask any question about EPFL courses, lectures, exercises, research, news, etc. or try the example questions below.

DISCLAIMER: The Graph Chatbot is not programmed to provide explicit or categorical answers to your questions. Rather, it transforms your questions into API requests that are distributed across the various IT services officially administered by EPFL. Its purpose is solely to collect and recommend relevant references to content that you can explore to help you answer your questions.

Text Representation Learning for Low Cost Natural Language Understanding

Jan Frederik Jonas Florian Mai

Natural language processing and other artificial intelligence fields have witnessed impressive progress over the past decade. Although some of this progress is due to algorithmic advances in deep learning, the majority has arguably been enabled by scaling ...

EPFL2023

Word-level Embeddings for Cross-Task Transfer Learning in Speech Processing

Milos Cernak, Pierre Anton Beckmann

Recent breakthroughs in deep learning often rely on representation learning and knowledge transfer. In recent years, unsupervised and self-supervised techniques for learning speech representation were developed to foster automatic speech recognition. Up to ...

EUROPEAN ASSOC SIGNAL SPEECH & IMAGE PROCESSING-EURASIP2021

Further results on latent discourse models and word embeddings

Youssef Allouah

We discuss some properties of generative models for word embeddings. Namely, (Arora et al., 2016) proposed a latent discourse model implying the concentration of the partition function of the word vectors. This concentration phenomenon led to an asymptotic ...

MICROTOME PUBL2021

L'hybridation, catalyseur de mixité fonctionnelle et sociale pour le quartier de Grosselin

A Genève, comme dans de nombreuses métropoles, la problématique de la gestion des bureaux vacants est primordiale et se traduit par de nombreuses surfaces de travail qui ne trouvent pas (ou plus) preneur. Alors que la ville ne cesse de se densifier, il par ...

2020

Concept Discovery for The Interpretation of Landscape Scenicness

Devis Tuia

In this paper, we study how to extract visual concepts to understand landscape scenicness. Using visual feature representations from a Convolutional Neural Network (CNN), we learn a number of Concept Activation Vectors (CAV) aligned with semantic concepts ...

2020

CBOW Is Not All You Need: Combining CBOW with the Compositional Matrix Space Model

Jan Frederik Jonas Florian Mai

Continuous Bag of Words (CBOW) is a powerful text embedding method. Due to its strong capabilities to encode word content, CBOW embeddings perform well on a wide range of downstream tasks while being efficient to compute. However, CBOW is not capable of ca ...

2019

On the Effect of Word Order on Cross-lingual Sentiment Analysis

Current state-of-the-art models for sentiment analysis make use of word order either explicitly by pre-training on a language modeling objective or implicitly by using recurrent neural networks (RNNS) or convolutional networks (CNNS). This is a problem for ...

2019

Better Word Embeddings by Disentangling Contextual n-Gram Information

Martin Jaggi, Matteo Pagliardini, Prakhar Gupta

Pre-trained word vectors are ubiquitous in Natural Language Processing applications. In this paper, we show how training word embeddings jointly with bigram and even trigram embeddings, results in improved unigram embeddings. We claim that training word em ...

2019

A novel biomechanical approach for animal behaviour recognition using accelerometers

Kamiar Aminian, Pritish Chakravarty

Data from animal-borne inertial sensors are widely used to investigate several aspects of an animal's life, such as energy expenditure, daily activity patterns and behaviour. Accelerometer data used in conjunction with machine learning algorithms have b ...

2019

Learning Word Vectors for 157 Languages

Prakhar Gupta, Edouard Grave

Distributed word representations, or word vectors, have recently been applied to many tasks in natural language processing, leading to state-of-the-art performance. A key ingredient to the successful application of these representations is to train them on ...

2018