Learning Entailment-Based Sentence Embeddings from Natural Language Inference

Large datasets on natural language inference are a potentially valuable resource for inducing semantic representations of natural language sentences. But in many such models the embeddings computed by the sentence encoder goes through an MLP-based interaction layer before predicting its label, and thus some of the information about textual entailment is encoded in the interpretation of sentence embeddings given by this parameterised MLP. In this work we propose a simple interaction layer based on predefined entailment and contradiction scores applied directly to the sentence embeddings. This parameter-free interaction model achieves results on natural language inference competitive with MLP-based models, demonstrating that the trained sentence embeddings directly represent the information needed for textual entailment. The inductive bias of this model leads to better generalisation to other natural language inference datasets, and the resulting sentence embeddings are better for other semantic tasks.

Learning computationally efficient static word and sentence representations

Prakhar Gupta

Most of the Natural Language Processing (NLP) algorithms involve use of distributed vector representations of linguistic units (primarily words and sentences) also known as embeddings in one way or another. These embeddings come in two flavours namely, static/non-contextual and contextual. In a static embedding, the vector representation of a word is independent of its context as opposed to a contextual embedding where the word representation incorporates additional information from its surrounding context.Recently, advancements in deep learning when applied to contextual embeddings have seen them outperforming their static counterparts. However, this improvement in performance with respect to that of the static embeddings has come at the cost of lesser computational efficiency in terms of both computational resources as well as training and inference times, relative lack of interpretability, and higher costs to the environment. Consequently, static embedding models despite not being as expressive and powerful as contextual embedding models continue to be of relevance in Natural Language Processing Research.In this thesis, we propose improvements to the current state-of-the-art static word embedding and sentence embedding models in three different settings. Firstly, we propose an improved algorithm to learn word and sentence embedding from raw text by proposing changes to the Word2Vec training objective formulation and adding n-grams to the training to incorporate local contextual information. Consequently, we end up obtaining improved unsupervised static word and sentence embeddings. Our second major contribution is learning cross-lingual static word and sentence representations from parallel bilingual data where two corpora are aligned sentence-wise. Our word and sentence embeddings thus obtained outperform other bag-of-words bilingual embeddings on cross-lingual sentence retrieval and monolingual word similarity tasks while staying competitive with them on cross-lingual word translation tasks. In our last major contribution, we aim towards harnessing the expressive power of the contextual embedding models by distilling static word embeddings from contextual embedding models to use improved word representations for computationally light tasks. This allows us to utilize the semantic information possessed by the contextual embedding models while maintaining computational efficiency for inference tasks at the same time.

EPFL2021

Answering questions about archived, annotated meetings

Marita Ailomaa

Retrieving information from archived meetings is a new domain of information retrieval that has received increasing attention in the past few years. Search in spontaneous spoken conversations has been recognized as more difficult than text-based document retrieval because meeting discussions contain two levels of information: the content itself, i.e. what topics are discussed, but also the argumentation process, i.e. what conflicts are resolved and what decisions are made. To capture the richness of information in meetings, current research focuses on recording meetings in Smart-Rooms, transcribing meeting discussion into text and annotating discussion with semantic higher-level structures to allow for efficient access to the data. However, it is not yet clear what type of user interface is best suited for searching and browsing such archived, annotated meetings. Content-based retrieval with keyword search is too naive and does not take into account the semantic annotations on the data. The objective of this thesis is to assess the feasibility and usefulness of a natural language interface to meeting archives that allows users to ask complex questions about meetings and retrieve episodes of meeting discussions based on semantic annotations. The particular issues that we address are: the need of argumentative annotation to answer questions about meetings; the linguistic and domain-specific natural language understanding techniques required to interpret such questions; and the use of visual overviews of meeting annotations to guide users in formulating questions. To meet the outlined objectives, we have annotated meetings with argumentative structure and built a prototype of a natural language understanding engine that interprets questions based on those annotations. Further, we have performed two sets of user experiments to study what questions users ask when faced with a natural language interface to annotated meeting archives. For this, we used a simulation method called Wizard of Oz, to enable users to express questions in their own terms without being influenced by limitations in speech recognition technology. Our experimental results show that technically it is feasible to annotate meetings and implement a deep-linguistic NLU engine for questions about meetings, but in practice users do not consistently take advantage of these features. Instead they often search for keywords in meetings. When visual overviews of the available annotations are provided, users refer to those annotations in their questions, but the complexity of questions remains simple. Users search with a breadth-first approach, asking questions in sequence instead of a single complex question. We conclude that natural language interfaces to meeting archives are useful, but that more experimental work is needed to find ways to incent users to take advantage of the expressive power of natural language when asking questions about meetings.

EPFL2009

Towards using slide information to enhance speech transcription of meetings

Hervé Bourlard, Artem Peregoudov, Alessandro Vinciarelli

In this paper we investigate the possibility of improving the speech recognition performance of meeting recordings by using slides captured during the recording process. The key hypothesis exploited in this work is that both slides and speech carry correlated contextual and semantic information. Thus, we propose an approach using the information extracted from slides aimed at reducing the speech recognition word error rate. The N-Best lists output by the recogniser are rescored through Information Retrieval techniques to maximise the similarity between speech and slides transcripts. Results obtained on three meeting recordings (for a total duration of about 90 minutes) show no statistically significant variation of the word error rate. Additional studies provide further insight based on both language properties and statistics of the word distributions in the two sources.