Natural language generation | EPFL Graph Search

Are you an EPFL student looking for a semester project?

Work with us on data science and visualisation projects, and deploy your project as an app on top of GraphSearch.

Natural language generation (NLG) is a software process that produces natural language output. A widely-cited survey of NLG methods describes NLG as "the subfield of artificial intelligence and computational linguistics that is concerned with the construction of computer systems than can produce understandable texts in English or other human languages from some underlying non-linguistic representation of information". While it is widely agreed that the output of any NLG process is text, there is some disagreement about whether the inputs of an NLG system need to be non-linguistic. Common applications of NLG methods include the production of various reports, for example weather and patient reports; image captions; and chatbots. Automated NLG can be compared to the process humans use when they turn ideas into writing or speech. Psycholinguists prefer the term language production for this process, which can also be described in mathematical terms, or modeled in a computer for psychological research. NLG systems can also be compared to translators of artificial computer languages, such as decompilers or transpilers, which also produce human-readable code generated from an intermediate representation. Human languages tend to be considerably more complex and allow for much more ambiguity and variety of expression than programming languages, which makes NLG more challenging. NLG may be viewed as complementary to natural-language understanding (NLU): whereas in natural-language understanding, the system needs to disambiguate the input sentence to produce the machine representation language, in NLG the system needs to make decisions about how to put a representation into words. The practical considerations in building NLU vs. NLG systems are not symmetrical. NLU needs to deal with ambiguous or erroneous user input, whereas the ideas the system wants to express through NLG are generally known precisely. NLG needs to choose a specific, self-consistent textual representation from many potential representations, whereas NLU generally tries to produce a single, normalized representation of the idea expressed.

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related publications (1)

Related concepts (13)

Related courses (19)

Related lectures (133)

, , ,

Patents have traditionally been used in the history of technology as an indication of the thinking process of the inventors, of the challenges or “reverse salients” they faced, or of the social groups influencing the construction of technology. More recently, historians of science and technology also read them to interpret the way people described technology and how the specific inscriptions of inventions mattered for the justification and operation of the patent system. The digitization of historical patents opens up unique opportunities to assess the feasibility of unsupervised machine learning and natural language methods for such explorations. In this project, we analyze over a million US historical patents from 1830-1930 using a variety of text-based methods, with two major aims: 1) categorizing patents into coherent technical categories, 2) identifying discourses of safety, reflexivity, and environmental concern in technological innovation. We use both frequency-based and context-based methods, and find that a bag of words-based methods such as TDF-IDF and topic modeling do not perform well on semantic categorization due to the linguistic peculiarities of patent specifications. This suggests that a successful approach to categorizing patents would require contextual semantic representations such as Transformers-based methods (e.g. BERT), or static embedding based methods (e.g Word2Vec, Doc2Vec) which have relatively low computational costs but less expressive in some scenarios. We run early experiments using these methods and find that word embedding models are effective in learning semantics from the descriptions of the patents. In this poster, we will describe our early results, as well as exploratory data analysis on this massive historical patents dataset.

2022

Variational Autoencoders

Covers Variational Autoencoders, a probabilistic approach to autoencoders for data generation and feature representation, with applications in Natural Language Processing.

Music Streaming Research: Natural Language Processing at Deezer

Showcases a student's internship at Deezer's research team working on natural language processing.

Model Analysis

Explores neural model analysis in NLP, covering evaluation, probing, and ablation studies to understand model behavior and interpretability.

Natural language generation

Question answering

Question answering (QA) is a computer science discipline within the fields of information retrieval and natural language processing (NLP) that is concerned with building systems that automatically answer questions that are posed by humans in a natural language. A question-answering implementation, usually a computer program, may construct its answers by querying a structured database of knowledge or information, usually a knowledge base. More commonly, question-answering systems can pull answers from an unstructured collection of natural language documents.

Transformer (machine learning model)

A transformer is a deep learning architecture that relies on the parallel multi-head attention mechanism. The modern transformer was proposed in the 2017 paper titled 'Attention Is All You Need' by Ashish Vaswani et al., Google Brain team. It is notable for requiring less training time than previous recurrent neural architectures, such as long short-term memory (LSTM), and its later variation has been prevalently adopted for training large language models on large (language) datasets, such as the Wikipedia corpus and Common Crawl, by virtue of the parallelized processing of input sequence.

The Human Language Technology (HLT) course introduces methods and applications for language processing and generation, using statistical learning and neural networks.

This seminar course explores advanced topics in natural language processing through a mix of reading, reviewing, and writing academic papers.

Discrete mathematics is a discipline with applications to almost all areas of study. It provides a set of indispensable tools to computer science in particular. This course reviews (familiar) topics a