Publication
Text Representation Learning for Low Cost Natural Language Understanding
Related concepts (38)
Object–subject–verb word order
In linguistic typology, object–subject–verb (OSV) or object–agent–verb (OAV) is a classification of languages, based on whether the structure predominates in pragmatically neutral expressions. An example of this would be "Oranges Sam ate." OSV is rarely used in unmarked sentences, which use a normal word order without emphasis. Most languages that use OSV as their default word order come from the Amazon basin, such as Xavante, Jamamadi, Apurinã, Warao, Kayabí and Nadëb.
Stable Diffusion
Stable Diffusion is a deep learning, released in 2022 based on diffusion techniques. It is primarily used to generate detailed images conditioned on text descriptions, though it can also be applied to other tasks such as inpainting, outpainting, and generating image-to-image translations guided by a text prompt. It was developed by researchers from the CompVis Group at Ludwig Maximilian University of Munich and Runway with a compute donation by Stability AI and training data from non-profit organizations.
Hyperparameter (machine learning)
In machine learning, a hyperparameter is a parameter whose value is used to control the learning process. By contrast, the values of other parameters (typically node weights) are derived via training. Hyperparameters can be classified as model hyperparameters, that cannot be inferred while fitting the machine to the training set because they refer to the model selection task, or algorithm hyperparameters, that in principle have no influence on the performance of the model but affect the speed and quality of the learning process.
Multilayer perceptron
A multilayer perceptron (MLP) is a misnomer for a modern feedforward artificial neural network, consisting of fully connected neurons with a nonlinear kind of activation function, organized in at least three layers, notable for being able to distinguish data that is not linearly separable. It is a misnomer because the original perceptron used a Heaviside step function, instead of a nonlinear kind of activation function (used by modern networks).
Natural-language understanding
Natural-language understanding (NLU) or natural-language interpretation (NLI) is a subtopic of natural-language processing in artificial intelligence that deals with machine reading comprehension. Natural-language understanding is considered an AI-hard problem. There is considerable commercial interest in the field because of its application to automated reasoning, machine translation, question answering, news-gathering, text categorization, voice-activation, archiving, and large-scale content analysis.
Verb–subject–object word order
In linguistic typology, a verb–subject–object (VSO) language has its most typical sentences arrange their elements in that order, as in Ate Sam oranges (Sam ate oranges). VSO is the third-most common word order among the world's languages, after SOV (as in Hindi and Japanese) and SVO (as in English and Mandarin Chinese).
Large language model
A large language model (LLM) is a language model characterized by its large size. Their size is enabled by AI accelerators, which are able to process vast amounts of text data, mostly scraped from the Internet. The artificial neural networks which are built can contain from tens of millions and up to billions of weights and are (pre-)trained using self-supervised learning and semi-supervised learning. Transformer architecture contributed to faster training.
Deeplearning4j
Eclipse Deeplearning4j is a programming library written in Java for the Java virtual machine (JVM). It is a framework with wide support for deep learning algorithms. Deeplearning4j includes implementations of the restricted Boltzmann machine, deep belief net, deep autoencoder, stacked denoising autoencoder and recursive neural tensor network, word2vec, doc2vec, and GloVe. These algorithms all include distributed parallel versions that integrate with Apache Hadoop and Spark.
Object–verb–subject word order
In linguistic typology, object–verb–subject (OVS) or object–verb–agent (OVA) is a rare permutation of word order. OVS denotes the sequence object–verb–subject in unmarked expressions: Oranges ate Sam, Thorns have roses. The passive voice in English may appear to be in the OVS order, but that is not an accurate description. In an active voice sentence like Sam ate the oranges, the grammatical subject, Sam, is the agent and is acting on the patient, the oranges, which are the object of the verb, ate.
Automated machine learning
Automated machine learning (AutoML) is the process of automating the tasks of applying machine learning to real-world problems. AutoML potentially includes every stage from beginning with a raw dataset to building a machine learning model ready for deployment. AutoML was proposed as an artificial intelligence-based solution to the growing challenge of applying machine learning. The high degree of automation in AutoML aims to allow non-experts to make use of machine learning models and techniques without requiring them to become experts in machine learning.