Longueur de description minimale

Êtes-vous un étudiant de l'EPFL à la recherche d'un projet de semestre?

Travaillez avec nous sur des projets en science des données et en visualisation, et déployez votre projet sous forme d'application sur GraphSearch.

Découvrez-en plus sur les applications Graph.

Minimum Description Length (MDL) is a model selection principle where the shortest description of the data is the best model. MDL methods learn through a data compression perspective and are sometimes described as mathematical applications of Occam's razor. The MDL principle can be extended to other forms of inductive inference and learning, for example to estimation and sequential prediction, without explicitly identifying a single model of the data. MDL has its origins mostly in information theory and has been further developed within the general fields of statistics, theoretical computer science and machine learning, and more narrowly computational learning theory. Historically, there are different, yet interrelated, usages of the definite noun phrase "the minimum description length principle" that vary in what is meant by description: Within Jorma Rissanen's theory of learning, a central concept of information theory, models are statistical hypotheses and descriptions are defined as universal codes. Rissanen's 1978 pragmatic first attempt to automatically derive short descriptions, relates to the Bayesian Information Criterion (BIC). Within Algorithmic Information Theory, where the description length of a data sequence is the length of the smallest program that outputs that data set. In this context, it is also known as 'idealized' MDL principle and it is closely related to Solomonoff's theory of inductive inference, which is that the best model of a data set is represented by its shortest self-extracting archive. Selecting the minimum length description of the available data as the best model observes the principle identified as Occam's razor. Prior to the advent of computer programming, generating such descriptions was the intellectual labor of scientific theorists. It was far less formal than it has become in the computer age. If two scientists had a theoretic disagreement, they rarely could formally apply Occam's razor to choose between their theories. They would have different data sets and possibly different descriptive languages.

À propos de ce résultat

Cette page est générée automatiquement et peut contenir des informations qui ne sont pas correctes, complètes, à jour ou pertinentes par rapport à votre recherche. Il en va de même pour toutes les autres pages de ce site. Veillez à vérifier les informations auprès des sources officielles de l'EPFL.

Publications associées (1)

Concepts associés (17)

Séances de cours associées (14)

Portfolio construction under information asymmetry

Lionel Coulot

We introduce in this thesis the idea of a variable lookback model, i.e., a model whose predictions are based on a variable portion of the information set. We verify the intuition of this model in the

EPFL2013

Inductive probability

Inductive probability attempts to give the probability of future events based on past events. It is the basis for inductive reasoning, and gives the mathematical basis for learning and the perception of patterns. It is a source of knowledge about the world. There are three sources of knowledge: inference, communication, and deduction. Communication relays information found using other methods. Deduction establishes new facts based on existing facts. Inference establishes new facts from data. Its basis is Bayes' theorem.

Longueur de description minimale

La longueur de description minimale ou LDM (MDL pour Minimum Description Length en anglais) est un concept inventé par Jorma Rissanen en 1978 et utilisé en théorie de l'information et en compression de données. Le principe est basé sur l'affirmation suivante : toute régularité dans un ensemble de données peut être utilisée afin de compresser l'information, c'est-à-dire l'exprimer à l'aide d'un nombre réduit de symboles. Théorie de l'information Jorma Rissanen, « Modeling by shortest data description », Automatica, vol 14, No 5, pp.

Minimum message length

Minimum message length (MML) is a Bayesian information-theoretic method for statistical model comparison and selection. It provides a formal information theory restatement of Occam's Razor: even when models are equal in their measure of fit-accuracy to the observed data, the one generating the most concise explanation of data is more likely to be correct (where the explanation consists of the statement of the model, followed by the lossless encoding of the data using the stated model).

Arbres de décision: Induction et taille CS-423: Distributed information systems

Explore les arbres de décision, de l'induction à l'élagage, en mettant l'accent sur l'interprétabilité et les forces de sélection automatique des fonctionnalités, tout en abordant des défis tels que l'ajustement excessif.

Théorème Kraft-McMillan COM-102: Advanced information, computation, communication II

Explore le théorème Kraft-McMillan, prouvant l'existence de codes uniques sans préfixes.

Compression: Inégalité de Kraft COM-406: Foundations of Data Science

Explique la compression et l'inégalité Kraft dans les codes et les séquences.