Kullback–Leibler divergenceIn mathematical statistics, the Kullback–Leibler divergence (also called relative entropy and I-divergence), denoted , is a type of statistical distance: a measure of how one probability distribution P is different from a second, reference probability distribution Q. A simple interpretation of the KL divergence of P from Q is the expected excess surprise from using Q as a model when the actual distribution is P.
Rényi entropyIn information theory, the Rényi entropy is a quantity that generalizes various notions of entropy, including Hartley entropy, Shannon entropy, collision entropy, and min-entropy. The Rényi entropy is named after Alfréd Rényi, who looked for the most general way to quantify information while preserving additivity for independent events. In the context of fractal dimension estimation, the Rényi entropy forms the basis of the concept of generalized dimensions. The Rényi entropy is important in ecology and statistics as index of diversity.
ParameterA parameter (), generally, is any characteristic that can help in defining or classifying a particular system (meaning an event, project, object, situation, etc.). That is, a parameter is an element of a system that is useful, or critical, when identifying the system, or when evaluating its performance, status, condition, etc. Parameter has more specific meanings within various disciplines, including mathematics, computer programming, engineering, statistics, logic, linguistics, and electronic musical composition.
Scale parameterIn probability theory and statistics, a scale parameter is a special kind of numerical parameter of a parametric family of probability distributions. The larger the scale parameter, the more spread out the distribution. If a family of probability distributions is such that there is a parameter s (and other parameters θ) for which the cumulative distribution function satisfies then s is called a scale parameter, since its value determines the "scale" or statistical dispersion of the probability distribution.
Divergence (statistics)In information geometry, a divergence is a kind of statistical distance: a binary function which establishes the separation from one probability distribution to another on a statistical manifold. The simplest divergence is squared Euclidean distance (SED), and divergences can be viewed as generalizations of SED. The other most important divergence is relative entropy (also called Kullback–Leibler divergence), which is central to information theory.
Library and information scienceLibrary and information science(s) or studies (LIS) is an interdisciplinary field of study that deals generally with organization, access, collection, and protection/regulation of information, whether in physical or digital forms. In spite of various trends to merge the two fields, some consider the two original disciplines, library science and information science, to be separate. However, it is common today to use the terms synonymously or to drop the term "library" and to speak about information departments or I-schools.
Bregman divergenceIn mathematics, specifically statistics and information geometry, a Bregman divergence or Bregman distance is a measure of difference between two points, defined in terms of a strictly convex function; they form an important class of divergences. When the points are interpreted as probability distributions – notably as either values of the parameter of a parametric model or as a data set of observed values – the resulting distance is a statistical distance. The most basic Bregman divergence is the squared Euclidean distance.
Mandelbrot setThe Mandelbrot set (ˈmændəlbroʊt,_-brɒt) is a two dimensional set with a relatively simple definition that exhibits great complexity, especially as it is magnified. It is popular for its aesthetic appeal and fractal structures. The set is defined in the complex plane as the complex numbers for which the function does not diverge to infinity when iterated starting at , i.e., for which the sequence , , etc., remains bounded in absolute value. This set was first defined and drawn by Robert W.
Public libraryA public library is a library, most often a lending library, that is accessible by the general public and is usually funded from public sources, such as taxes. It is operated by librarians and library paraprofessionals, who are also civil servants. There are five fundamental characteristics shared by public libraries: they are generally supported by taxes (usually local, though any level of government can and may contribute); they are governed by a board to serve the public interest; they are open to all, and every community member can access the collection; they are entirely voluntary, no one is ever forced to use the services provided and they provide library and information services without charge.
Sufficient statisticIn statistics, a statistic is sufficient with respect to a statistical model and its associated unknown parameter if "no other statistic that can be calculated from the same sample provides any additional information as to the value of the parameter". In particular, a statistic is sufficient for a family of probability distributions if the sample from which it is calculated gives no additional information than the statistic, as to which of those probability distributions is the sampling distribution.
