Concept

Divergence (statistics)

Summary

In information geometry, a divergence is a kind of statistical distance: a binary function which establishes the separation from one probability distribution to another on a statistical manifold. The simplest divergence is squared Euclidean distance (SED), and divergences can be viewed as generalizations of SED. The other most important divergence is relative entropy (also called Kullback–Leibler divergence), which is central to information theory. There are numerous other specific divergences and classes of divergences, notably f-divergences and Bregman divergences (see ). Given a differentiable manifold of dimension , a divergence on is a -function satisfying: for all (non-negativity), if and only if (positivity), At every point , is a positive-definite quadratic form for infinitesimal displacements from . In applications to statistics, the manifold is typically the space of parameters of a parametric family of probability distributions. Condition 3 means that defines an inner product on the tangent space for every . Since is on , this defines a Riemannian metric on . Locally at , we may construct a local coordinate chart with coordinates , then the divergence is where is a matrix of size . It is the Riemannian metric at point expressed in coordinates . Dimensional analysis of condition 3 shows that divergence has the dimension of squared distance. The dual divergence is defined as When we wish to contrast against , we refer to as primal divergence. Given any divergence , its symmetrized version is obtained by averaging it with its dual divergence: Unlike metrics, divergences are not required to be symmetric, and the asymmetry is important in applications. Accordingly, one often refers asymmetrically to the divergence "of q from p" or "from p to q", rather than "between p and q". Secondly, divergences generalize squared distance, not linear distance, and thus do not satisfy the triangle inequality, but some divergences (such as the Bregman divergence) do satisfy generalizations of the Pythagorean theorem.

Official source

https://en.wikipedia.org/wiki/Divergence_(statistics)

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Divergence (statistics)

On Speed and Advantage : Results in Information Velocity and Monitoring Problems

A KL Divergence-Based Loss for In Vivo Ultrafast Ultrasound Image Enhancement with Deep Learning

A structured prediction approach for robot imitation learning

On Speed and Advantage : Results in Information Velocity and Monitoring Problems

A KL Divergence-Based Loss for In Vivo Ultrafast Ultrasound Image Enhancement with Deep Learning

A structured prediction approach for robot imitation learning