Fisher information metric | EPFL Graph Search

Êtes-vous un étudiant de l'EPFL à la recherche d'un projet de semestre?

Travaillez avec nous sur des projets en science des données et en visualisation, et déployez votre projet sous forme d'application sur GraphSearch.

Découvrez-en plus sur les applications Graph.

In information geometry, the Fisher information metric is a particular Riemannian metric which can be defined on a smooth statistical manifold, i.e., a smooth manifold whose points are probability measures defined on a common probability space. It can be used to calculate the informational difference between measurements. The metric is interesting in several respects. By Chentsov’s theorem, the Fisher information metric on statistical models is the only Riemannian metric (up to rescaling) that is invariant under sufficient statistics. It can also be understood to be the infinitesimal form of the relative entropy (i.e., the Kullback–Leibler divergence); specifically, it is the Hessian of the divergence. Alternately, it can be understood as the metric induced by the flat space Euclidean metric, after appropriate changes of variable. When extended to complex projective Hilbert space, it becomes the Fubini–Study metric; when written in terms of mixed states, it is the quantum Bures metric. Considered purely as a matrix, it is known as the Fisher information matrix. Considered as a measurement technique, where it is used to estimate hidden parameters in terms of observed random variables, it is known as the observed information. Given a statistical manifold with coordinates , one writes for the probability distribution as a function of . Here is drawn from the value space R for a (discrete or continuous) random variable X. The probability is normalized by The Fisher information metric then takes the form: The integral is performed over all values x in X. The variable is now a coordinate on a Riemann manifold. The labels j and k index the local coordinate axes on the manifold. When the probability is derived from the Gibbs measure, as it would be for any Markovian process, then can also be understood to be a Lagrange multiplier; Lagrange multipliers are used to enforce constraints, such as holding the expectation value of some quantity constant.

À propos de ce résultat

Cette page est générée automatiquement et peut contenir des informations qui ne sont pas correctes, complètes, à jour ou pertinentes par rapport à votre recherche. Il en va de même pour toutes les autres pages de ce site. Veillez à vérifier les informations auprès des sources officielles de l'EPFL.

Publications associées (2)

Differential Entropy of the Conditional Expectation Under Additive Gaussian Noise

Michael Christoph Gastpar, Alper Köse, Ahmet Arda Atalik

The conditional mean is a fundamental and important quantity whose applications include the theories of estimation and rate-distortion. It is also notoriously difficult to work with. This paper establ

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Differential Entropy of the Conditional Expectation under Gaussian Noise

Michael Christoph Gastpar, Alper Köse, Ahmet Arda Atalik

This paper considers an additive Gaussian noise channel with arbitrarily distributed finite variance input signals. It studies the differential entropy of the minimum mean-square error (MMSE) estimato

IEEE2021

Concepts associés (4)

En statistique, l'information de Fisher quantifie l'information relative à un paramètre contenue dans une distribution. Elle est définie comme l'espérance de l'information observée, ou encore comme la variance de la fonction de score. Dans le cas multi-paramétrique, on parle de matrice d'information de Fisher. Elle a été introduite par R.A. Fisher. Soit f(x ; θ) la distribution de vraisemblance d'une variable aléatoire X (qui peut être multidimensionnelle), paramétrée par θ.

Information geometry is an interdisciplinary field that applies the techniques of differential geometry to study probability theory and statistics. It studies statistical manifolds, which are Riemannian manifolds whose points correspond to probability distributions. Historically, information geometry can be traced back to the work of C. R. Rao, who was the first to treat the Fisher matrix as a Riemannian metric. The modern theory is largely due to Shun'ichi Amari, whose work has been greatly influential on the development of the field.

Cours associés (1)

COM-406: Foundations of Data Science

We discuss a set of topics that are important for the understanding of modern data science but that are typically not taught in an introductory ML course. In particular we discuss fundamental ideas an

Séances de cours associées (22)

Estimation : Estimateur linéaire

Explore l'estimation linéaire, les critères optimaux et le principe d'orthogonalité pour de bons choix en matière d'estimation.

Probabilité maximale, MSE, Fisher Information, Cramér-Rao Bound

Explique l'estimation de maximum de vraisemblance, MSE, Fisher information, et Cramér-Rao lié dans l'inférence statistique.

Intervalles de confiance : Estimation gaussienne

Explore les intervalles de confiance, l'estimation gaussienne, l'inégalité Cramér-Rao et les estimations maximales de vraisemblance.