Êtes-vous un étudiant de l'EPFL à la recherche d'un projet de semestre?
Travaillez avec nous sur des projets en science des données et en visualisation, et déployez votre projet sous forme d'application sur Graph Search.
Concept
Conditional mutual information
Résumé
In probability theory, particularly information theory, the conditional mutual information is, in its most basic form, the expected value of the mutual information of two random variables given the value of a third.
For random variables , , and with support sets , and , we define the conditional mutual information as
This may be written in terms of the expectation operator: .
Thus is the expected (with respect to ) Kullback–Leibler divergence from the conditional joint distribution to the product of the conditional marginals and . Compare with the definition of mutual information.
For discrete random variables , , and with support sets , and , the conditional mutual information is as follows
where the marginal, joint, and/or conditional probability mass functions are denoted by with the appropriate subscript. This can be simplified as
For (absolutely) continuous random variables , , and with support sets , and , the conditional mutual information is as follows
where the marginal, joint, and/or conditional probability density functions are denoted by with the appropriate subscript. This can be simplified as
Alternatively, we may write in terms of joint and conditional entropies as
This can be rewritten to show its relationship to mutual information
usually rearranged as the chain rule for mutual information
or
Another equivalent form of the above is
Another equivalent form of the conditional mutual information is
Like mutual information, conditional mutual information can be expressed as a Kullback–Leibler divergence:
Or as an expected value of simpler Kullback–Leibler divergences:
A more general definition of conditional mutual information, applicable to random variables with continuous or other arbitrary distributions, will depend on the concept of regular conditional probability.
Let be a probability space, and let the random variables , , and each be defined as a Borel-measurable function from to some state space endowed with a topological structure.
Source officielle
À propos de ce résultat
Cette page est générée automatiquement et peut contenir des informations qui ne sont pas correctes, complètes, à jour ou pertinentes par rapport à votre recherche. Il en va de même pour toutes les autres pages de ce site. Veillez à vérifier les informations auprès des sources officielles de l'EPFL.