The data processing inequality is an information theoretic concept that states that the information content of a signal cannot be increased via a local physical operation. This can be expressed concisely as 'post-processing cannot increase information'.
Let three random variables form the Markov chain , implying that the conditional distribution of depends only on and is conditionally independent of . Specifically, we have such a Markov chain if the joint probability mass function can be written as
In this setting, no processing of , deterministic or random, can increase the information that contains about . Using the mutual information, this can be written as :
With the equality if and only if , i.e. and contain the same information about , and also forms a Markov chain.
One can apply the chain rule for mutual information to obtain two different decompositions of :
By the relationship , we know that and are conditionally independent, given , which means the conditional mutual information, . The data processing inequality then follows from the non-negativity of .
This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.