Summary
The data processing inequality is an information theoretic concept that states that the information content of a signal cannot be increased via a local physical operation. This can be expressed concisely as 'post-processing cannot increase information'. Let three random variables form the Markov chain , implying that the conditional distribution of depends only on and is conditionally independent of . Specifically, we have such a Markov chain if the joint probability mass function can be written as In this setting, no processing of , deterministic or random, can increase the information that contains about . Using the mutual information, this can be written as : With the equality if and only if , i.e. and contain the same information about , and also forms a Markov chain. One can apply the chain rule for mutual information to obtain two different decompositions of : By the relationship , we know that and are conditionally independent, given , which means the conditional mutual information, . The data processing inequality then follows from the non-negativity of .
About this result
This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.
Related publications

No results

Related people

No results

Related units

No results

Related concepts

No results

Related courses

No results

Related lectures

Loading

Related MOOCs

No results