Are you an EPFL student looking for a semester project?
Work with us on data science and visualisation projects, and deploy your project as an app on top of GraphSearch.
Concept
Data processing inequality
Summary
The data processing inequality is an information theoretic concept that states that the information content of a signal cannot be increased via a local physical operation. This can be expressed concisely as 'post-processing cannot increase information'.
Let three random variables form the Markov chain , implying that the conditional distribution of depends only on and is conditionally independent of . Specifically, we have such a Markov chain if the joint probability mass function can be written as
In this setting, no processing of , deterministic or random, can increase the information that contains about . Using the mutual information, this can be written as :
With the equality if and only if , i.e. and contain the same information about , and also forms a Markov chain.
One can apply the chain rule for mutual information to obtain two different decompositions of :
By the relationship , we know that and are conditionally independent, given , which means the conditional mutual information, . The data processing inequality then follows from the non-negativity of .
Official source
About this result
This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.