Concept
Entropy (information theory)
Related lectures (341)
Variational Formulation: Information Measures
Explores variational formulation for measuring information content and divergence between probability distributions.
Information Theory: Prefix-Free Codes
Covers prefix-free codes, Kraft inequality, Huffman coding, and entropy.
Mutual Information and Entropy
Explores mutual information and entropy calculation between random variables.
Thermodynamics: Entropy and Ideal Gases
Explores entropy, ideal gases, and TDS equations in thermodynamics, emphasizing the importance of the Clausius inequality and the Carnot cycle.
Bethe Free Entropy
Covers the computation of Bethe free entropy and the interpretation of messages between variables and factors.
Entropy and Disorder
Explores entropy, disorder, and the calculation of maximum entropy under constraints using Lagrange multipliers.
Information Measures: Part 1
Covers information measures, tail bounds, subgaussions, subpossion, independence proof, and conditional expectation.
Quantum Source Coding
Covers entropic notions in quantum sources, Shannon entropy, Von Neumann entropy, and source coding.
Information Theory: Review and Mutual Information
Reviews information measures like entropy and introduces mutual information as a measure of information between random variables.
Conclusions on Module II
Concludes Module II by presenting two theories on optimal signal representation.