Concept
Entropy rate
Related lectures (24)
Source Coding Theorem
Explores the Source Coding Theorem, entropy, Huffman coding, and conditioning's impact on entropy reduction.
Information Theory: Source Coding
Covers source coding, typical sequences, stationarity, and efficient encoding in information theory.
Quantum Information
Explores the CHSH operator, self-testing, eigenstates, and quantifying randomness in quantum systems.
Entropy Bounds: Conditional Entropy Theorems
Explores entropy bounds, conditional entropy theorems, and the chain rule for entropies, illustrating their application through examples.
Information Theory: Quantifying Messages and Source Entropy
Covers quantifying information in messages, source entropy, common information, and communication channel capacity.
Source Coding Theorem: Fundamentals and Models
Covers the Source Coding Theorem, source models, entropy, regular sources, and examples.
Stationary Sources: Properties and Entropy
Explores stationary sources, entropy, regularity, and coding efficiency, including a challenging problem with billiard balls.
Markov Chains and Algorithm Applications
Covers Markov chains and their applications in algorithms, focusing on Markov Chain Monte Carlo sampling and the Metropolis-Hastings algorithm.
Information Measures: Entropy and Information Theory
Explains how entropy measures uncertainty in a system based on possible outcomes.
Conditional Entropy: Review and Definitions
Covers conditional entropy, weather conditions, function entropy, and the chain rule.