Lectures related to Compression

Diffusion Models

Explores diffusion models, focusing on generating samples from a distribution and the importance of denoising in the process.

Statistical Estimation: Properties and Distributions

Explores statistical parameter estimation, sample accuracy, and Bernoulli variables' properties.

Data Compression and Shannon-Fano Algorithm

Explores the Shannon-Fano algorithm for data compression and its efficiency in creating unique binary codes for letters.

Data Compression and Shannon's Theorem Summary

Summarizes Shannon's theorem, emphasizing the importance of entropy in data compression.

Generalization Error

Discusses mutual information, data processing inequality, and properties related to leakage in discrete systems.

Binary Choice Model

Covers the binary choice model, error term assumptions, specific constants, invariances, and distribution properties.

Entropy and Algorithms: Twenty Questions Problem

Explores the Twenty Questions Problem, Huffman codes, and optimal querying strategies in algorithms, demonstrating efficient outcomes through prefix-free and ternary codes.

Information Measures: Entropy and Information Theory

Explains how entropy measures uncertainty in a system based on possible outcomes.