Lecture
This lecture introduces the concept of entropy as a fundamental bound in algorithms, leading to impossibility results and guiding the design of efficient source coding algorithms. It covers the purpose of source coding, the setup involving source and encoder, examples of encoding maps, decodability, and prefix-free codes. The lecture explores the relationship between uniquely decodable and prefix-free codes, emphasizing the importance of instantaneous codes in minimizing decoding delays. It delves into Kraft-McMillan's inequality as a necessary condition for uniquely decodable codes, providing examples and proofs. The lecture concludes with the decoding tree, codeword length, and the contrapositive of Kraft-McMillan's theorem.