Information Theory: Source Coding, Cryptography, Channel Coding

This lecture covers three main topics: source coding for compressing information, cryptography for protecting information integrity and confidentiality, and channel coding for protecting information from natural damages. These topics are fundamental building blocks of communication systems, intimately related to probability theory, linear algebra, and number theory, and share a common root in the notion of entropy. The lecture also explores digital communication systems, point-to-point communication, entropy, conditional entropy, uniquely decodable codes, Shannon-Fano and Huffman codes, and stationary sources. Additionally, it delves into cryptography using number theory, Euler, Shannon, Fermat, group theory, and the work of Rivest, Shamir, and Adleman. The lecture concludes with an overview of error channels, linear codes, Reed-Solomon codes, and future courses related to signals, systems, networks, and security.