Markov Chains: Basics and Applications

This lecture covers the fundamentals of Markov chains, focusing on discrete time and discrete space Markov chains. It explains the Markov property, transition matrices, generation algorithms, and examples like random walks. It also delves into continuous space Markov chains, transition kernels, and Poisson processes. The lecture concludes with discussions on non-homogeneous Poisson processes and general continuous time Markov chains.