This lecture covers the definitions and state probabilities of continuous-time Markov chains, focusing on stochastic models for communications. Topics include homogeneous Markov chains, transition probabilities, state probabilities, and time of transitions and stays.