Markov Chains: States Classification

About
Privacy
Disclaimer

Graph Chatbot

Related lectures (29)

Page 3 of 3

Markov Chain Games

Explores Markov chain games, hitting probabilities, and expected hitting times in a target set.

Markov Chains: Ergodic Chains Examples

Covers stochastic models for communications, focusing on discrete-time Markov chains.

Markov Chains: Transition Densities

Covers Markov processes, transition densities, and distribution conditional on information, discussing classification of states and stationary distributions.

Markov Chains and Algo Applications

Covers Markov chains, Metropolis algorithm, Glauber dynamics, and heat bath dynamics.

Stochastic Models for Communications: Discrete-Time Markov Chains - First Passage Time

Explores discrete-time Markov chains, emphasizing first passage time probabilities and minimal solutions.

Markov Chains: Definitions and State Probabilities

Covers the definitions and state probabilities of discrete-time Markov chains.

Markov Chains: Absorbing Classes

Explores Markov chains with absorbing classes through exercises on transition matrices and expected values.

Continuous Time Markov Chains

Introduces continuous time Markov chains on a finite state space with exponential waiting times and jump probabilities.

Markov Chains: Challenges and Surprises

Explores challenges and surprises in Markov chains, analyzing convergence, state grouping, and transition probabilities.