Hitting Probabilities: Markov ChainsCovers hitting probabilities in Markov chains with disjoint subsets, the function h(i), theorems, proofs, and expected time to hit calculations.
Continuous Time Markov ChainsCovers the basic theory for continuous time Markov chains and discusses communication, hitting probabilities, recurrence, and transience.