Markov chain
stochastic matrix
transition matrix
Remark.
Proof.
Proof.
leads to
communicates with
Proof.
Proof.
communicating classes closed
irreducible
absorbing
hitting time
hitting probability
mean hitting time
Note.
Proof.
Proof.
any
stopping time
Proof.
recurrent
transient
Proof.
Proof.
Proof.
Proof.
Proof.
Proof.
Proof.
invariant equilibrium stationary
Proof.
Proof.
Remark.
Proof.
Proof.
Remark.
positive recurrent
null recurrent
Proof.
Remark.
Remark.
Proof.
reversible
detailed balance equations
Remark.
Proof.
Remark.
period
Proof.
Proof.
Proof.
Proof.
