Today I Learned

Some of the things I've learned every day since Oct 10, 2016

84: Some Properties of Transition Matrices (Probability)

Leave a comment Posted by zwaltman on January 4, 2017

The following are some miscellaneous related properties of transition matrices describing stochastic processes such as Markov chains.

A ‘transition matrix’ is a square matrix $A$ with real entries in $[0, 1]$ such that the sum of the entries of any given of $A$ is equal to 1, the interpretation being that $A_{i, j}$ is the probability of a stochastic process transitioning from state $j$ to state $i$ . A ‘regular’ matrix is a square matrix $A$ such that $A^n$ contains strictly positive entries for some $n > 1$ .

For any transition matrix:

Every transition matrix has $1$ as an eigenvalue.

For regular transition matrices:

The algebraic multiplicity of $1$ as an eigenvalue of $A$ is $1$ .
The columns of $\lim _{n \rightarrow \infty} A^n$ are identical, all being the unique $A$ -invariant probability vector $v$ such that the sum of the entries of $v$ is $1$ (because it’s a probability vector) and $A v = v$ .
With $v$ defined as above, for any probability vector $w$ , $\lim _{n \rightarrow \infty} A^n w = v$ . That is, the system or process described by $A$ eventually converges to a distribution described by $v$ , regardless of it’s initial state.