Today I Learned

Some of the things I've learned every day since Oct 10, 2016

84: Some Properties of Transition Matrices (Probability)

The following are some miscellaneous related properties of transition matrices describing stochastic processes such as Markov chains.

A ‘transition matrix’ is a square matrix $A$ with real entries in $[0, 1]$ such that the sum of the entries of any given of $A$ is equal to 1, the interpretation being that $A_{i, j}$ is the probability of a stochastic process transitioning from state $j$ to state $i$. A ‘regular’ matrix is a square matrix $A$ such that $A^n$ contains strictly positive entries for some $n > 1$.

For any transition matrix:

1. Every transition matrix has $1$ as an eigenvalue.

For regular transition matrices:

1. The algebraic multiplicity of $1$ as an eigenvalue of $A$ is $1$.
2. The columns of $\lim _{n \rightarrow \infty} A^n$ are identical, all being the unique $A$-invariant probability vector $v$ such that the sum of the entries of $v$ is $1$ (because it’s a probability vector) and $A v = v$.
3. With $v$ defined as above, for any probability vector $w$$\lim _{n \rightarrow \infty} A^n w = v$. That is, the system or process described by $A$ eventually converges to a distribution described by $v$, regardless of it’s initial state.