[Math] Linear Algebra Stochastic Matrix and Markov Chains

Question

I have a few true and false questions I need help with. Can someone please check my work?

1. The product of two stochastic matrices is a stochastic matrix.

This is false I found a counterexample.

2 . Two Markov chains with the same initial state vector and a common steady state vector must have the same stochastic matrix.

I think this is true because if you multiply them together, $Pq=q$, so they will approach the same vector.

3 . The set of stochastic matrices forms a vector space.

I think this is true because stochastic matrices can be scalar of each other by multiplying them by each other.

4 . The transpose of a stochastic matrix is stochastic.

I think this is true because you are simply rearranging the columns and rows. This should not affect the matrix if multiplying together.

5 . Suppose P is a stochastic matrix and x0 is an initial state. If $x_{k+1}=Px_k$, then $\{x_k\}$ converges as $k \rightarrow \infty$.

I think this true because as k becomes greater, it converges towards a stochastic matrix.

Answer 1

If this is homework, please make sure you tag it as such. I offer some hints:

1. You are correct.
2. See Given an invariant distribution is the (finite state) Markov transition matrix unique?
3. What properties do elements of a vector space need to satisfy? You need to be able to add the elements and multiply by scalars while remaining in the space. What happens if you multiply a stochastic matrix by 0.5?
4. What about this matrix? What is its transpose? $$P=\begin{pmatrix}1 & 0 \\ 1 & 0 \end{pmatrix}$$
5. What concept is being described here? The long-term, steady state or equilibrium behaviour of the system. What conditions are necessary and sufficient for this property to exist?