how would the processes of computing a conditional expectation on a discrete time markov chain normally go about?
Given a markov chain with states $\{1,2,\dots, N\}$ with given transition probabilities. How should I handle computing $\mathbb{E}(X_{n+1}|X_{n}=i) $?
Is it true that
$$\mathbb{E}(X_{n+1}|X_{n}=i)=\sum_k^N kP(X_{n+1}|X_{n}=i) =\sum_k^N kp_{ij}$$
Thanks!
Best Answer
There is no single "conditional expectation". The expected value always depends on what you're trying to calculate.
For example, let's say we have a discrete Markov chain with state-space $\left\{0, 1, ..., N\right\}$ where each state $i$ represents gaining $i$ golden coins. Let's say that we are looking for the expected value of golden coins we'll gain tomorrow given that today we found $k$ coins:
$$ E(X_1|X_0 = k)=\sum_{i=0}^N iP(X_1=i|X_0=k) = \sum_{i=0}^N ip_{ki} $$
If we are looking for the expected coins we'll gain in two days, the above won't generally work and you'll need to take the square of the transition matrix and calculate the respective probability.