Compute conditional expectation in Markov Chain

conditional-expectationmarkov chains

There are $d$ spheres numbered $1,2,3,…,d$ distributed in two marked boxes. Let $X_0$ be the number of spheres in box $1$ at time $0$ and at each time $n= 1,2,…,d$ number is randomly selected from the set $\{1,2,…,d\}$ and the sphere marked with that number is taken out of the box it is in and placed in the other box. We denote by $X_n$ the number of spheres in box $1$ at time $n$; prove that $\{X_n,n≥0\}$ is a Markov chain with state space $S=\{1,2,…,d\}$ and determine the transition probabilities, finally compute $\mathbb{E}(X_{n+1}|X_n)$.
I've tried this:
The transition matrix is
$$\mathbb{P_{ij}}=\left\lbrace\begin{array}{c} 1-\frac{i}{d}~~~~~~if~j=i+1 \\ \frac{i}{d}~~~~~~~~~~~~if~j=i-1\\0~~~~~~~~~~~otherwise \end{array}\right.$$

Best Answer

I would use the following:

$$\mathbb{E}​[X_{n+1}​|X_n=i]=\sum_{k=0}^d k\cdot \mathbb{P}(X_{n+1}​=k|X_n=i)​=(i+1)(1-\frac{i}{d})+(i-1)\frac{i}{d}=1+i(1-\frac{2}{d}),$$ Where in the second inequality I used the transition probabilities you derived.

Thus I think you can conclude $$\mathbb{E}​[X_{n+1}​|X_n]=1+X_n(1-\frac{2}{d})$$.