Probability – Simple Random Walk on Cycle Graph Ending on Specific Vertex

conditional probabilityconditional-expectationexpected valuemarkov chainsprobability

I'm considering a simple random walk on a cycle graph comprising a number of vertices, labelled $1$ to $5$ consecutively. Suppose I start at vertex 1 and can traverse to either side ($2$ or $5$). I continue this random walk until I have covered all vertices. What is the probability that I finish on node $3$, and the expected number of steps to get there? How do I calculate the same quantities if I finish on the other vertices? How do I generalise to say $n$ vertices?

I have seen multiple resources discussing the cover time on a cycle graph, but in this context, I guess I have to include a discussion on either stopping time after visiting all vertices (which I don't really know how to include here, perhaps via a conditional probability) or transforming the end state as an absorbing state (which I guess won't make it a Markov chain anymore). I also suspect that for the first part (ending on node 3) we can exploit symmetry, but I'm not sure how. What should I do here? Thanks!

Edit: after simulating this, I've noticed that the probabilities of ending at any node is $1/4$. The expected number of steps terminating at $3$ (or $4$) should be $11$, and the expected number of steps terminating at $2$ (or $5$) should be $9$.

Best Answer

Look at my comment under

This problem can be decomposed as several mini problems. Each of these mini problems are versions of a common expected value problem which looks something like a line of lily pads numbered $0,1,\ldots n$ with a frog starting on $k\in (0,n]$ and something bad (like a crocodile) on lilypad $0$ and something good (like food) on lilypad $n$.

Let's call the expected value of the number of steps it takes the frog to reach $n$ when starting from $k$ as $E(k,n)$. We get the following system of equations using states (assuming $n\geq 2$) $$\begin{cases} E(1,n)=1+E(2,n)\\ E(x,n)=1+\frac{1}{2}E(x-1,n)+\frac{1}{2}E(x+1,n)~\forall x\in [2,n-1]\\ E(n,n)=0\end{cases}$$

We can see that $E(x,n)$ has a really nice recursion for the non-edge cases (if $a_x=E(x,n)$, then $a_x-3a_{x-1}+3a_{x-2}-a_{x-3}=0$) We also have $3$ "initial" conditions ($E(1,n)=1+\frac{1}{2}E(2,n)$, $E(n,n)=0$, and $2E(x,n)-E(x-1,n)-E(x+1,n)=2$). We can use the properties of linear recurrences to conclude that $E(x,n)$ will be some quadratic polynomial in $x$. Let's say $E(x,n)=ax^2+bx+c$ (the constants $a,b,c$ will likely be dependent on $n$).

From the recursion, we have that $$2E(x,n)-E(x-1,n)-E(x+1,n)=2$$ $$(E(x,n)-E(x-1,n))-(E(x+1,n)-E(x,n))=2$$ $$(2ax+b-1)-(2ax+2a+b-1)=2$$ $$-2a=2$$ $$a=-1$$ So $E(x,n)=-x^2+bx+c$. Now we also have $$\begin{cases} E(1,n)=1+E(2,n)\\E(n,n)=0\end{cases}$$ $$\begin{cases} -1+b+c=1+(-4+2b+c)\\-n^2+bn+c=0\end{cases}$$ $$\begin{cases} b=2\\c=n^2-bn\end{cases}$$ $$\begin{cases} b=2\\c=n^2-2n\end{cases}$$ Hence, we have $\boxed{E(x,n)=-x^2+2x+n^2-2n}$. This result will be useful for the rest of the problem

There is also another problem we need to solve that will be useful. This is the probability that given lilypads numbered $0,1,\ldots n$ with a frog at $x$, what is the probability that they will go to $n$ before going to $0$. If we call this probability $P(x,n)$, we get a similar system of equations using states (assuming $n\geq 2$) $$\begin{cases} P(0,n)=0\\ P(x,n)=\frac{1}{2}P(x-1,n)+\frac{1}{2}P(x+1,n)~\forall x\in [1,n-1]\\ P(n,n)=1\end{cases}$$ Using a similar approach as previously, we can use the fact that there is a nice recursion for $P(x,n)$ to see that $P(x,n)$ must be a linear polynomial in $x$. Hence, we have $P(x,n)=ax+b$ Since $P(0,n)=0$ and $P(n,n)=1$, we can quickly see that $\boxed{P(x,n)=\frac{x}{n}}$.

Back to the original problem, to finish on $3$, you must first get to $2$ or $4$. Then you must go to the other of $2$ or $4$. Then you must go to $3$.

The only decision we have to make is whether to go to $2$ or $4$ first. After we have made this decision, there are no more decisions left to make so that we achieve our goal of finishing on $3$

Let's say that we choose to go to $2$ before going to $4$. We can treat $4$ as the $0$ lilypad and $2$ as the $n$ lilypad. The probability that we actually choose this path of going to $2$ before going to $4$ is $P(2,3)$. The expected number of steps to get to $2$ is $E(2,3)$.

From there, we must go to $4$ without going to $3$. The expected number of steps to do that is $E(1,4)$. From there we simply need to go to $3$ anyway possible. We can't actually use any of the formulae we derived above, but this is a straightforward application of states with a lot of symmetry. So I'll leave it as an exercise to the reader to prove that the expected number of steps to do this is $4$.

Now we deal with the case that we choose to go to $4$ before going to $2$. We can treat $2$ as the $0$ lilypad and $4$ as the $n$ lilypad. The probability that we actually choose this path of going to $4$ before going to $2$ is $P(1,3)$. The expected number of steps to get to $4$ is $E(1,3)$.

From there, we must go to $2$ without going to $3$. The expected number of steps to do that is $E(1,4)$. From there we simply need to go to $3$ anyway possible. As before, the expected number of steps to do this is $4$.

In total, the expected number of steps to execute this 2-part procedure is $$P(2,3)(E(2,3)+E(1,4)+4)+P(1,3)(E(1,3)+E(1,4)+4)$$ $$=\frac{2}{3}(3+9+4)+\frac{1}{3}(4+9+4)$$ $$=\frac{32}{3}+5$$ $$=\frac{47}{3}$$

Similarly, it's not hard to see that the probability you end on $3$ is $$P(2,3)P(1,4)(1)+P(1,3)P(1,4)(1)$$ $$=P(1,4)(P(2,3)+P(1,3))$$ $$=\frac{1}{4}$$

Code for finding expected number of steps to end on 3. Should return approximately 11 if trials is high enough

def f1(trials):
    successful_trials = 0
    total_steps = 0
    for i in range(trials):
        steps = 0
        pos = 1
        unvisited = [0,2,4]
        while(pos % 5 != 3):
            pos += get_random_step()
            steps += 1
            if pos % 5 in unvisited:
                unvisited.remove(pos % 5)
        if unvisited == []:
            successful_trials += 1
            total_steps += steps
    return total_steps/successful_trials

Code for finding $E(x,n)$

def f2(x,n,trials):
    successful_trials = 0
    total_steps = 0
    for i in range(trials):
        steps = 0
        pos = x
        while(pos != n and pos != 0):
            pos += get_random_step()
            steps += 1
        if pos == n:
            successful_trials += 1
            total_steps += steps
    return total_steps/successful_trials
Related Question