This's the problem I'm facing (from Norris , Markov Chain p.18) :
" A gambler has £2 and needs to increase it to £10 in a hurry. He can play a game with the following rules: a fair coin is tossed; if a player bets on the right side, he wins a sum equal to his stake, and his stake is returned; otherwise he loses his stake. The gambler decides to use a bold strategy in which he stakes all his money if he has £5 or less, and otherwise stakes just enough to increase his capital, if he wins, to £10.
Let $X_0$ = 2 and let $X_n$ be his capital after n throws. Prove that the gambler will achieve his aim with probability 1/5."
I've seen other approaches to this question , but the same book provided a theorem and I want see if it is applicable here , the theorem states : (when markov property is satisfied ) The vector of hitting probabilities [on a set A $\subset$ the state space I] $h^A = \; (h_i^A : i \in I)$ is the minimal non-negative solution to the system of linear equations
$$ \left\{ \begin{array}{cc}
h_i^A = 1 & for \; i \in A \\
h_i^A = \sum_{j\in I} p_{ij}h_j^A & for \; i \notin A
\end{array}\right.
$$
where
- $p_{ij}$ is conditional transition probability from state i to j
- you may take $h_i^A = P_i(hit\;A)$ , interpreted as : starting at state i , the probability of the random variable assuming a value in A in finite time
A proof of this theorem is given in the book p.13 .
I tried to apply this theorem to the above problem : The process $(X_n)_{n\ge 0}$ where n is integral time assumes value in state space $\{0,…,10\}$ . Denote hitting probability starting at state i as $h_{i}^{\{10\}}$ .
I think the "probability 1/5" is $h_2^{\{10\}}$ . Using information provided to define
$$
\left\{ \begin{array}{cc}
h_0^{\{10\}} = 0 \\
h_{10}^{\{10\}} = 1 \\
h_{i}^{\{10\}} = \frac{1}{2} h_{2i}^{\{10\}} + \frac{1}{2} h_{0}^{\{10\}} \quad i = 1,…,5 \\
h_{i}^{\{10\}} = \frac{1}{2} h_{10}^{\{10\}} + \frac{1}{2} h_{2i-10}^{\{10\}} \quad i = 6,…,10 \\
\end{array}\right.
$$
But I couldn't solve for $h_{2}^{\{10\}} $ , is the theorem applicable here ?
Best Answer
Ignoring superscript . There are four equations relating $ℎ_2,ℎ_4,ℎ_6 \; and \; ℎ_8$ ,
$$ h_2 = .5h_4 $$ $$ h_4 = .5h_8 $$ $$ h_6 = .5 + .5h_2 $$ $$ h_8 = .5 + .5 h_6 $$ which gives $$ h_2 = .5(.5(.5+.5(.5+.5h_2))) $$ $$ h_2 = 1/5$$