MATLAB: Probability in Markov chain

markov chainprobability

Given, a state sequence (100 elements) of two different states '1' or '2'. find the probability of occurance of '1' and '2'(P1 and P2).

Also find the probability of occurance of P11,P12,P21,P22 where P11(probability of the state remaining in the state 1 during a transition and similarly others) represents the elements of state transition probability matrix.

Any sort of help would be appreciated.

Thanks & Regards

Best Answer

I can write the code in a couple of minutes. But let's see if you can help me to help you.

1. Generate a vector of 100x1 or 1x100. The data is a random distribution of '1' and '2'. You can use rand(). Or randint() is even better.

2. set your P=zeros(2,2)

3. Go through a for-loop. If the vector value goes from 1 to 1, P(1,1) is increased by 1. If the values goes from 1 to 2, P(1,2) is increased by 1. And so on.

4. At the end, divide P by 100 (your data length), you get your probability matrix.

EDIT This is incorrect. Should divide the first row by the total number of 1s and divide the second row by the total number of 2s.

Start writing your MATLAB code, if you have particular questions regarding using MATLAB functions, or M-script syntax. People here would love to help you.

Related Solutions

MATLAB: Two state markov chain realization

I have to admit that my memory of MC is a bit rusty. Does this do what you want? Do you expect a convergence to a state probability vector [0.5 0.5]?

If this is right, there are faster implementations, but I wanted to lay out the basics clearly.

transitionMatrix = [0.9 0.1; 0.1 0.9];
initialProbabilityState = [0.4; 0.6]; % Made it a column vector, rather than a row.
nStates = 100;
states = zeros(2,nStates);
states(:,1) = initialProbabilityState;
for ns = 2:nStates
    states(:,ns) = transitionMatrix*states(:,ns-1);
end

MATLAB: Markov chain with two states using MATLAB

Assuming you want to recover the original 2x2 transition matrix from only the steady state probabilities, this is not uniquely possible. However, if you simply want to generate one of the possibilities, here is a method (assuming first state is ON and second state is OFF):

q = your steady state ON probability (a given input between 0 < q < 1)
% Generate one of the many possibilities
if( q <= 0.5 )
    p21 = rand*q/(1-q);
    p12 = ((1-q)/q)*p21;
else
    p12 = rand*(1-q)/q;
    p21 = (q/(1-q))*p12;
end
P = [ 1-p12, p12; p21, 1-p21 ];  % A Markov 2x2 transition matrix w/desired steady state

Best Answer

Related Solutions

MATLAB: Two state markov chain realization

MATLAB: Markov chain with two states using MATLAB

Related Question