MATLAB: Two state markov chain realization

doit4mehomeworkmarkov chainprobabilitystatistics

I have a state transition probability matrix and a state probability vector

[0.9 0.1; 0.1 0.9] & [0.4 0.6] respectively.

Now, I want to generate the states according to this. say 100 state sequence.

Any sort of help would be appreciated.

Thanks.

Best Answer

I have to admit that my memory of MC is a bit rusty. Does this do what you want? Do you expect a convergence to a state probability vector [0.5 0.5]?

If this is right, there are faster implementations, but I wanted to lay out the basics clearly.

transitionMatrix = [0.9 0.1; 0.1 0.9];
initialProbabilityState = [0.4; 0.6]; % Made it a column vector, rather than a row.
nStates = 100;
states = zeros(2,nStates);
states(:,1) = initialProbabilityState;
for ns = 2:nStates
    states(:,ns) = transitionMatrix*states(:,ns-1);
end

Related Solutions

MATLAB: Error in the code for markov chain

I guess you want a matrix Chain which has in the end on each row some probabilities... You try to put a vector curr_state into 1 element Chain(1) that's why you get an error. You should change that to Chain(1, 1:2) = cur_state; and the same 2 lines bellow: Chain(i, 1:2) = cur_state;

MATLAB: What are the formats of the “seq” and “states” vectors used by the HMMESTIMATE function in the Statistics Toolbox 4.0 (R13)

To explain the format of the "seq" and "states" parameters of the HMMESTIMATE function, a hidden Markov model will be used.

First, take a non-fair coin, with a probability "p" of coming up heads. The two states shall be defined as "heads" and "tails", and the transition from one state to the next is defined by the next flip of the coin. Therefore, the transition matrix would be:

 TM = [p 1-p; ...
 p 1-p];

Assume that whenever the coin is in state 1, the value of the model is 2, and whenever the coin is in state 2, the value of the model is 1 or 3, with equal probability. Therefore, the emission matrix will be:

 E = [0 1 0; ...
 1/2 0 1/2];

If you simulate the hidden Markov model states using the TM matrix (choosing a state at random each simulation step based on the probabilities given in TM), and record the state of the model at each simulation step, the "states" parameter for HMMESTIMATE is obtained. Likewise, if you take that "states" parameter and generate a random value for the model at each simulation step (using the probabilities from the E matrix), the vector of sequence values "seq" is obtained.

To clarify, if the coin is twice as likely to come up heads as tails, you would expect "states" to have twice as many 1's as 2's. You would also expect that the "seq" vector will be one-fourth 1, one-half 2, and one-fourth 3:

p = 2/3; % (1-p) = tails probability = 1/3 = p/2 = half the heads probability
TM = [p 1-p; ...
p 1-p];
E = [0 1 0; ...
1/2 0 1/2];
[SEQ, STATES] = hmmgenerate(1000, TM, E); % Generate 1000 simulation steps
[TM_est, E_est] = hmmestimate(SEQ, STATES) % TM_est should be roughly TM and E_est should be roughly E

Best Answer

Related Solutions

MATLAB: Error in the code for markov chain

MATLAB: What are the formats of the “seq” and “states” vectors used by the HMMESTIMATE function in the Statistics Toolbox 4.0 (R13)

Related Question