First, let me acknowledge up front that I'm not as well versed in statistics and mathematics as I'd like to be. Some might say have just enough knowledge to be dangerous. 😀 I apologize if I'm not using terminology correctly.
I am trying to model the probabilities of a system transitioning from one state to another. A simple Markov model is a good start. (Set of states, set of initial state probabilities, set of transition probabilities between states.)
However, the system I'm modeling is more complex than that. The transition probabilities leading to a state at time T are most certainly dependent on variables other than the state at T-1. For example, S1 -> S2 might have a transition probability of 40% when the sun is shining, but S1 -> S2 probability goes to 80% when it is raining.
Additional info from commenters' questions:
- The states are observable.
- There will only be 5-10 states.
- There are currently about 30 covariates that we want to investigate, though the final model will certainly have fewer than this.
- Some covariates are continuous, others are discrete.
Three questions:
- How can I incorporate conditional transition probabilities into my Markov model?
- Or, is there another perspective entirely from which I should approach this issue?
- Also, what keywords/concepts should I search for online to learn more about this?
I've already been around the web searching for things like "markov models with conditional transition probabilities," but so far nothing has slapped me in the face and said, "This is your answer, dummy!"
Thank you for your help and patience.
Best Answer
You can always have a 2nd order or higher order markov chain. In that case your model all ready includes all probabilistic transition information in it. You can check Dynamic Bayesian Networks which is a graphical model generalization of Markov Chains that are utilized frequently in machine learning.