Solved – LSTM for predicting probabilities

distributionslstmprobabilityrestricted-boltzmann-machine

As a newbie to machine learning, I've been playing around with theano and deeplearning4j libraries and as an interesting application I thought of applying long short-term memory (LSTM) to horse racing.

I have results of previous runs. For example: 10 horses, all the data about each horse: his weight, height, past performance…
Then the result is time. (lowest for winner, highest for the last horse)

My question is, if my data looks like this. Is it possible to get probabilities of each horse winning? As far as I understand it is not hard to train the network to give me the order of horses(horse A first, horse B second …), but what about the probabilities of winning?

Also, am I correct in wanting to apply LSTM to such problem? Wouldn't RBM be better suited to predict probability distribution?

Best Answer

Using ANNs

If the inputs are all of the same length, you don't need a RNN. You can just simply train a ANN with with fixed input length. That would be easier and faster.

If you fit the ANN with (many) samples weight, height, past performance at time $t_i$, and then later at time $t_{i+1}$ the data of a given horse $h_k$ changes, you can just run the new data through the ANN and get a prediction. This is possible because the structure of the data is still the same, i.e. weight, height, etc.

Using RNNS

An RNN assumes a time series as input and the data of a horse is not a time series, e.g. there is no temporal relation between the weight and past performance. There could however be a causal relation. You could however train it on the result list as it is a time series. The input could then e.g. be the past $n$ performances of a given horse $h_i$ and you want to know how it may perform in the next race:

$\text{input} = \{t_0, t_1, t_2, t_3, ..., t_{n-1}, t_n\}$

$\text{output} = \{t_{n+1}\}$

Getting multidimensional output

So for your example 

INPUT(race.length, race.condition, ..., horse1, horse2, horse3) 
OUTPUT(horse1.time, horse2.time, horse3.time)

i don't think you'd have to change much. The ANN now has 3 output nodes and you train it similarly to the one from above. If you have a set of input and output pairs you can just use those for training.

Combining ANN and RNN

You could also combine both methods. First you'd train the ANN on the horse data and the RNN on the performance data.

Then you could e.g. add the output of your RNN (prediction of the next performance based on the past performances) as input of the ANN together with the current data of the horse.

Related Question