Solved – How to train a Recurrent Neural Network for Temporal XOR

Question

I have coded a Elman RNN using BackPropagation Through Time. In order to check my implementation, I have chosen Temporal XOR(a sequence of binary digits with the third being the xor of previous two and the rest random).

I have questions regarding how to train the model for this input. I am confused with the following strategies,

• Should I give input $x_i$, and ask it to predict the next number in the sequence $x_{i+1}$ and do the error backpropagation
• Should I give input $x_i$ ask it predict $x_{i+1}$, and $x_{i+1}$ ask it predict $x_{i+2}$ and then perform error backpropagation
• Should I give a sequence of $t$ inputs i.e, $x_i,\ldots,x_t$ and ask it to predict $x_{i+1},\ldots,x_{t+1}$ and then perform error backpropagation

Could anyone please help me find the right training strategy? Also, if there are other sample problems like XOR for MLP, please suggest me.

Answer 1

Want to improve this post? Provide detailed answers to this question, including citations and an explanation of why your answer is correct. Answers without enough detail may be edited or deleted.

Thanks everyone. I found the classic paper by Elman "Finding Structure in Time". In this paper he clearly explains the process of training the network on Temporal XOR problem.