In several months, I have developed a framework for using neural networks (FANN library) in a chart trading software.
The framework allows me to combine any inputs for the NN, choose a learning rate, momentum, activation function's variables, number of layers and number of neurons in them…almost anything FANN library offers. The following is the UI.
(tl;dr version below)
Basically, I don't know what to do with it now.
The internet has been so far my only source of information and I'd appreciate ANYTHING since I'm pretty much stuck right now and can't move anywhere (the project is a requirement for my master's degree).
I have tried different setups and right now, I'm just aimlessly changing the inputs and the parameters of the NN. The problem is, everything I do and then start with teaching, the NN is just approximating itself to zero.
I have tried incremental learning, batch learning, various values of learning rate and momentum. As an output with which I'm teaching the network, I'm using the slope of the linear regression curve for the following X bars (10 in the picture in the field Output Window).
I've been using all sorts of things to the input. The picture shows two CCI oscilators – they are both having 10 as the input window which means there are total 20 inputs to the neural network, the difference between them is the length of the inner simple moving average (15 and 50) and they are both scaled by the maximum and minimum of the whole data set to the range -1, 1 since I'm using the bipolar sigmoid as the activation function.
But really, I can insert the price itself, the volume or the day/hour, any average or any derivative such as the CCI as the input and choose how it should be scaled (by the min/max of the whole dataset or by the min/max of the input window). I have tried simple things like choosing only the close prices of the last 10 bars or incredibly complicated things like three CCIs with various lengths, two exponential moving averages, hour, day, last volume change…..there were over 50 inputs, I've used different hidden layer setups…
…but in the end, the neural network is just approximating to zero.
I don't know what that means. I thought that the neural network is simply saying "Don't bother with that, there is no relation between the inputs you use and the linear regression curve slope." but then I think it cannot be true – I can see there really are double tops and double bottoms, I can see that there really are some patterns combining the two CCIs that are working repeatedly (patterns described in a book for intraday trading begginers).
I'm using 3 months of trading data of the S&P 500 e-mini grouped by 1000 ticks. I was thinking that maybe 3 months is too much, I should probably try just one month and, well, the approximation to zero is still occuring but it is only taking a longer time. Only by using a really limited dataset (like just a few days), the neural network is actually trying to fit itself but I can clearly tell that it is being overtrained for such a short timespan.
I have been studying the neural networks a lot, yet I feel I have made some false assumptions about how they should or should not be used. So far, I thought that using various inputs and the linear regression slope as the output will lead the NN to find a relation between the two. I'd like the NN to exaggerate even the slightiest relation found and I thought that might be done using the high momentum and high learning rate…NN is still approximating to zero 🙁
Maybe NN is not supposed to work like that at all … I've learned a lot about them yet I know very little to come to any conclusions. If you have anything you think could help me (apart from leaving the project), I'd be very grateful.
TL;DR:
My neural network is approximating to zero for different sorts of inputs with diferrent settings, do you know why?
Also, any ideas, advices, interesting sources on the topic of neural network usage for trading? Or do you know, at least, where else to ask?
EDIT – problem found, see my answer below
Best Answer
In my experience, when the neural network ends up with all zeros at the output, it is likely that the rails are being saturated. That is that the common sigmoid function processing element will only swing to 0 or +1 limits and if the input happens to be far outside of the linear input sensitivity threshold range, it will force the sigmoid to stick at one of the output levels. You could check if all of the inputs are limited to a small range (say -1 to 1) or force them to be limited by using a transformation, such as a scaling or normalization of data. And do not use raw price, but an approximately stationary transformation such as log returns.
There is a simple tutorial on this blog, but keep in mind it is using Weka, a java based GUI.