I am watching some beginner level video training on neural networks using Tensorflow / Keras to get a better understanding of how they work and how to best implement them.
I have some questions on how one feeds back signals to get the network to train itself. For example, lets say I am building a stock trading NN with I/O as follows:
Inputs:
- The last say 20 daily prices, volumes, moving averages, MACD signals
or other financial indicators - Signals that I supply, indicating where I would buy and sell
Output:
- A Buy signal and a Sell signal (or one signal that goes to 1 after buy
and returns to 0 after sell)
Ideally the network would learn how to output buy and sell signals such that the difference between the buy and sell signals, ie. profit, is maximized. For example when price is trending upward it would issue a buy, and when it starts to slope down a sell occurs. The optimized network would adjust itself to buy at the lowest point and sell at the highest point possible.
My Question
I am not clear how the 'profit' signal is used to train the system. The profit is independent of the input and output signals, it is a function of both, and it is calculable only after a sell. How do I go about telling the NN to optimize itself to maximize an arbitrarily calculated value?
From my understanding this seems like an ideal application for NNs, Im just a bit unclear on how to proceed. Thanks for any advice.
And if I may ask a second question, let's say I wanted to run my NN against multiple stocks, each with its own unique behaviours; eg. it might be especially volatile or its median price may be 100 times the first stock.
In that case, would I create a whole new NN for each stock, or is it as simple as just adding a new input with a fixed constant for each different stock?
- StockA = 0.1
- StockB = 0.2
- StockC = 0.3
Would this cause training to apply a whole new set of weightings / biases for each stock?
Best Answer
More of an extended comment than of an explicit answer - but too long for comments:
1) A distinction needs to be made between prediction and optimization. Prediction is used to model phenomenon over which we have no control (i.e. we are merely a passive observer) - in your case this is the whether the stock will be trending up or trending down.
Optimization is used for variables over which we have control, i.e. we decide what that variable is (i.e. we interact with environment). In your case this whether to buy or sell.
There will be several scenarios for example where the stock is trending up, but buying isn't the best decision to take. At the very least, you should have three target values to train your network (1: Buy, 0: Wait, -1: Sell) - but even if you did that, your approach is still problematic because your conflating a prediction problem and a decision/optimization problem.
2) You might be better off using reinforcement learning instead of conventional neural networks. W/R to your problem, reinforcement learning has two advantages: In reinforcement learning, a model is trained to maximize a target function, as opposed to conventional neural networks where the model is trained to minimize a loss function. And reinforcement learning is better suited for problems where the goal is a long term goal, compared to supervised learning.