Poor neural network performance can be described as two situations:
1. The error does not reach the goal.
If this occurs then you can try the following:
a. Raise the error goal of the training function. While it seems that the lowest error goal is best, it can cause invalid training as well as hinder network generalization. For example, if you are using the TRAINLM function, the default error is 0. You may want to set the error to 1e-6 in order to make the network capable of reaching the error goal.
[x,t] = simplefit_dataset;
net = feedforwardnet(10);
net.trainParam.goal=1e-6
b. You may want to use a different performance function. The default performance function is MSE (the mean squared error). For example, try SSE (Sum squared error performance function):
However, the more a network is generalized the more difficult it is to achieve the lowest error. Therefore you may want to consider sacrificing some generalization and improving network performance by raising the performance ratio which lies in the range [0 1]. For more information on improving network generalization see the Related Solution listed at the bottom of the page.
c. You may want to increase the number of epochs for training in some situations. This will take a longer time to train the network but may yield more accurate results. To set the number of epochs see the following example which is an extension to the one above:
net.trainParam.epochs=1000;
2. The next issue that arises in neural network training is the speed and memory usage of training a network to reach the goal.
The following are some suggestions to improving these issues:
a. You may want to preprocess your data to make the network training more efficient. Preprocessing scales the inputs so that they fall into the range of [-1 1]. If preprocessing is done before training, then postprocessing must be done to successfully analyze the results afterwards. For more information on Preprocessing and Postprocessing please refer to Chapter 5 of the Neural Network User's Guide:
b) You may also want to try different training functions. TRAINLM is the most widely used because it is the fastest, but it requires significantly more memory than other training functions. If you want to use this function, but have less memory use, then the "mem_reduc" property needs to be raised:
net.trainParam.mem_reduc=2;
Some networks may also perform faster when using the 'Fast Elliot Sigmoid' transfer function, compared with the standard 'tansig' sigmoid. This is described at the link below:
c) Reduce the number of neurons that are being used. A general rule is to have less input paramters than output parameters. Also, it is not necessary to have as many neurons as input parameters, and the number of output neurons should be significantly less than the number of input parameters. For example, if you have your input P with 200 input parameters then it not necessarily beneficial to have 200 input neurons. You may want to try and use 20 input neurons. It is very difficult to give an exact ratio of input parameters to input neurons because each application calls for specific network architectures.
Best Answer