According to this answer, a general rule of thumb is that your hidden layer size should be between your input and output sizes.
In developing my JavaScript neural network, this has proven to be about right. However, for boolean logic it seems to be wrong. For example, an xor task seems to need more than 2 hidden neurons in order to complete (I have tested this on my own net and it seems about right, as well as this other JS net here needing 3 neurons). Additionally, a subtraction task seems to need a great many as well, at least on my network.
Is there a separate rule of thumb for these types of logic tasks?
Best Answer
The universal approximation theorem of neural networks states that a neural network with a bounded, continuous activation functions and given enough hidden units in only one hidden layer can approximate any function. So, in theory you should be able to get the correct output, but you do not know in advance what should be the right number of hidden units for your problem. So, you may not necessarily need to choose a number of hidden units less than the number of your input units (however that's usually the case in practice)
Also, what I'm seeing is your network is a Perceptron network. If that's the case, then your network is only able to learn linearly-separable data points and you cannot learn complex functions.
Sources