Here are my thoughts on what could be going wrong:
Accuracy (what is being measured)
Perhaps your network is in fact doing well.
Let's consider binomial classification. If we have 50-50 distribution of labels, then 50% accuracy means the model is no better than chance (flipping a coin). If the Bernoulli distribution is 80%-20% and the accuracy is 50%, then the model is worse than chance.
No matter what I try, I'm not seeing better than 20% accuracy when I add a hidden layer.
If the accuracy is 20%, just negate the output and you have 80% accuracy, well done! (well at least for the binomial case).
Not so fast!
I believe that in your case the accuracy is misleading.
This is a good read on the matter.
For classification, the AUC (area under the curve) is often used.
It's common to also examine the Receiver operating characteristic (ROC) and the confusion matrix.
For the multi-class case this becomes more tricky. Here is an answer that I found. Ultimately, this involves a strategy of 1-vs-rest
or 1-vs-1 pairs, more on that here.
Pre-processing
Are the features scaled? Do they have the same bounds? e.g [0,1]
Have you tried standardizing the features? This renders each feature normally distributed with zero mean and unit variance.
Perhaps normalization might help? Dividing each input vector by it's norm places it on the unit circle (for L2 norm) and also bounds the features (but scaling should be performed first otherwise the larger numbers will spike).
Training
As to the learning rate and momentum, if you're not in a big hurry, I would just set a low learning rate and the algorithm will converge better (although slower). This is valid for stochastic gradient descent where examples are shown at random (are you shuffling the data?).
From your code I can't figure out how this happens.
Are you going one pass only through the training data? For SGD, multiple iterations are made. Perhaps try smaller batches? Have you tried weight decay as a regularization method?
Architecture
Cross-entropy as loss function: check.
Softmax at outputs: check.
Might be a longshot at this point but have you tried projection to a higher dimension in the first hidden layer then collapsing to a lower space in the next one two hidden layers?
There is also the cost in your output, I wonder if it could be scaled to make more sense. I would try to plot the evolution of the cost (log loss here) and see if it fluctuates or how steep it is. Your network might be stuck in a local minima plateau. Or it might be doing very well in which case double check the metric?
Hope this helped or generated some new ideas.
EDIT:
Example of how normalization (L2) can make things worse when features are not scaled relative to the other features. Plots for one sample:
In the left image the blue line is a vector of 10 values generated randomly with a mean zero and std of 1. In the right image I added an 'outlier' or out of scale feature no.6 where I set its value to 10. Clearly out of scale. When we normalize the out of scale vector, all other features become very close to 0 as it can be seen in the orange line on the right.
Standardizing the data might be a good thing to do before anything else in this case. Try plotting some histograms of the features or box plots.
You mentioned you are normalizing the vectors to sum up to 1 and now it works better with 10.
That means you are dividing by the 1-norm = sum(abs(X)) instead of the 2-norm (Euclidean) = sum(abs(X).^2)^(1/2). The L1 normalization generates sparser vectors, look at the figure below, where each axis is one feature, so this is a two dimensional space, however it can be generalized to an arbitrary number of dimensions.
Normalizing effectively places each vector on the edge of either shape. For L1 it will lie on the diamond somewhere. For L2 on the circle. When it hits the axis it is zero.
Best Answer
This is impossible to answer in general. If you're working on a problem with strongly predictive features, your task is easier -- smaller sample sizes will estimate a highly performant model. But a problem with only weakly-relevant features will struggle to find signal.
In extrema, if all of your features are pure noise, no network will generalize well, even if you have arbitrarily large volumes of data.
Clever regularization and feature selection can help. And if regularization and feature selection can change the number of parameters you need to estimate a network with a specific level of performance, then this question seems even more complicated than a simple guideline.