I am not clear the reason that we normalise the image for CNN by (image – mean_image)? Thanks!
Why Normalize Images Before Using CNN? – Explained
conv-neural-networkdeep learningimage processing
conv-neural-networkdeep learningimage processing
I am not clear the reason that we normalise the image for CNN by (image – mean_image)? Thanks!
Best Answer
First note: you really should be also dividing by the standard deviation of each feature (pixel) value as well. Subtracting the mean centers the input to 0, and dividing by the standard deviation makes any scaled feature value the number of standard deviations away from the mean.
To answer your question: Consider how a neural network learns its weights. C(NN)s learn by continually adding gradient error vectors (multiplied by a learning rate) computed from backpropagation to various weight matrices throughout the network as training examples are passed through.
The thing to notice here is the "multiplied by a learning rate".
If we didn't scale our input training vectors, the ranges of our distributions of feature values would likely be different for each feature, and thus the learning rate would cause corrections in each dimension that would differ (proportionally speaking) from one another. We might be over compensating a correction in one weight dimension while undercompensating in another.
This is non-ideal as we might find ourselves in a oscillating (unable to center onto a better maxima in cost(weights) space) state or in a slow moving (traveling too slow to get to a better maxima) state.
It is of course possible to have a per-weight learning rate, but it's yet more hyperparameters to introduce into an already complicated network that we'd also have to optimize to find. Generally learning rates are scalars.
Thus we try to normalize images before using them as input into NN (or any gradient based) algorithm.