Normalization rescales the values into a range of [0,1]. This might be useful in some cases where all parameters need to have the same positive scale. However, the outliers from the data set are lost.
$$ X_{changed} = \frac{X - X_{min}}{X_{max}-X_{min}} $$
Standardization rescales data to have a mean ($\mu$) of 0 and standard deviation ($\sigma$) of 1 (unit variance).
$$ X_{changed} = \frac{X - \mu}{\sigma} $$
For most applications standardization is recommended.
First off, standardization usually is taken to be
subtraction of the mean
division by the standard deviation.
The result has a mean 0 and standard deviation of 1.
Dividing by the variance will be wrong for any variable that is not a pure number. One of the reasons for standardization is to remove any influence of the units of measurement. The standard deviation always has the same units of measurement as the variable itself and division washes out those units.
There is no reason in principle why e.g. subtraction of the median and division by the interquartile range or in general any scaling
(value - measure of level) / measure of scale
might not be useful, but using mean and SD is by far the most common procedure. The idea that the Gaussian or normal is a reference distribution often underlies this, but using measures of level and scale other than the mean and standard deviation would often be useful, especially if you were interested in simple methods for identifying outliers (a very big topic covered by many threads in this forum).
The answer to your general question is pretty much tautologous: standardization is useful whenever difference in level, scale or units of measurement would obscure what you want to see. If you are interested in relative variations, standardize first.
If you wanted to compare the heights of mean and women, the units of measurement should be the same (metres or inches, whatever), and standardization is not required. But if the scientific or practical question requires comparing values relative to the mean, subtract the mean first. If it requires adjusting for different amounts of variability, divide by the standard deviation too.
Freedman, D., Pisani, R., Purves, R. Statistics New York: W.W. Norton (any edition) is good on this topic.
Best Answer
A short recap. Given a model $y=X\beta+\varepsilon$, where $X$ is $n\times p$, $\hat\beta=(X'X)^{-1}X'y$ and $\hat y=X\hat\beta=X(X'X)^{-1}X'y=Hy$, where $H=X(X'X)^{-1}X'$ is the "hat matrix". Residuals are $$e=y-\hat y=y-Hy=(I-H)y$$ The population variance $\sigma^2$ is unknown and can be estimated by $MSE$, the mean square error.
Semistudentized residuals are defined as $$e_i^*=\frac{e_i}{\sqrt{MSE}}$$ but, since the variance of residuals depends on both $\sigma^2$ and $X$, their estimated variance is: $$\widehat V(e_i)=MSE(1-h_{ii})$$ where $h_{ii}$ is the $i$th diagonal element of the hat matrix.
Standardized residuals, also called internally studentized residuals, are: $$r_i=\frac{e_i}{\sqrt{MSE(1-h_{ii})}}$$
However the single $e_i$ and $MSE$ are non independent, so $r_i$ can't have a $t$ distribution. The procedure is then to delete the $i$th observation, fit the regression function to the remaining $n-1$ observations, and get new $\hat y$'s which can be denoted by $\hat y_{i(i)}$. The difference: $$d_i=y_i-\hat y_{i(i)}$$ is called deleted residual. An equivalent expression that does not require a recomputation is: $$d_i=\frac{e_i}{1-h_{ii}}$$ Denoting the new $X$ and $MSE$ by $X_{(i)}$ and $MSE_{(i)}$, since they do not depend on the $i$th observation, we get: $$t_i=\frac{d_i}{\sqrt{\frac{MSE_{(i)}}{1-h_{ii}}}} =\frac{e_i}{\sqrt{MSE_{(i)}(1-h_{ii})}}\sim t_{n-p-1}$$ The $t_i$'s are called studentized (deleted) residuals, or externally studentized residuals.
See Kutner et al., Applied Linear Statistical Models, Chapter 10.
Edit: I must say that the answer by rpierce is perfect. I thought that the OP was about standardized and studentized residuals (and dividing by the population standard deviation to get standardized residuals looked odd to me, of course), but I was wrong. I hope that my answer can help someone even if OT.