I read in Wilcox, 2003 p. 247 that the standard error of the difference between two sample means is (assuming the normality and homoskedasticity assumptions):
$\sqrt{\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}}$
Rather than simply adding the two sample standard errors as in:
$\frac{\sigma_1}{\sqrt{n_1}} + \frac{\sigma_2}{\sqrt{n_2}}$
What is the intuition behind taking the square of the sum of the two variances divided into their respective sample size, rather than the sum of the standard errors?
Best Answer
You seem to be thinking that $\sqrt{\text{Var}(\bar X-\bar Y)} = \sqrt{\text{Var}(\bar X)} + \sqrt{\text{Var}(\bar Y)}$.
This is not the case for independent variables.
For $X,Y$ independent, $\text{Var}(\bar X-\bar Y) = \text{Var}(\bar X) + \text{Var}(\bar Y)$
Further,
$\text{Var}(\bar X) = \text{Var}(\frac{1}{n}\sum_iX_i) = \frac{1}{n^2}\text{Var}(\sum_iX_i)= \frac{1}{n^2}\sum_i\text{Var}(X_i)= \frac{1}{n^2}\cdot n\cdot\sigma^2_1= \sigma^2_1/n$
(if the $X_i$ are independent of each other).
http://en.wikipedia.org/wiki/Variance#Basic_properties
In summary: the correct term:
$\color{red}{(1)}$ has $\sigma^2/n$ terms because we're looking at averages and that's the variance of an average of independent random variables;
$\color{red}{(2)}$ has a $+$ because the two samples are independent, so their variances (of the averages) add; and
$\color{red}{(3)}$ has a square root because we want the standard deviation of the distribution of the difference in sample means (the standard error of the difference in means). The part under the bar of the square root is the variance of the difference (the square of the standard error). Taking square roots of squared standard errors gives us standard errors.
The reason why we don't just add standard errors is standard errors don't add - the standard error of the difference in means is NOT the sum of the standard errors of the sample means for independent samples - the sum will always be too large. The variances do add, though, so we can use that to work out the standard errors.
Here's some intuition about why it's not standard deviations that add, rather than variances.
To make things a little simpler, just consider adding random variables.
If $Z = X+Y$, why is $\sigma_Z < \sigma_X+\sigma_Y$?
Imagine $Y = kX$ (for $k\neq 0$); that is, $X$ and $Y$ are perfectly linearly dependent. That is, they always 'move together' in the same direction and in proportion.
Then $Z = (k+1)X$ - which is simply a rescaling. Clearly $\sigma_Z = (k+1)\sigma_X = \sigma_X+\sigma_Y$.
That is, when $X$ and $Y$ are perfectly positively linearly dependent, always moving up or down together, standard deviations add.
When they don't always move up or down together, sometimes they move opposite directions. That means that their movements partly 'cancel out', yielding a smaller standard deviation than the direct sum.