As I understand it the IQR specifies the dispersion of the data by taking the difference between Q3 and Q1 (i.e., the range that the middle 50% of the data lies), while std specifies dispersion using all the data, to where 68% of the data belong. Does this imply that the std is always larger than (or equal to?) iqr?
Solved – QR ever be larger than standard deviation
descriptive statisticsdistributionsstandard deviationvariance
Related Question
- Solved – Interquartile range exceeds the median
- Solved – Why would we ever use Covariance over Correlation and Variance over Standard Deviation
- Solved – Are variances within a group always larger than between groups? When is this not true
- Solved – Is it bad to have error bars constructed with standard deviation that spans to the negative scale while the variable itself shouldn’t be negative
Best Answer
Your argument about $68\%$ of the density contained within $1$ standard deviation (of the mean) is true for the Normal distribution, but not in general. There are examples where the standard deviation exceeds the IQR and examples in the other direction as well.
Let $X$ have a t-distribution with $v$ degrees of freedom.
The standard deviation of $X$ is $$\sigma = \sqrt{\text{Var}(X)} = \sqrt{\frac{v}{v-2}}$$
The IQR can be found using the quantile function of the t-distribution. In R, we have
We have that $\sigma = IQR$ for $v\approx 3.61$. When $v < 3.61$, we have $\sigma > IQR$ and we have $\sigma < IQR$ otherwise. Since a normal distribution can be achieved in the limit as $v\rightarrow \infty$, this demonstrates that $\sigma < IQR$ for a normal distribution.![enter image description here](https://i.stack.imgur.com/Iq1OG.png)
Sal considers the interesting question, can IQR be larger than $2\sigma$?. Indeed, this will occur for the symmetric Beta distribution when the shape parameter is small.
Let $X \sim Beta(\alpha, \alpha)$. Then $IQR = 2\sigma$ when $\alpha = 0.5$, $IQR > 2\sigma$ when $\alpha < 0.5$ and $IQR < 2\sigma$ otherwise.![enter image description here](https://i.stack.imgur.com/nNpBL.png)
We have established examples, where $IQR > \sigma$ and examples where $IQR > 2\sigma$. It is not possible, however, for $IQR$ to exceed $4\sigma$ when the distribution of $X$ is symmetric about its mean $\mu$. Applying Chebyshev's Inequality, we have $$P(|X-\mu| \leq 2\sigma) \geq \frac{3}{4}.$$
If the density function of $X$ is symmetric about $\mu$, we have $$P(|X-\mu| \leq \text{IQR}/2) = \frac{3}{4}.$$
This implies that $2\sigma \geq \text{IQR}/2$, or equivalently $$\text{IQR} \leq 4\sigma$$