descriptive statisticsdistributionshistorymathematical-statistics
For a unimodal distribution that is moderately skewed, we have the following empirical relationship between the mean, median and mode:
$$
\text{(Mean – Mode)}\sim 3\,\text{(Mean – Median)}
$$
How was this relationship derived?
Did Karl Pearson plot thousands of these relationships before forming this conclusion, or is there a logical line of reasoning behind this relationship?
Thank you for this simple-yet-profound question about the fundamental statistical concepts of mean, median, and mode. There are some wonderful methods /demonstrations available for explaining and grasping an intuitive -- rather than arithmetic -- understanding of these concepts, but unfortunately they are not widely known (or taught in school, to my knowledge).
1. Balance Point: Mean as the fulcrum
The best way to understand the concept of mean it to think of it as the balance point on a uniform rod. Imagine a series of data points, such as {1,1,1,3,3,6,7,10}. If each of these points are marked on a uniform rod and equal weights are placed at each point (as shown below) then the fulcrum must be placed at the mean of the data for the rod to balance.
This visual demonstration also leads to an arithmetic interpretation. The arithmetic rationale for this is that in order for the fulcrum to balance, the total negative deviation from the mean (on the left side of the fulcrum) must equal to the total positive deviation from the mean (on the right side). Hence, the mean acts as the balancing point in a distribution.
This visual allows an immediate understanding of the mean as it relates to the distribution of the data points. Other property of the mean that becomes readily apparent from this demonstration is the fact that the mean will always be between the min and the max values in the distribution. Also, the effect of outliers can be easily understood – that a presence of outliers would shift the balancing point, and hence, impact the mean.
2. Redistribution (fair share) value
Another interesting way to understand the mean is to think of it as a redistribution value. This interpretation does require some understanding of the arithmetic behind the calculation of the mean, but it utilizes an anthropomorphic quality – namely, the socialist concept of redistribution – to intuitively grasp the concept of the mean.
The calculation of the mean involves summing up all values in a distribution (set of values) and dividing the sum by the number of data points in the distribution.
$$ \bar{x} = (\sum_{i=1}^n{x_i})/n $$
One way to understand the rationale behind this calculation is to think of each data point as apples (or some other fungible item). Using the same example as before, we have eight people in our sample: {1,1,1,3,3,6,7,10}. The first person has one apple, the second person has one apple, and so on. Now, if one wants to redistribute the number of apples such that it is “fair” to everyone, you can use the mean of the distribution to do this. In other words, you can give four apples (i.e., the mean value) to everyone for the distribution to be fair/equal. This demonstration provides an intuitive explanation for the formula above: dividing the sum of a distribution by the number of data points is equivalent to partitioning the whole of the distribution equally to all of the data points.
3. Visual Mnemonics
These following visual mnemonics provide the interpretation of the mean in a unique way:
This is a mnemonic for the leveling value interpretation of the mean. The height of the A's crossbar is the mean of the heights of the four letters.
And this is another mnemonic for the balance point interpretation of the mean. The position of the fulcrum is roughly the mean of the positions of the M, E, and doubled N.
Once the interpretation of mean as the balancing point on a rod is understood, the median can be demonstrated by an extension of the same idea: the balancing point on a necklace.
Replace the rod with a string, but keep the data markings and weights. Then at the ends, attach a second string, longer than the first, to form a loop [like a necklace], and drape the loop over a well-lubricated pulley.
Suppose, initially, that the weights are distinct. The pulley and loop balance when the same number of weights are to each side. In other words, the loop ‘balances’ when the median is the lowest point.
Notice that if one of the weights is slid way up the loop creating an outlier, the loop doesn’t move. This demonstrates, physically, the principle that the median is unaffected by outliers.
The mode is probably the easiest concept to understand as it involves the most basic mathematical operation: counting. The fact that it’s equal to the most frequently occurring data point leads to an acronym: “Most-often Occurring Data Element”.
The mode can also be thought of the most typical value in a set. (Although, a deeper understanding of ‘typical’ would lead to the representative, or average value. However, it’s appropriate to equate ‘typical’ with the mode based on the very literal meaning of the word ‘typical’.)
Sources:
The question has already been answered in the affirmative, but let's approach this from the point of view of construction -- how do we make a set of data that does this?
First, note that we can always make all three location-measures greater than the range. Simply construct a preliminary data set that has median > mode > mean and compute the range. Now add (range-mean) + $\epsilon$ (for some small positive $\epsilon$) to all of the data values to get the final data set, whereupon the three location-measures will all exceed the range.
So we have now reduced the problem to one of finding a data set where median > mode > mean .
Imagine we already had some data with a suitable median and mode. To make the mean smaller than the median and mode, you simply place a single value far enough below the bulk of the data that the mean is pulled down; we can place a second value just above the bulk of the data to keep the median where it was, without changing the mode. So now we can modify an existing data set that simply has median > mode and obtain one which has the mean where we want.
So let us create one with median > mode. We can do this by having one value repeated (if it's the only value that occurs twice, it's the sample mode) and then adding enough other values to make the median larger. This is an example:
21, 21, 22, 23, 24
The median is 22 but the mode is 21.
Now let's add the two points as previously described, in such a way to make the mean 20 without changing the median or mode. The present points sum to 111, so we need two points that add to 140-111 = 29, and one of them should be just larger that 24. Let's make it 25. Then the smaller point is 29-25 = 4.
So now our data set is:
4, 21, 21, 22, 23, 24, 25
It has mean 20, mode 21 and median 22.
Now let's fix the relationship of those with the range. What's the range? It's 25-4=21, which is presently larger than the mean. We need simply add something to every data value to make the mean larger than 21, which leaves the range unaltered. Adding 2 will suffice. (Note that range-mean+1=2, so we can see that we took $\epsilon=1$)
So our final data set is
6, 23, 23, 24, 25, 26, 27
The range is still 21, the mean is now 22, the mode is 23, the median is 24
So this step by step approach is quite easy to use. In summary:
Make a small data set with median > mode by repeating the smallest value and having all the larger values distinct (it's easiest to use sorted values). Having 5 points is convenient (since it lets you specify the median by moving the middle value) but 4 is feasible if needed.
Obtain a mean below the median by adding two points that don't alter the median or mode (i.e. two distinct/singleton values will not disturb the mode, and placing them one either side the previous data will preserve the median; place the larger value just above all the present data and then compute the smallest so that the overall mean comes out just below the mode. This takes us to 7 data points.
Compute the range. Add a constant (range - mean + $\epsilon$) to all the data values, which guarantees that the mean exceeds the range. This is the final data set.
Checking those calculations in R:
x <- c(6, 23, 23, 24, 25, 26, 27)
data.frame(
range=diff(range(x)),
mean=mean(x),
mode=max(as.numeric(names(table(x))[table(x)==max(table(x))])),
median=median(x)
)
range mean mode median
1 21 22 23 24
(note that if we somehow happened to generate more than one mode, this calculation tries to find the largest of them)
Best Answer
Denote $\mu$ the mean ($\neq$ average), $m$ the median, $\sigma$ the standard deviation and $M$ the mode. Finally, let $X$ be the sample, a realization of a continuous unimodal distribution $F$ for which the first two moments exist.
It's well known that
$$|\mu-m|\leq\sigma\label{d}\tag{1}$$
This is a frequent textbook exercise:
\begin{eqnarray} |\mu-m| &=& |E(X-m)| \\ &\leq& E|X-m| \\ &\leq& E|X-\mu| \\ &=& E\sqrt{(X-\mu)^2} \\ &\leq& \sqrt{E(X-\mu)^2} \\ &=& \sigma \end{eqnarray} The first equality derives from the definition of the mean, the third comes about because the median is the unique minimiser (among all $c$'s) of $E|X-c|$ and the fourth from Jensen's inequality (i.e. the definition of a convex function). Actually, this inequality can be made tighter. In fact, for any $F$, satisfying the conditions above, it can be shown [3] that
$$|m-\mu|\leq \sqrt{0.6}\sigma\label{f}\tag{2}$$
Even though it is in general not true (Abadir, 2005) that any unimodal distribution must satisfy either one of $$M\leq m\leq\mu\textit{ or }M\geq m\geq \mu$$ it can still be shown that the inequality
$$|\mu-M|\leq\sqrt{3}\sigma\label{e}\tag{3}$$
holds for any unimodal, square integrable distribution (regardless of skew). This is proven formally in Johnson and Rogers (1951) though the proof depends on many auxiliary lemmas that are hard to fit here. Go see the original paper.
A sufficient condition for a distribution $F$ to satisfy $\mu\leq m\leq M$ is given in [2]. If $F$:
$$F(m−x)+F(m+x)\geq 1 \text{ for all }x\label{g}\tag{4}$$
then $\mu\leq m\leq M$. Furthermore, if $\mu\neq m$, then the inequality is strict. The Pearson Type I to XII distributions are one example of family of distributions satisfying $(4)$ [4] (for example, the Weibull is one common distribution for which $(4)$ does not hold, see [5]).
Now assuming that $(4)$ holds strictly and w.l.o.g. that $\sigma=1$, we have that $$3(m-\mu)\in(0,3\sqrt{0.6}] \mbox{ and } M-\mu\in(m-\mu,\sqrt{3}]$$
and since the second of these two ranges is not empty, it's certainly possible to find distributions for which the assertion is true (e.g. when $0<m-\mu<\frac{\sqrt{3}}{3}<\sigma=1$) for some range of values of the distribution's parameters but it is not true for all distributions and not even for all distributions satisfying $(4)$.