Mathematical Statistics – How to Calculate the 50th Percentile in a Small Data Set

mathematical-statisticsquantiles

I'm trying to learn the concept of percentile.

Question: Given these numbers: {1, 2, 3, 900}, I'm trying to calculate the 50th percentile.

My answer: 3. But different websites are saying: 2.5

My reasoning: Two values (1 and 2) are below the value number 3. There are overall 4 values in the data set, so 50% (2 our of 4) of the values are smaller than 3.
I'm using wikipedia's definition:

A percentile is a measure indicating the value below which a given percentage of observations in a group of observations falls

What am I missing?

Best Answer

The Wikipedia wording isn't wildly wrong but it doesn't give a precise rule, which is what you need.

Consider this variant on your argument. Two numbers of 1, 2, 3, 900 are above 2. There are 4 values in total, so 50% are larger than 2. So choose 2 as the answer.

What is reported as the middlemost (a word Galton used) value should not depend on whether you start at the lowest value and work up or start at the highest value and work down. There is a clear answer either way if the number of values is odd but we need a rule for the number of values being even, as is 4.

With an even number of values, using the midpoint between the two middle values (the "comedians", naturally) as the median or 50th percentile is explained as a convention to mathematical audiences and as a rule to everybody else.

NB: Which calculation rule to use for arbitrary percentiles is (surprisingly perhaps) wide open territory with on one count nine different ways to do it. That is well covered in other threads. Here I focus on the small fallacy exposed in the question.

Related Solutions

Solved – Is calculating a percentile the same as evaluating a cumulative density function

You are close but not exactly right. Remember that the area under a probability distribution has to sum to 1. The cumulative density function (CDF) is a function with values in [0,1] since CDF is defined as $$ F(a) = \int_{-\infty}^{a} f(x) dx $$ where f(x) is the probability density function. Then 50th percentile is the total probability of 50% of the samples which means the point where CDF reaches 0.5. Or in more general terms, the p'th percentile is the point where the CDF reaches p/100.

Terminology in Quantiles – Are There 99 or 100 Percentiles and Their Definitions

Both of these senses of percentile, quartile, and so on are in widespread use. It’s easiest to illustrate the difference with quartiles:

the “divider” sense — there are 3 quartiles, which are the values dividing the distribution (or sample) into 4 equal parts:
```
   1   2   3
---|---|---|---
```
(Sometimes this is used with max and min values included, so there are 5 quartiles numbered 0–4; note this doesn’t conflict with the numbering above, it just extends it.)
the “bin” sense: there are 4 quartiles, the subsets into which those 3 values divide the distribution (or sample)
```
 1   2   3   4
---|---|---|---
```

Neither usage can reasonably be called “wrong”: both are used by many experienced practitioners, and both appear in plenty of authoritative sources (textbooks, technical dictionaries, and the like).

With quartiles, the sense being used is usually clear from context: speaking of a value in the third quartile can only be the “bin” sense, while speaking of all values below the third quartile most likely means the “divider” sense. With percentiles, the distinction is more often unclear, but it’s also not so significant for most purposes, since 1% of a distribution is so small — a narrow strip is approximately a line. Speaking of everyone above the 80th percentile might mean the top 20% or the top 19%, but in an informal context that’s not a major difference, and in rigorous work, the meaning needed should be presumably clarified by the rest of the context.

(Parts of this answer are adapted from https://math.stackexchange.com/questions/1419609/are-there-3-or-4-quartiles-99-or-100-percentiles, which also gives quotations + references.)

Best Answer

Related Solutions

Solved – Is calculating a percentile the same as evaluating a cumulative density function

Terminology in Quantiles – Are There 99 or 100 Percentiles and Their Definitions

Related Question