Statistics – Real Life Usage of Benford’s Law

applicationsbig-listsoft-questionstatistics

I recently discovered Benford's Law. I find it very fascinating. I'm wondering what are some of the real life uses of Benford's law. Specific examples would be great.

Best Answer

Some of the data generated by Math Stack Exchange itself (and, presumably, by similar sites) ought to approximately follow Benford's law. These would include the distribution of first digits of

  1. Frequency of tag use,
  2. Number of votes for questions,
  3. User reputation,
  4. Number of views for questions.

This is because Benford's law applies to exponentially growing quantities, and the total number of all of these quantities ought to be growing exponentially. It's only approximate because of artifacts in the way that some of these quantities are determined and because you need several orders of magnitude in order to see Benford's law really kick in.

Anyway, I decided to test this for the first three. The first digits weren't actually that hard to compile because the site allows you to sort the first three from highest to lowest. (Unfortunately, it does not allow you to sort the fourth that way, and I don't feel like wading through 2200+ questions to collect the data.) Here are the results I got on the first three.

Tag Use

(Data collected October 25, 2010.)

alt text

Not a bad fit, especially when you consider that there are only three orders of magnitude represented in this measure. There are a disproportionately large number of tags that were created but only used once, which explains the larger frequency of 1 as a first digit.

Votes Per Question

(Data collected October 26, 2010.)

alt text

Also not a bad fit, especially since there are currently only two orders of magnitude represented in this measure. Also, I threw out the questions with 0 and negative numbers of votes.

User Reputations

(Data collected October 25 and 26, 2010.)

alt text

This is the worst fit of the three, as the frequency of 1 as a first digit is so much larger than the others. However, there are a very large number of users who have never posted a question or an answer and so have a rep of 1. And, for reasons unknown to me, there are also a large number of users who have a reputation of 101, despite never having asked or answered a question.

If you remove 1 as a possible first digit and then rescale the Benford law probabilities to consider only 2 through 9 as possible first digits then the picture looks like the following, which is a much better Benford fit.

alt text

Admittedly, none of these data sets would pass Pearson's chi-square test for goodness-of-fit with respect to the Benford probabilities at a reasonable level of significance. However, given some of the artifacts in the data and the fact that there are relatively few orders of magnitude represented, the fit with Benford's law is really not that bad.

Related Question