Let's say Johnny has nothing better to do but toss a coin all day long and record the results. By the end of the day, Johnny flipped the coin 1 million times and his results show 300,000 heads and 700,000 tails. Considering the coin and Johnny are both fair, the results show tails has a higher probability of 3:7 (heads:tails).
Then, instead of telling Johnny to find a better hobby, do you tell him to toss the coin a million more times to prove the Law of Large Numbers works? And, considering you know Johnny's results, do you bet on heads for the next million coin tosses knowing that the results should converge close to 1:1? In which case, are you saying the coin doesn't have a 50/50 chance any longer considering the Law of Large Numbers is now in your favor with those results?
I think most people would agree that a million coin tosses is a large enough result already. However, as a mathematician, do you say in such a case the amount of coin tosses isn't large enough for the Law of Large Numbers to take effect? And then if it's still not 1:1 after 2 million tosses but still at 3:7 (heads:tails), do you tell Johnny to toss it a million more times? When is it large enough or is it only enough when the results are near 1:1?
Best Answer
The Law of Large Numbers is not an empirical result. You cannot demonstrate this with your own finite-trial experiment. It is a mathematical statement which states that - if you're familiar with calculus and a bit of probability:
$$\lim_{n \to \infty}\dfrac{1}{n}\sum_{i=1}^{n}X_i = \mathbb{E}[X_1]$$
where $X_1, \dots, X_n$ are independent and identically distributed random variables.
In your situation, how the Law of Large Numbers applies is this: if I flip a coin infinitely many times, I get the theoretical probability of getting a heads or tails. Since, of course, you cannot flip a coin infinitely many times, it is thus impossible to demonstrate the Law of Large Numbers through a coin flip experiment.
So, you might ask, what do you do then? How can you test to see if the theoretical proportion of heads/tails is a certain value? Well, you use statistics. A situation like this is what a one-sample proportion $Z$-test is used for - however, one must always be cautious about using canned procedures to test hypotheses. You can read all about the faults of $p$-values and null-hypothesis significance tests (NHSTs) online through a quick search.
A famous quote which is quite relevant in this situation (of which I do not know the originator, apologies): Statistics means never having to say you're certain.