I have a question on the fundamental meaning of probability.
The most familiar example of probability, is the probability of $\frac 12$ for $\text{H}$ and $\text{T}$, each for a single toss of an unbiased coin. This means, that any random toss of a coin (assumed to be unbiased) has equal chances of yielding a $\text{H}$ or a $\text{T}$.
But does it also imply that with a large number of tosses we shall
have an equal number of $\text{H}$'s and $\text{T}$'s? Or at least,
the general tendency(I don't know the technical term for it) of the
experimental result of a large number of tosses is to produce an equal
occurrence of $\text{H}$'s and $\text{T}$'s, though for a finite
number of tosses there still might be deviations form the expected
relative frequency?
Consider for example, a large (say $10^9$) number of tosses. For a unbiased coin, there should be $\frac {10^9}2$ occurrences of $\text{H}$'s and $\text{T}$'s at the end of all the tosses. Say we used a computer to toss all the coins and store the result as a single string of characters $\text{H}$ and $\text{T}$, unknown to us. After the experiment is over, the expected frequency is $\frac 12$ for each outcome. Suppose we start reading the generated string and about halfway, we realize we had much more $\text{H}$'s than $\text{T}$'s.
Then, is the probability of finding the next character to be
T greater than that, for finding it to be H? Does
the answer change if we know, that after the $10^9$ tosses, the total
occurences of both the outcomes were equal, even if we still are
unaware of the outcomes of individual experiments (the generated
string)?
Best Answer
The probabilities of outcomes and/or events of a random process are an inherent property of the process and have nothing to do with individual trials or any particular number of trials. Classical probability is the theoretical relative number for an infinite number of trials, or, if you will, the limit of the relative number as the number of trials goes to infinity. Classical probability theory deals with the many interesting situations and questions that arise in the case where all probabilities are known or determined by given conditions.
People have attempted to broaden this concept of probability to include belief etc. but that is not classical probability theory and in fact is really statistics and not probability theory. Statistics deals with situations where a theoretical probability is NOT known or determined. In that case we are limited to estimate or - in special cases that rarely arise - test. (Statistical testing is vastly overused and misunderstood.)