[Math] Revising probability using Bayes’ rule

bayesianprobability

Let's say $100$ numbers is picked from a set of numbers one by one with replacement, which is between $000$ to $999$. I'm required to give my subjective probability before the experiment and revise (or update) the my subjective probability based on the results obtained. I would like to know if I'm doing it correctly or is it logical to use Bayes' rule for revision: (I'am so confused with my answers)

Since there are $^{10}P_{3}=(10)(9)(8)=720$ 3-different-digits number (e.g. $012,256$) from $000-999$, my prior probability, $P(H)=\frac{720}{1000}=0.72$

Based on the results obtained, $80$ out of $100$ numbers are $3$-different-digits number. So the probability based on the data, $P(E)=\frac{80}{100}=0.80$

According to Bayes' rule, (from Wikipedia)
$$P(H|E)=\frac{P(E|H)\cdot P(H)}{P(E)}$$
where $P(H|E)$ is the posterior probability, which will also be my revised probability for this question

$P(E|H)$, or the likelihood, is calculated as follows:
$$P(E|H)=\binom{100}{80}(0.72)^{80}(0.28)^{20}=0.01815146392$$
Hence,
$$P(H|E)=\frac{(0.01815146392)(0.72)}{0.80}=0.01633631753$$

Is there something wrong with my method? Because my revised probability is so small compared to my probability given before the experiment ……

Best Answer

You are mixing up two different "levels" of probability. You have an unknown distribution for the bag of numbers, and let $H$ denote the proportion of mixed digit numbers in the bag. Your prior probability should be a distribution over the possibilities for $H$ (which is some number between $0$ and $1$, you don't know how many numbers are in the bag even!). So, you assumed a prior that suggests with certainty that the proportion of mixed digit numbers in the bag is $0.72$, i.e. your prior distribution is $P(H=0.72) = 1$ and $P(H\neq 0.72) = 0$.

This doesn't seem like a good starting guess (and it looks like if you applied the update rule anyway, you get the same distribution back, deterministically 0.72). Maybe you should start with $H$ being the uniform distribution on $[0,1]$?

Then the update rule will update the distribution over $H$. With the given observation you'd expect the new distribution of $H$ to become more centered towards $0.80$ after this one update? In this case you'll have to compute conditional densities.

Alternatively, if you want to work with a discrete distribution, you could assume a prior that $H$ can only be one of $\{0,0.25,0.5,0.75,1\}$ with equal probability (also probably not a good prior, as in the limit the update will center on one of these values, even though it's possible the bag has a proportion other than these ones), and after one update at least $0$ and $1$ will be eliminated. The point of the question I guess is to get a feel for how updates work.

Related Question