Is it easier to pass a test with more questions and more mistakes allowed, or less questions but less mistakes allowed

binomial distributionprobabilityprobability distributions

$Q$ is a set of yes or no questions. I know the answer to $q_{know}$ of these questions ($0\leq q_{know}\leq|Q|)$, but I have to guess for the remaining ones, with a $0.5$ probability of guessing correctly. A test $T$ is generated by randomly selecting $n$ of these questions
($|T|=n$). To pass the test, I can make $k$ mistakes at the most, with $k>0$.

  1. What is the probability of passing the test, as a function of these parameters?
  2. Assuming $n/k%$ is constant (e.g., for every ten questions in the test, 1 mistake is allowed) is it better to take the test with more or less questions?

This question came up during a discussion and I thought that the number of questions doesn't matter because the probability is always the same, but I'm starting to think that it might be more complicated than this. Should it be a product of binomial distributions?

Best Answer

Consider flipping a fair coin 10 times vs 1000 times. Is it more likely to get 7 or more heads in the first case, or 700 or more heads in the second case? I think you can intuit that the former is more likely than the latter, and you'd be exactly right. This is the Law of Big Numbers: the bigger your sample size, the closer you can expect the observed frequency to get to the probability. As you yourself started to suspect, this is indeed all about the distribution: the distribution curve gets more narrow the greater the sample size is. So, with a probability of 0.5, you can expect to get the number of heads with 1000 flips to be closer to 0.5 than if you just flip 10 times.

So: if the probability of you guessing the answer an individual question correctly is greater than the percentage of question you need to get correctly, then you should go for the test with more questions. If the probability of guessing correctly is smaller than the needed percentage, then you should go for the smaller test.