Solved – How to calculate margin of error for a survey question

confidence intervalerrormultinomial-distributionsurvey

I am working on a perception survey of a nationally-representative sample ($n=1243$). I want to calculate the margin of error for each question. A typical question is coded like this:

1. very good
2. good
3. bad
4. very bad
98. refused (vol.)
99. don't know (vol.)

I believe I know how to calculate the margin of error for a dichotomous variable: $MoE=z\sqrt{p(1-p)/n}$. But I mostly have 4 options.

Best Answer

Often newpapers will present polls "yes/no", like/dislike, success/failure type polls with a margin of error of 3%. A version of this calculation, similar to the one you mention, as presented on page 92 of Schaeffer, Mendehall, and Ott's Elementary Survey Sampling text (6E) is

$2 \sqrt{\hat{V}(\hat{p})} = 2 \sqrt{\frac{\hat{p}\hat{q}}{n - 1} \frac{N - n}{N}}$

where $\hat{q} = 1 - \hat{p}$ and $\sqrt{\frac{N - n}{N}}$ is the, often ignored, finite population correction.

Now, these papers will poll roughly 1,000 respondents and (ignoring the finite population correction) this calculation returns roughly 0.032 at its maximum which is when $\hat{p} = \hat{q} = 0.5$.

To calculate a margin of error in the presence of multiple categories, consider a single category as the "success" and combine the remaining categories. Then, as mentioned in the comment by @rolando2, the above margin of error would put a bound on the proportion of respondents who chose that category.

Note that the identical formulas are also grounded in the identical variances of the closely related binomial and multinomial distributions which are both $n \hat{p}_i (1- \hat{p}_i)$.

links to mathworld
multinomial distribution
binomial

Related Question