Sample Sizes and You!

by Niclas Darville

How to Use Polls in Writing

Bad

25% of Americans prefer cats, while 27% prefer dogs.

Good

Of the 500 people surveyed, 25% answered they prefer cats, while 27% that they prefer dogs.
  • The good example is factually correct; the bad one is not.
  • The bad example implies a 100% certainty betrayed by factors like variance and methodology.1
  • The good example demonstrates a differentiation between accuracy and precision.2
  • Based on the small sample size of 500, you actually can’t conclude more people prefer dogs than cats.
  • The good example states what the respondents said, not what they necessarily believe.

What Is the Margin of Error?

An expression you probably shouldn’t use. To hear it from AAPOR:

Margin of sampling error is an actual quantity we can measure. There is no such thing as a measurable margin of error for a poll.

Poll results are subject to lots of sources of errors ranging from how well the questions were designed and asked to how well the interview was conducted to how well the sample design was implemented. Good pollsters and researchers do everything in their power to minimize these other possible sources of errors, but they are non-measurable in any case, and one can never know the precise amount of error associated with any poll finding.

“Margin of Sampling Error”

“Sampling error is just one kind of survey error.” Instead, use the term (margin of) sampling error.

Margins of Standard Error

Margins of sample error.
Source code.
Sample size 95% conf. 98% conf. 99% conf.
50 ±13.9 ±16.5 ±18.2
100 ±9.8 ±11.7 ±12.9
200 ±6.9 ±8.2 ±9.1
400 ±4.9 ±5.8 ±6.4
500 ±4.4 ±5.2 ±5.8
700 ±3.7 ±4.4 ±4.9
1000 ±3.1 ±3.7 ±4.1
1300 ±2.7 ±3.2 ±3.6
1500 ±2.5 ±3.0 ±3.3
2500 ±2.0 ±2.3 ±2.6
5000 ±1.4 ±1.6 ±1.8

In Maths

function se68(p, n) {
    p = parseFloat(p);
    return Math.sqrt(p * (1-p) / n);
}

function se95(p, n) { return se68(p, n) * 1.96;  }
function se98(p, n) { return se68(p, n) * 2.33;  }
function se99(p, n) { return se68(p, n) * 2.575; }
        

In English