2018–19 MacroMonitor Sampling Tolerance Tables

The numbers in the sampling tolerance tables are calculated using a design effect of √1.78.

Sampling Tolerance for Individual Percentages

Sampling Tolerance for Individual Percentages
(95% Confidence Level)

  Approximate Sampling Tolerance for a Survey Percentage at or near These Levels
Size of Sample
on Which Survey
Result Is Based
10% or 90% 20% or 80% 30% or 70% 40% or 60% 50%
4100 1.7% 2.3% 2.6% 2.8% 2.9%
3500 1.9 2.5 2.9 3.1 3.1
2000 2.5 3.3 3.8 4.0 4.1
1500 2.9 3.8 4.4 4.7 4.8
1000 3.5 4.7 5.4 5.7 5.8
750 4.0 5.4 6.2 6.6 6.8
500 5.0 6.6 7.6 8.1 8.3
250 7.0 9.4 10.7 11.5 11.7
100 11.1 14.8 16.9 18.1 18.5

How to Use This Table

Use this table when you are trying to determine if a statistically significant difference exists between two percentages in the same sample. Look at this chart for the column closest to the two percentages. Look for the row that is closest to the unweighted sample size. At the location where they cross is a number that must be smaller than the gap between the two percentages. For example, if you were looking at a question where 34% of the respondents had selected one answer and 25% had selected another and where the sample consisted of 1,100 unweighted respondents, you would look at where the 30% column intersected with the 1,000 sample size and find 5.4. Because the gap between 25% and 34% is greater than 5.4, these two numbers are statistically significant. Put another way: Significantly more respondents selected the 34% answer than the 25% answer.


Statistical Significance at 95% Confidence for Percentages of 10% and 90%

Table for Testing Two-Sample Statistical Significance at the 95% Confidence Interval for Percentages of 10% and 90%

Sample Size 4000 3500 2000 1500 1000 750 500 250 100
4000 1.8% 1.8% 2.1% 2.4% 2.8% 3.1% 3.7% 5.1% 7.9%
3500 1.8 1.9 2.2 2.4 2.8 3.2 3.8 5.1 8.0
2000 2.1 2.2 2.5 2.7 3.0 3.4 3.9 5.3 8.0
1500 2.4 2.4 2.7 2.9 3.2 3.5 4.0 5.4 8.1
1000 2.8 2.8 3.0 3.2 3.5 3.8 4.3 5.6 8.2
750 3.1 3.2 3.4 3.5 3.5 4.0 4.5 5.7 8.4
500 3.7 3.8 3.9 4.0 4.3 4.5 5.0 6.1 8.6
250 5.1 5.1 5.3 5.4 5.6 5.7 6.1 7.0 9.3
100 7.9 8.0 8.0 8.1 8.2 8.4 8.6 9.3 11.1

How to Use This Table

Use this table when you are trying to determine if a statistically significant difference exists between two percentages that are close to 10% or 90% of different (sub) samples. First, determine the unweighted sample sizes for the two populations. Next, look at the above chart for the intersection that comes closest to the two unweighted sample sizes. Finally, if the gap between the two percentages that you have is larger than the number in the intersection, then the two percentages are statistically different at the 95% confidence interval. For example, suppose the two percentages that you are comparing are 9% and 12% and the unweighted sample sizes are 450 and 705, respectively. Looking at the table above, the closest intersection of these numbers occurs at the 500 and 750 sample sizes. The number in this cell is 4.5. The gap between the two percentages is 3. Therefore, the difference is not statistically significant at the 95% confidence interval.


Statistical Significance at 95% Confidence for Percentages of 20% and 80%

Table for Testing Two-Sample Statistical Significance at the 95% Confidence Interval for Percentages of 20% and 80%

Sample Size 4000 3500 2000 1500 1000 750 500 250 100
4000 2.3% 2.4% 2.9% 3.2% 3.7% 4.2% 5.0% 6.8% 10.6%
3500 2.4 2.5 2.9 3.2 3.8 4.2 5.0 6.9 10.6
2000 2.9 2.9 3.3 3.6 4.1 4.5 5.2 7.0 10.7
1500 3.2 3.2 3.6 3.8 4.3 4.7 5.4 7.2 10.8
1000 3.7 3.8 4.1 4.3 4.7 5.1 5.7 7.4 11.0
750 4.2 4.2 4.5 4.7 5.1 5.4 6.0 7.6 11.1
500 5.0 5.0 5.2 5.4 5.7 6.0 6.6 8.1 11.5
250 6.8 6.9 7.0 7.2 7.4 7.6 8.1 9.4 12.4
100 10.6 10.6 10.7 10.8 11.0 11.1 11.5 12.4 14.8

How to Use This Table

Use this table when you are trying to determine if a statistically significant difference exists between two percentages that are close to 20% or 80% of different (sub) samples. First, determine the unweighted sample sizes for the two populations. Next, look at the above chart for the intersection that comes closest to the two unweighted sample sizes. Finally, if the gap between the two percentages that you have is larger than the number in the intersection, then the two percentages are statistically different at the 95% confidence interval. For example, suppose the two percentages that you are comparing are 18% and 26% and the unweighted sample sizes are 431 and 728, respectively. Looking at the table above, the closest intersection of these numbers occurs at the 500 and 750 sample sizes. The number in this cell is 6.0. The gap between the two percentages is 8. Therefore, the difference is statistically significant at the 95% confidence interval.


Statistical Significance at 95% Confidence for Percentages of 30% and 70%

Table for Testing Two-Sample Statistical Significance at the 95% Confidence Interval for Percentages of 30% and 70%

Sample Size 4000 3500 2000 1500 1000 750 500 250 100
4000 2.7% 2.8% 3.3% 3.6% 4.2% 4.8% 5.7% 7.8% 12.1%
3500 2.8 2.9 3.4 3.7 4.3 4.8 5.7 7.9 12.2
2000 3.3 3.4 3.8 4.1 4.6 5.1 6.0 8.0 12.3
1500 3.6 3.7 4.1 4.4 4.9 5.4 6.2 8.2 12.4
1000 4.2 4.3 4.6 4.9 5.4 5.8 6.6 8.5 12.6
750 4.8 4.8 5.1 5.4 5.8 6.2 6.9 8.8 12.8
500 5.7 5.7 6.0 6.2 6.6 6.9 7.6 9.3 13.1
250 7.8 7.9 8.0 8.2 8.5 8.8 9.3 10.7 14.2
100 12.1 12.2 12.3 12.4 12.6 12.8 13.1 14.2 16.9

How to Use This Table

Use this table when you are trying to determine if a statistically significant difference exists between two percentages that are close to 30% or 70% of different (sub) samples. First, determine the unweighted sample sizes for the two populations. Next, look at the above chart for the intersection that comes closest to the two unweighted sample sizes. Finally, if the gap between the two percentages that you have is larger than the number in the intersection, then the two percentages are statistically different at the 95% confidence interval. For example, suppose the two percentages that you are comparing are 68% and 72% and the unweighted sample sizes are 1,431 and 728, respectively. Looking at the table above, the closest intersection of these numbers occurs at the 1,500 and 750 sample sizes. The number in this cell is 5.4. The gap between the two percentages is 4. Therefore, the difference is not statistically significant at the 95% confidence interval.


Statistical Significance at 95% Confidence for Percentages of 40% and 60%

Table for Testing Two-Sample Statistical Significance at the 95% Confidence Interval for Percentages of 40% and 60%

Sample Size 4000 3500 2000 1500 1000 750 500 250 100
4000 2.9% 3.0% 3.5% 3.9% 4.5% 5.1% 6.1% 8.4% 13.0%
3500 3.0 3.1 3.6 4.0 4.6 5.2 6.1 8.4 13.0
2000 3.5 3.6 4.0 4.4 5.0 5.5 6.4 8.6 13.1
1500 3.9 4.0 4.4 4.7 5.2 5.7 6.6 8.8 13.2
1000 4.5 4.6 5.0 5.2 5.7 6.2 7.0 9.1 13.4
750 5.1 5.2 5.5 5.7 6.2 6.6 7.4 9.4 13.6
500 6.1 6.1 6.4 6.6 7.0 7.4 8.1 9.9 14.0
250 8.4 8.4 8.6 8.8 9.1 9.4 9.9 11.5 15.2
100 13.0 13.0 13.1 13.2 13.4 13.6 14.0 15.2 18.1

How to Use This Table

Use this table when you are trying to determine if a statistically significant difference exists between two percentages that are close to 40% or 60% of different (sub) samples. First, determine the unweighted sample sizes for the two populations. Next, look at the above chart for the intersection that comes closest to the two unweighted sample sizes. Finally, if the gap between the two percentages that you have is larger than the number in the intersection, then the two percentages are statistically different at the 95% confidence interval. For example, suppose the two percentages that you are comparing are 38% and 44% and the unweighted sample sizes are 950 and 3,450, respectively. Looking at the table above, the closest intersection of these numbers occurs at the 1,000 and 3,500 sample sizes. The number in this cell is 4.6. The gap between the two percentages is 6. Therefore, the difference is statistically significant at the 95% confidence interval.


Statistical Significance at 95% Confidence for Percentages around 50%

Table for Testing Two-Sample Statistical Significance at the 95% Confidence Interval for Percentages around 50%

Sample Size 4000 3500 2000 1500 1000 750 500 250 100
4000 2.9% 3.0% 3.6% 4.0% 4.6% 5.2% 6.2% 8.6% 13.2%
3500 3.0 3.1 3.7 4.0 4.7 5.3 6.3 8.6 13.3
2000 3.6 3.7 4.1 4.5 5.1 5.6 6.5 8.8 13.4
1500 4.0 4.0 4.5 4.8 5.3 5.9 6.7 8.9 13.5
1000 4.6 4.7 5.1 5.3 5.8 6.3 7.2 9.3 13.7
750 5.2 5.3 5.6 5.9 6.3 6.8 7.5 9.5 13.9
500 6.2 6.3 6.5 6.7 7.2 7.5 8.3 10.1 14.3
250 8.5 8.6 8.8 8.9 9.3 9.5 10.1 11.7 15.5
100 13.2 13.3 13.4 13.5 13.7 13.9 14.3 15.5 18.5

How to Use This Table

Use this table when you are trying to determine if a statistically significant difference exists between two percentages that are close to 50% of different (sub) samples. First, determine the unweighted sample sizes for the two populations. Next, look at the above chart for the intersection that comes closest to the two unweighted sample sizes. Finally, if the gap between the two percentages that you have is larger than the number in the intersection, then the two percentages are statistically different at the 95% confidence interval. For example, suppose the two percentages that you are comparing are 48% and 53% and the unweighted sample sizes are 1,031 and 528, respectively. Looking at the table above, the closest intersection of these numbers occurs at the 1,000 and 500 sample sizes. The number in this cell is 7.2. The gap between the two percentages is 5. Therefore, the difference is not statistically significant at the 95% confidence interval.