Is this percentage different from ...? The Binomial Test - Basics

A binomial test can be used to test if a percentage from a sample is significantly different from another percentage. For example: An advertisement claims that 80% of the respondents said that they would keep on using the product. You would like to test this claim and ask 5 people yourself. Only 3 of your respondents mention they would keep on using the product.

The question is now if based on your data you can say the claim is likely to be correct or not?

The exact binomial test was first described by Bernoulli (1713). The number of calculations increase quickly when the sample size becomes bigger. Before the computer did the calculations these calculations could be avoided by using some form of an approximation. The null hypothesis tested is as shown in the figure below.

In the example this is therefor: H_o: π = 0.80.

Please note that I use the Greek letter pi for the population proportion, and p for the sample proportion, while some other textbooks use p for the population proportion and p-hat for the sample.

As for the alternative hypothesis (H_a) there are three variations:

1. One-sided left-tailed
   In words: The probability of success in the population is less than …
   In symbols: H_a: π < π_H0
2. One-sided right-tailed
   In words: The probability of success in the population is more than …
   In symbols: H_a: π > π_H0
3. Two sided
   In words: The probability of success in the population is not …
   In symbols: H_a: π ≠ π_H0

In the example we would probably be interested in the first one (if the actual percentage turns out to be more than 80% we wouldn’t really get angry at them):
H_o: π < 0.80.

Now we need to decide on how we are going to test this. It might surprise you that there are actually various tests that could be used, and there is no general agreed upon method to decide which test to use. More on this in the next section.

The basic binomial test will look at how many possible combinations there are to get the same or less successes, determine the probability for each and add them all up. It does this by using the following formula.



An explanation of where this formula comes from can be found in Appendix I . To find the probability of getting a result or more extreme as in our sample we can either:

• Perform the calculation by hand using the formula
• Use a table to find the corresponding probability
• Use a software package
• Use an online calculator (e.g. StatTrek, VasSarStats, or GraphPad)

Each of the above methods needs three values for the input: The sample size (usually denoted by n, in the example n = 5) or sometimes called number of trials, the probability for success on each trial (usually denoted by π, in the example π = 0.8), and the number of success found in the sample (usually denoted by k, in the example k = 3).

We are interested in the probability of obtaining the same or more extreme output. This is why we look at ‘cumulative’ probability tables, or in the output look for P(X ≤ …). In the example all methods should yield a p-value of  0.2627 (26%). This is the probability of obtaining 3 or less out of 5 in a sample, if the probability for success on each trial in the population is 0.80.

In reporting the results of a binomial test the proportion in the sample, the assumed proportion and the significance of the test should be reported, as well as an interpretation. Our final conclusion could read something like:
Advertiser X claimed that 80% of their respondents would like to continue using their product. A sample of five people was selected to test this claim. Only three people in the sample indicated they would like to continue using the product. A one-tailed binomial test was performed to test if the claim could be rejected. The test resulted in p = .263 indicating that there is insufficient evidence to reject the claim.

References

Bernoulli, J. (1713). Ars conjectandi. Impensis Thurnisiorum, fratrum.