The alpha level tells us how likely we are to reject the null hypothesis when it is in fact true (false positive). In other words, if the alpha is set at 0.05, there is a 5% chance we will incorrectly reject the null hypothesis.

If we set alpha at 0.01, there is a 1% chance we will incorrectly reject the null hypothesis.

The alternative is that we say if the alpha is set at 0.05, there is a 95% chance we will accept the null hypothesis when we should not (false negative). If the alpha is 0.01, there is a 99% chance we will accept the null hypothesis when we should not (Andale, 2012).

When studying medication. If the hypothesis is the medicine is safe, the null hypothesis is the medication is not safe. Here we would want our alpha level low, a low chance of saying the medication is safe when we shouldn't.

If we are determining if singing in the shower improves attitude, we could set our alpha level higher. There would be little harm if we incorrectly accept the hypothesis and reject the null hypothesis.