Power of the hypothesis test

Introduction

While studying inferential statistics many times we come across hypothesis testing. Hypothesis testing is a powerful tool in the statistics to test the claim by gathering sampled data. One might be doubtful of whether the conclusion that we get from the hypothesis testing is reliable or not. For this purpose, we calculate the power of the hypothesis test. It is often challenging to get the power of the test. In this article, you are going to learn about the power of the hypothesis test.

Real-life scenario

Consider the situation where we need to check if a person is guilty or innocent. We work on the principle that “No innocent person is liable for punishment”. Therefore, the person is innocent until proven guilty. We gather some evidence to prove the person guilty. If we fail in gathering such evidence, then we conclude that the person is innocent. A statistician can fix the hypotheses as follows.

H₀: The person is innocent

Vs

H₁: The person is guilty

In this case, we come across the four scenarios as follows.

We might conclude that the person is guilty when he is guilty, or we might conclude that the person is innocent when he is innocent. In both cases, we are in the favour of justice.

On the other hand, we might conclude that the person is guilty when he is innocent, or we might conclude that the person is innocent when he is actually guilty. Both these scenarios are in the favour of injustice.

We try none of the two errors occurs, but it is not possible to reduce the chances to occurring these two errors at the same time. Let us understand this in more detail as follows.

Procedure

In the hypothesis testing, first, we decide the null (H₀) and the alternative (H_a) hypotheses. After performing all the testing procedures, we either reject the null hypothesis or fail to reject the null hypothesis. At this point, four scenarios arise just like in the example above. These are as follows.

We might reject the null hypothesis (H₀) when it is actually false, or we might fail to reject the null hypothesis when it is actually true. These two are win-win situations where our decision is correct.

However, if we reject the null hypothesis (H₀) when it is actually true then we commit an error. Such type of an error is called Type I error. In addition, if we fail to reject the null hypothesis (H₀) when it is actually false then we commit another type of error. This is called a Type II error. Consider the following table.

 

Fact		The decision based on Data
		Reject Null	Fail to reject/accept Null
	Actually False	Reject null when it is actually false (Correct Decision)	Accept null when it is actually false (Type II Error)
	Actually True	Reject null when it is actually true (Type I Error)	Accept null when it is actually true (Correct Decision)

 

Among all these four scenarios, we focus on the last two scenarios when we commit errors. Our aim is to reduce both types of errors. Both the errors are the two sides of a coin. Therefore, if we try to reduce one then another gets increased.

The probability of committing type I error is denoted by α while the probability of committing type II error is denoted by ß.

Definition

The probability of correctly rejecting the null hypothesis is called the power of the hypothesis test. It is denoted by (1-ß).

Consider the following figure.

As you can see in the above figure, the power of the test (1-ß) increases with an increase in the significance level (α). It is impossible to reduce both errors at the same time. Therefore, we must reach the middle of the situation where our aim is to keep both types of errors as small as possible.

Conclusion

More the power of the hypothesis test more the reliability of the hypothesis test. Therefore, the power of the test is crucial in deciding the reliability of the hypothesis test.