What is Statistic Hypothesis? Type 1 and Type 2 Errors

Statistical hypothesis H is an assumption about the type of distribution of the general population, which is tested on the available samples. If the distribution of the general population is known and it is necessary to check the assumption regarding the values of the distribution parameters using the samples, then such hypotheses are called parametric.

The hypothesis we directly want to confirm or disprove is called the main hypothesis (also called null hypothesis), the rest of the hypotheses are alternative. Let's look at an example for a better understanding.

Example 1

Suppose there are samples, and we want to check whether they are distributed according to the Gaussian law. In this case, the main and alternative hypotheses can look like this:

Main hypothesis: samples have Gaussian distribution.
Alternative hypothesis: samples have some other distribution.

Example 2

Suppose we know the data is Gaussian. We estimated the mean value, and the estimation is equal to 3.98. We want to check if the real mean is 3.98 or greater. In this case, the hypotheses are:

Main hypothesis: actual population's mean is equal to 3.98.
Alternative hypothesis: real populations mean greater than 3.98.

What is statistical criterion

Сriterion is a rule that is used to accept or reject hypotheses, usually it is some function that has samples as arguments. By the value of this function, we determine whether the main hypothesis is true or not.

If the function value falls into a certain area S, then we reject the main hypothesis, such an area S is called critical.

Type 1 and 2 errors

We will define the critical area using the statistical properties of our criterion. For this, we need to introduce the following two concepts:

1. Type 1 error (α-error) is a false positive error that occurs when we reject the null hypothesis even when it is true. It represents the probability of rejecting a true null hypothesis. The level of significance (α) is the probability of making a Type 1 error;
2. Type 2 error (β-error) is a false negative error that occurs when we accept the null hypothesis even when it is false. It represents the probability of failing to reject a false null hypothesis. The power of the test (1-β) is the probability of correctly rejecting a false null hypothesis.

In practice, we often manually choose a significance level that determines the critical region of our test. The critical value separates the rejection region (the tails of the distribution) from the non-rejection region based on the chosen significance level.

Example

For example, let's say we're testing a hypothesis with Gaussian-distributed criterion values. If we set the significance level to 0.05, the critical region will be determined accordingly.

Thus, if the criterion value falls into the rejection region, we consider that the null hypothesis is rejected. In this case, we are testing the right-handed hypothesis; respectively, the critical region is on the right.
If the alternative hypothesis was formulated as follows: real populations mean is less than 3.98 we would have other left-handed hypothesis and other critical region:

There are also two-sided critical regions in which the areas of rejection of the main hypothesis are both on the right and the left.