Course Content
The Art of A/B Testing
The Art of A/B Testing
1. What is A/B testing?
The First T-Test
The t-test provides two main results:
- t-statistic. This is the calculated measure of the significance of the difference between the group means. A higher t-statistic value indicates a larger difference between the groups.
- p-value. It represents the probability of obtaining the observed or even larger difference between the groups if the actual difference is zero (i.e. if the null hypothesis is true). A smaller p-value suggests that it is less likely for such a large difference to occur by chance alone and provides more evidence in favor of the alternative hypothesis of a difference between the groups. Typically, if the p-value is less than 0.05, we consider the difference statistically significant.
The t-test is widely used in scientific research, medicine, economics, and other fields to determine the statistical significance of differences between groups.
Let's formulate hypotheses:
H₀: The mean values of the 'Impression'
column in the control group and the test group are not different.
Hₐ: The mean value of the 'Impression'
column in the control group is different from the mean value in the test group, indicating a statistically significant difference between the groups.
The result indicates that there is a statistically significant difference between the two groups in terms of their mean values.
The p-value is smaller than the specified level of significance.
This suggests rejecting the H₀ of no difference between the groups.
The negative value of the t-statistic may indicate that the mean value in the first group is smaller than in the second group. Now it's your turn to perform your first t-test.
Everything was clear?