Notice: This page requires JavaScript to function properly.
Please enable JavaScript in your browser settings or update your browser.
The First T-Test | T-Test
The Art of A/B Testing
course content

Зміст курсу

The Art of A/B Testing

The Art of A/B Testing

1. What is A/B testing?
2. Normality Check
3. Variances in A/B Testing
4. T-Test
5. U-Test

bookThe First T-Test

The t-test provides two main results:

  • t-statistic. This is the calculated measure of the significance of the difference between the group means. A higher t-statistic value indicates a larger difference between the groups.
  • p-value. It represents the probability of obtaining the observed or even larger difference between the groups if the actual difference is zero (i.e. if the null hypothesis is true). A smaller p-value suggests that it is less likely for such a large difference to occur by chance alone and provides more evidence in favor of the alternative hypothesis of a difference between the groups. Typically, if the p-value is less than 0.05, we consider the difference statistically significant.

The t-test is widely used in scientific research, medicine, economics, and other fields to determine the statistical significance of differences between groups.

Let's formulate hypotheses:

H₀: The mean values of the 'Impression' column in the control group and the test group are not different.

Hₐ: The mean value of the 'Impression' column in the control group is different from the mean value in the test group, indicating a statistically significant difference between the groups.

123456789101112131415161718
# Import libraries import pandas as pd from scipy.stats import ttest_ind # Read .csv files df_control = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/c3b98ad3-420d-403f-908d-6ab8facc3e28/ab_control.csv', delimiter=';') df_test = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/c3b98ad3-420d-403f-908d-6ab8facc3e28/ab_test.csv', delimiter=';') # Select only the 'Impression' columns data_control = df_control['Impression'] data_test = df_test['Impression'] # Do T-Test statistic, p_value = ttest_ind(data_control, data_test, equal_var=True) # Print result of T-test print('Statistic:', statistic) print('p-value:', p_value)
copy

The result indicates that there is a statistically significant difference between the two groups in terms of their mean values.

The p-value is smaller than the specified level of significance.

This suggests rejecting the H₀ of no difference between the groups.

The negative value of the t-statistic may indicate that the mean value in the first group is smaller than in the second group. Now it's your turn to perform your first t-test.

Все було зрозуміло?

Як ми можемо покращити це?

Дякуємо за ваш відгук!

Секція 4. Розділ 2
some-alt