Notice: This page requires JavaScript to function properly.
Please enable JavaScript in your browser settings or update your browser.The First T-Test | T-Test
The Art of A/B Testing
course content

Course Content

The Art of A/B Testing

The Art of A/B Testing

1. What is A/B testing?
Let's StartPrepare for the TestExample A/A Test
2. Normality Check
About NormalityDescriptive StatisticsChallenge: Descriptive StatsHistograms and Box Plots Challenge: Plotting HistsChallenge: Plotting BoxplotsShapiro TestChallenge: Shapiro Test
3. Variances in A/B Testing
Violin and Swarm PlotsChallenge: Violin and Swarm Plots (Part 1)Challenge: Violin and Swarm Plots (Part 2)Levene's TestChallenge: First Levene's TestChallenge: Second Levene's Test
4. T-Test
Graphical Results of Previous StudiesThe First T-TestChallenge: Second T-testThe Third T-TestChallenge: Fourth T-Test
5. U-Test
Metrics Challenge: Define metricChallenge: Another Normality TestThe First U-TestChallenge: The Second U-TestChallenge: The Last U-TestThe Third U-testResults

bookThe First T-Test

The t-test provides two main results:

  • t-statistic. This is the calculated measure of the significance of the difference between the group means. A higher t-statistic value indicates a larger difference between the groups.
  • p-value. It represents the probability of obtaining the observed or even larger difference between the groups if the actual difference is zero (i.e. if the null hypothesis is true). A smaller p-value suggests that it is less likely for such a large difference to occur by chance alone and provides more evidence in favor of the alternative hypothesis of a difference between the groups. Typically, if the p-value is less than 0.05, we consider the difference statistically significant.

The t-test is widely used in scientific research, medicine, economics, and other fields to determine the statistical significance of differences between groups.

Let's formulate hypotheses:

H₀: The mean values of the 'Impression' column in the control group and the test group are not different.

Hₐ: The mean value of the 'Impression' column in the control group is different from the mean value in the test group, indicating a statistically significant difference between the groups.


              
123456789101112131415161718

            
# Import libraries
import pandas as pd
from scipy.stats import ttest_ind

# Read .csv files
df_control = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/c3b98ad3-420d-403f-908d-6ab8facc3e28/ab_control.csv', delimiter=';')
df_test = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/c3b98ad3-420d-403f-908d-6ab8facc3e28/ab_test.csv', delimiter=';')

# Select only the 'Impression' columns
data_control = df_control['Impression']
data_test = df_test['Impression']

# Do T-Test 
statistic, p_value = ttest_ind(data_control, data_test, equal_var=True)

# Print result of T-test
print('Statistic:', statistic)
print('p-value:', p_value)
copy

The result indicates that there is a statistically significant difference between the two groups in terms of their mean values.

The p-value is smaller than the specified level of significance.

This suggests rejecting the H₀ of no difference between the groups.

The negative value of the t-statistic may indicate that the mean value in the first group is smaller than in the second group. Now it's your turn to perform your first t-test.

Everything was clear?

How can we improve it?

Thanks for your feedback!

Section 4. Chapter 2
some-alt