Notice: This page requires JavaScript to function properly.
Please enable JavaScript in your browser settings or update your browser.The Third T-Test | T-Test
The Art of A/B Testing
course content

Course Content

The Art of A/B Testing

The Art of A/B Testing

1. What is A/B testing?
Let's StartPrepare for the TestExample A/A Test
2. Normality Check
About NormalityDescriptive StatisticsChallenge: Descriptive StatsHistograms and Box Plots Challenge: Plotting HistsChallenge: Plotting BoxplotsShapiro TestChallenge: Shapiro Test
3. Variances in A/B Testing
Violin and Swarm PlotsChallenge: Violin and Swarm Plots (Part 1)Challenge: Violin and Swarm Plots (Part 2)Levene's TestChallenge: First Levene's TestChallenge: Second Levene's Test
4. T-Test
Graphical Results of Previous StudiesThe First T-TestChallenge: Second T-testThe Third T-TestChallenge: Fourth T-Test
5. U-Test
Metrics Challenge: Define metricChallenge: Another Normality TestThe First U-TestChallenge: The Second U-TestChallenge: The Last U-TestThe Third U-testResults

bookThe Third T-Test

Let's review the plots for the 'Purchase' column of the control and test groups.

Levene's Test:


              
1234567891011121314151617181920

            
# Import libraries
import pandas as pd
from scipy.stats import levene

# Read .csv files
df_control = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/c3b98ad3-420d-403f-908d-6ab8facc3e28/ab_control.csv', delimiter=';')
df_test = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/c3b98ad3-420d-403f-908d-6ab8facc3e28/ab_test.csv', delimiter=';')

# Do Levene's test 
statistic, p_value = levene(df_control['Purchase'], df_test['Purchase'])

# Print result of Levene's test
print('Statistic:', statistic)
print('p-value:', p_value)

# Determine whether the variances are similar
if p_value > 0.05:
  print('The variances of the two groups are NOT statistically different')
else:
  print('The variances of the two groups are statistically different')
copy

Now perform a t-test for the 'Purchase' columns:

H₀: The mean values of the column do not differ between the groups.

Hₐ: The mean values of the column differ between the groups.


              
123456789101112131415161718

            
# Import libraries
import pandas as pd
from scipy.stats import ttest_ind

# Read .csv files
df_control = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/c3b98ad3-420d-403f-908d-6ab8facc3e28/ab_control.csv', delimiter=';')
df_test = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/c3b98ad3-420d-403f-908d-6ab8facc3e28/ab_test.csv', delimiter=';')

# Select only the 'Purchase' columns
data_control = df_control['Purchase']
data_test = df_test['Purchase']

# Do T-Test 
statistic, p_value = ttest_ind(data_control, data_test, equal_var=True)

# Print result of T-test
print('Statistic:', statistic)
print('p-value:', p_value)
copy

In this case, the p-value (0.350) is higher than the acceptable significance level (0.05), indicating that there is insufficient evidence to suggest that the mean values differ between the groups. Now it's your turn!

Everything was clear?

How can we improve it?

Thanks for your feedback!

Section 4. Chapter 4
some-alt