Notice: This page requires JavaScript to function properly.
Please enable JavaScript in your browser settings or update your browser.The First U-Test | U-Test
The Art of A/B Testing
course content

Course Content

The Art of A/B Testing

The Art of A/B Testing

1. What is A/B testing?
Let's StartPrepare for the TestExample A/A Test
2. Normality Check
About NormalityDescriptive StatisticsChallenge: Descriptive StatsHistograms and Box Plots Challenge: Plotting HistsChallenge: Plotting BoxplotsShapiro TestChallenge: Shapiro Test
3. Variances in A/B Testing
Violin and Swarm PlotsChallenge: Violin and Swarm Plots (Part 1)Challenge: Violin and Swarm Plots (Part 2)Levene's TestChallenge: First Levene's TestChallenge: Second Levene's Test
4. T-Test
Graphical Results of Previous StudiesThe First T-TestChallenge: Second T-testThe Third T-TestChallenge: Fourth T-Test
5. U-Test
Metrics Challenge: Define metricChallenge: Another Normality TestThe First U-TestChallenge: The Second U-TestChallenge: The Last U-TestThe Third U-testResults

bookThe First U-Test

The U-test, also known as the Mann-Whitney test, is a nonparametric statistical test used for comparing two independent samples. It is used when the data do not meet the assumptions of parametric tests, such as when the data distribution is non-normal or when the data are measured on an ordinal scale.

The U-test assesses whether there is a statistically significant difference between the medians of the two groups. It does not rely on any prior assumptions about the data distribution and is relatively robust to outliers or unusual values. Let's see how the U-test performs for our samples. Let's formulate the hypotheses:

H₀: The medians of the 'Conversion' metric in the control and test groups are equal.

Hₐ: The medians of the 'Conversion' metric differ between the control and test groups.


              
123456789101112131415161718

            
# Import libraries
import pandas as pd
from scipy.stats import mannwhitneyu

# Read .csv files
df_control = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/c3b98ad3-420d-403f-908d-6ab8facc3e28/ab_control.csv', delimiter=';')
df_test = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/c3b98ad3-420d-403f-908d-6ab8facc3e28/ab_test.csv', delimiter=';')

# Define the metric
df_test['Conversion'] = df_test['Purchase'] / df_test['Click']
df_control['Conversion'] = df_control['Purchase'] / df_control['Click']

# U-Test
statistic, p_value = mannwhitneyu(df_control['Conversion'], df_test['Conversion'])

# Result of the U-Test
print('Statistic:', statistic)
print('p-value:', p_value)
copy

The obtained statistical value and the low p-value indicate a statistically significant difference between the medians of the 'Conversion' metric in the control and test groups. We have sufficient evidence to reject the null hypothesis and accept the alternative hypothesis that the medians differ.

This means that there are statistically significant differences in the 'Conversion' metric between the control and test groups.

Based on the box plot, we can conclude that the median of the 'Conversion' metric in the test group is higher than the median in the control group.

Now it's your turn to conduct the U-test.

The conditions required for the **U-Test** are:

The conditions required for the U-Test are:

Select the correct answer

Everything was clear?

How can we improve it?

Thanks for your feedback!

Section 5. Chapter 4
some-alt