The Art of A/B Testing

The Art of A/B Testing

0%

1. What is A/B testing?

Welcome to the A/B testing course. In this section, we will tell you what controlled experiments are, where they are used, and why they are used. We will look at a specific example of a controlled experiment and talk about A/A testing. The Python programming language will help us with this. Well, let's get started!

Prepare for the Test

2. Normality Check

Before conducting a controlled experiment, we need to determine the characteristics of the sample. The type of tests that we will conduct a little later will depend on these characteristics. You will learn to build distribution histograms, box plots, and check the normality of the distribution. Let's get started right now!

About Normality

Descriptive Statistics

Challenge: Descriptive Stats

Histograms and Box Plots

Challenge: Plotting Hists

Challenge: Plotting Boxplots

Challenge: Shapiro Test

3. Variances in A/B Testing

The last thing to do before conducting a controlled experiment is to check the sampling variances. Some types of graphs and Levene's test will help us with this. Let's talk about it in more detail in this section!

Violin and Swarm Plots

Challenge: Violin and Swarm Plots (Part 1)

Challenge: Violin and Swarm Plots (Part 2)

Challenge: First Levene's Test

Challenge: Second Levene's Test

4. T-Test

So we got to A/B testing. Soon you will make your first parametric tests, or as they are also called - T-tests. We'll use all the knowledge we've learned in the previous sections: metric definition, normality testing, variance testing, and plotting. All this will help us get statistically significant results. Well, to work!

Graphical Results of Previous Studies

The First T-Test

Challenge: Second T-test

The Third T-Test

Challenge: Fourth T-Test

5. U-Test

It turns out that sometimes the samples do not have a normal distribution. This means that we cannot perform parametric testing. What to do in such a situation? Non-parametric testing. We will talk about him in the last section.

Challenge: Define metric

Challenge: Another Normality Test

The First U-Test

Challenge: The Second U-Test

Challenge: The Last U-Test

The Third U-test

Course Content

The Art of A/B Testing

Challenge: Define metric

Task

In this task, you must add the 'Average Earnings per Click metric' to the control and test samples. After that, you should plot the histograms of the new columns and mark the mean values of the metrics on the graph.

Import libraries.
Read the files.
Define the 'Average Earnings per Click' metric and add it to both samples.
Add a line of the mean value.

Task

In this task, you must add the 'Average Earnings per Click metric' to the control and test samples. After that, you should plot the histograms of the new columns and mark the mean values of the metrics on the graph.

Import libraries.
Read the files.
Define the 'Average Earnings per Click' metric and add it to both samples.
Add a line of the mean value.

Everything was clear?

Section 5. Chapter 2

toggle bottom row

Challenge: Define metric

Task

In this task, you must add the 'Average Earnings per Click metric' to the control and test samples. After that, you should plot the histograms of the new columns and mark the mean values of the metrics on the graph.

Import libraries.
Read the files.
Define the 'Average Earnings per Click' metric and add it to both samples.
Add a line of the mean value.

Task

In this task, you must add the 'Average Earnings per Click metric' to the control and test samples. After that, you should plot the histograms of the new columns and mark the mean values of the metrics on the graph.

Import libraries.
Read the files.
Define the 'Average Earnings per Click' metric and add it to both samples.
Add a line of the mean value.

Everything was clear?

Section 5. Chapter 2

toggle bottom row

Challenge: Define metric

Task

In this task, you must add the 'Average Earnings per Click metric' to the control and test samples. After that, you should plot the histograms of the new columns and mark the mean values of the metrics on the graph.

Import libraries.
Read the files.
Define the 'Average Earnings per Click' metric and add it to both samples.
Add a line of the mean value.

Task

In this task, you must add the 'Average Earnings per Click metric' to the control and test samples. After that, you should plot the histograms of the new columns and mark the mean values of the metrics on the graph.

Import libraries.
Read the files.
Define the 'Average Earnings per Click' metric and add it to both samples.
Add a line of the mean value.

Everything was clear?

Task

In this task, you must add the 'Average Earnings per Click metric' to the control and test samples. After that, you should plot the histograms of the new columns and mark the mean values of the metrics on the graph.

Import libraries.
Read the files.
Define the 'Average Earnings per Click' metric and add it to both samples.
Add a line of the mean value.

Section 5. Chapter 2

Switch to desktop for real-world practiceContinue from where you are using one of the options below

some-alt