Example

Task

Your company wants to increase sales conversion on the website by testing different page design options.

Step 1

Which metric is better to choose?

If the goal of the A/B test is to increase the conversion of sales on the site, then the metric that should be chosen is the conversion rate. The conversion rate reflects the ratio of the number of users who performed a certain targeted action (for example, made a purchase) to the total number of visitors to the site.

Average purchase frequency per click:

So, if we are testing different page design options and offers with the goal of increasing sales conversions, we should choose conversion rate as the main metric for evaluating test results.

Step 2

For the conversion rate metric, it is better to choose samples that reflect the real flow of users. For example, if the metric is purchase conversion, then the sample should contain site visitors who have the potential to make a purchase. It is important that the samples are represented by a sufficiently large number of users to ensure the statistical validity of the test results. In addition, it is worth considering that the samples should be equal in size and nature in order to avoid distortion of the test results.

Step 3

The design of the product purchase button will be changed for the experimental group. For the control group, the design will remain unchanged.

Step 4

We test both buttons and record the results in two csv format tables. Let's look at them:


              12345678
            
# Import pandas library
import pandas as pd

# Read .csv file
control_group = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/c3b98ad3-420d-403f-908d-6ab8facc3e28/Section+1/datasets/control_group_1.csv')

# Print head of dataframe
print(control_group.head())


              12345678
            
# Import pandas library
import pandas as pd 

# Read .csv file
test_group = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/c3b98ad3-420d-403f-908d-6ab8facc3e28/Section+1/datasets/test_group.csv')

# Print head of dataframe
print(test_group.head())

Now we need to add our metric.


              12345678910111213
            
#Import pandas library
import pandas as pd 

#Read .csv file
control_group = control_group = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/c3b98ad3-420d-403f-908d-6ab8facc3e28/Section+1/datasets/control_group_1.csv')
test_group = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/c3b98ad3-420d-403f-908d-6ab8facc3e28/Section+1/datasets/test_group.csv')

#Define metric
control_group['Conversion'] = (control_group['Purchase']/control_group['Page view']).round(2)
test_group['Conversion'] = (test_group['Purchase']/test_group['Page view']).round(2)

#Print head of dataframe
print(test_group.head())

Step 5

What about A/B testing?

Not so far, unfortunately, we cannot be sure that the results of the experiment are representative. How to make sure of this?

We have to perform A/A testing.

Everything was clear?

Thanks for your feedback!

Section 1. Chapter 3