Apprendre Metric Construction | Preparing Experiment Data

Constructing the right metrics from raw experiment data is essential for meaningful A/B test analysis. Metrics are quantitative measures that summarize the outcome of your experiment, such as conversion rate (the proportion of users who complete a desired action) or average order value (the mean revenue per transaction). The process begins by identifying which behaviors or outcomes matter most for your experiment goals. For example, if you are testing a new checkout page, your primary metric might be the conversion rate from visit to purchase. Secondary metrics could include average cart size or time on site.

Once you define your metrics, you need to translate raw data into these summary statistics. This often involves aggregating, filtering, and computing ratios or averages using Python libraries like pandas. The most common metrics in experiment analysis include:

Conversion rate: number of users who took a target action divided by total users;
Average order value: total revenue divided by number of orders;
Retention rate: number of returning users divided by total users;
Click-through rate: number of clicks divided by number of impressions.

Choosing clear, actionable metrics ensures your experiment results are interpretable and aligned with business objectives.


              1234567891011121314151617181920
            
import pandas as pd

# Example experiment dataset
data = {
    "user_id": [1, 2, 3, 4, 5, 6],
    "group": ["control", "control", "treatment", "treatment", "control", "treatment"],
    "converted": [1, 0, 1, 1, 0, 0],
    "order_value": [100, 0, 150, 200, 0, 0]
}
df = pd.DataFrame(data)

# Calculate conversion rate for each group
conversion_rates = df.groupby("group")["converted"].mean()
print("Conversion rate by group:")
print(conversion_rates)

# Calculate average order value for each group (only for users who converted)
average_order_value = df[df["converted"] == 1].groupby("group")["order_value"].mean()
print("\nAverage order value by group (converted users):")
print(average_order_value)

Tout était clair ?

Merci pour vos commentaires !

Section 4. Chapitre 5

Demandez à l'IA

Posez n'importe quelle question ou essayez l'une des questions suggérées pour commencer notre discussion

Suggested prompts:

Can you explain how to calculate other metrics like retention rate or click-through rate using this data?

What do these results tell us about the control and treatment groups?

How can I visualize these metrics for better comparison?

Awesome!

Completion rate improved to 3.23

Glissez pour afficher le menu

Conversion rate: number of users who took a target action divided by total users;
Average order value: total revenue divided by number of orders;
Retention rate: number of returning users divided by total users;
Click-through rate: number of clicks divided by number of impressions.

Choosing clear, actionable metrics ensures your experiment results are interpretable and aligned with business objectives.


              1234567891011121314151617181920
            
import pandas as pd

# Example experiment dataset
data = {
    "user_id": [1, 2, 3, 4, 5, 6],
    "group": ["control", "control", "treatment", "treatment", "control", "treatment"],
    "converted": [1, 0, 1, 1, 0, 0],
    "order_value": [100, 0, 150, 200, 0, 0]
}
df = pd.DataFrame(data)

# Calculate conversion rate for each group
conversion_rates = df.groupby("group")["converted"].mean()
print("Conversion rate by group:")
print(conversion_rates)

# Calculate average order value for each group (only for users who converted)
average_order_value = df[df["converted"] == 1].groupby("group")["order_value"].mean()
print("\nAverage order value by group (converted users):")
print(average_order_value)

Tout était clair ?

Merci pour vos commentaires !

Section 4. Chapitre 5