Summary  
This chapter explains how to classify and choose primary metrics, which directly measure an experiment’s main goal, and secondary metrics, which provide context and reveal side effects, to ensure reliable interpretation of A/B test results.

General domain of usage  
E-commerce checkout optimization

When you design and interpret an A/B test, you must carefully choose which metrics to monitor and analyze. Metrics in experiments are typically divided into **primary** and **secondary** categories. Understanding the distinction is crucial for drawing valid conclusions and making sound business decisions.

## Primary Metrics

**Primary metrics** are the main outcomes you want to influence with your experiment. These metrics are directly tied to your experiment's goal or business objective.

- Represent the main success criteria for your test;
- Used to determine if the treatment (the change you are testing) is successful;
- Directly related to what you want to improve.

**Example:** If you are testing a new checkout process, your primary metric might be the **conversion rate**—the percentage of users who complete a purchase.

## Secondary Metrics

**Secondary metrics** are additional measurements that provide context or help you spot unintended side effects.

- Offer supporting information about user behavior or system performance;
- Help you interpret results more fully;
- Reveal negative changes or trade-offs that the primary metric alone might miss.

**Examples:**
- **Average order value**;
- **Bounce rate**;
- **Page load time**.

If your new checkout process increases conversion rate but also increases the number of failed payments, you need to know this to make an informed decision.

## Why Metric Selection Matters

Selecting the right primary and secondary metrics is essential:

- If your primary metric does not align with your business objective, you may make changes that do not benefit your organization;
- Ignoring secondary metrics can blind you to negative side effects or missed opportunities for improvement.

Your choice of metrics also impacts how you communicate findings:

- Stakeholders need to know whether the experiment improved the primary metric;
- They also need to understand any trade-offs or unexpected outcomes revealed by secondary metrics.

Choosing and reporting on both types of metrics ensures your A/B test results are reliable, actionable, and aligned with your business goals.

Why is it important to distinguish between primary and secondary metrics when designing and interpreting an A/B test?

Master the principles and practical application of hypothesis testing and A/B testing. Learn statistical tests, experiment design, data preparation, and analysis techniques using Python to make data-driven decisions in real-world scenarios.

Explore the core concepts of hypothesis testing, including t-tests, hypothesis formulation, and statistical assumptions, with practical Python examples.

Learn about z-tests for means and proportions, chi-square tests, and how to select the appropriate statistical test for different data types.

Build a strong foundation in A/B testing by understanding experiment structure, metric selection, randomization, and sample size considerations.

Learn how to structure, clean, and validate experimental data, including outlier detection and metric construction for robust analysis.

Apply statistical tests to A/B test data, interpret results, and understand key concepts like confidence intervals, uplift, and p-values.

Understand and implement guardrail metrics, monitor variance, and apply stopping rules to ensure experiment integrity and reliability.

Primary/Secondary Metrics

Primary/Secondary Metrics