When you design and interpret an A/B test, you must carefully choose which metrics to monitor and analyze. Metrics in experiments are typically divided into primary and secondary categories. Understanding the distinction is crucial for drawing valid conclusions and making sound business decisions.

Primary Metrics

Primary metrics are the main outcomes you want to influence with your experiment. These metrics are directly tied to your experiment's goal or business objective.

• Represent the main success criteria for your test;
• Used to determine if the treatment (the change you are testing) is successful;
• Directly related to what you want to improve.

Example: If you are testing a new checkout process, your primary metric might be the conversion rate—the percentage of users who complete a purchase.

Secondary Metrics

Secondary metrics are additional measurements that provide context or help you spot unintended side effects.

• Offer supporting information about user behavior or system performance;
• Help you interpret results more fully;
• Reveal negative changes or trade-offs that the primary metric alone might miss.

Examples:

• Average order value;
• Bounce rate;
• Page load time.

If your new checkout process increases conversion rate but also increases the number of failed payments, you need to know this to make an informed decision.

Why Metric Selection Matters

Selecting the right primary and secondary metrics is essential:

• If your primary metric does not align with your business objective, you may make changes that do not benefit your organization;
• Ignoring secondary metrics can blind you to negative side effects or missed opportunities for improvement.

Your choice of metrics also impacts how you communicate findings:

• Stakeholders need to know whether the experiment improved the primary metric;
• They also need to understand any trade-offs or unexpected outcomes revealed by secondary metrics.

Choosing and reporting on both types of metrics ensures your A/B test results are reliable, actionable, and aligned with your business goals.