A/B testing
Comparing two versions with separate audience groups to learn which performs better on a defined measure.
A/B testing compares versions of a subject line, signup page, or other element using separate groups. Define the question, primary metric, and observation period before starting. Random assignment helps reduce differences between the groups that could otherwise explain the result.
To test a signup headline, show each version to a comparable share of visitors and measure completed subscriptions. Changing the headline, layout, and offer together may compare two page concepts, but it will not identify which individual change mattered.
Small differences in small samples can be chance. Avoid declaring a winner after the first few clicks, and consider tracking limitations when choosing a metric. Email opens, for example, can be distorted by privacy features.
Related terms and guides
Explore the glossary
Build your understanding of newsletters, social media, and audience growth, one topic at a time.