Short answer: A/B testing cold email means sending two versions that differ in one thing, then keeping the version that gets more replies. Test one element at a time, split the list evenly, send enough volume to trust the result, and change only the winner. The goal is steady improvement, not a single perfect email.
What to test, in order
Test the elements that move the most first. Start with the subject line because it drives opens, then the first line because it drives whether people keep reading, then the offer and the call to action. Design and length matter less, so test them only after the big levers are settled.
- Subject line, for open rate.
- First line, for whether they keep reading.
- Offer, for whether they reply.
- Call to action, for whether they book.
How to set up a clean test
Split your list randomly into two equal groups. Send version A to one and version B to the other at the same time of day. Change only one element between them. If you change the subject and the offer at once, you will not know which one moved the result, and the test teaches you nothing.
How much volume you need
Small tests lie. A difference across twenty emails is noise. Aim for at least one hundred to two hundred sends per version before you trust the winner, and more if the two numbers are close. Reply rate is a small percentage, so you need volume for the gap to mean something real.
| Sample per version | How much to trust it |
|---|---|
| Under 50 | Noise, do not decide |
| 100 to 200 | Usable for clear gaps |
| Over 300 | Reliable even for small gaps |
Read the result honestly
Pick the metric that matches what you tested. Judge subject lines on open rate and offers on reply rate. If the two versions are within a point or two, call it a tie and keep the simpler one. Then take the winner, make it the new control, and test the next element against it. This loop is how a plain email becomes a strong one over a month.
Frequently asked questions
Q: Can I test more than one thing at once?
A: Not when you are learning. Change one element per test so you know what caused the change. Multivariate testing needs far more volume.
Q: How long should a test run?
A: Run until each version has enough sends and a few days of replies, usually one to two weeks. Replies lag, so do not call it on day one.
Q: What if both versions perform the same?
A: Keep the simpler or shorter one and move on to testing a different element. A tie still removed one variable.
Q: Should I test send times too?
A: Yes, once copy is settled. Send time can shift open rates, so test it after the subject and offer are strong.
Q: Do I need a special tool to A/B test?
A: Most sending platforms have a built in split test. If yours does not, split the list by hand and compare the two groups.
Next step
Want help building a testing plan for your campaign? Join a free live session or see all courses. Related: what a good reply rate is and the best time to send cold emails.
For the complete testing and sending system, take the Cold Email Outreach Blueprint course.