Pardot's A/B Testing for emails allows you to change elements in your email, segment your recipient list into a test audience and winning audience, and then use the email's engagement data to automatically determine and distribute the winning email.
- If an A/B test includes multiple prospects with the same email address, only the prospect with the most recent Last Activity field value will receive an email. They will receive version A, version B, or the winner, but not more than one email.
- You cannot pause an A/B test to edit the email before the winner is sent. However, you can pause the test, copy the email, and edit the new copy.
- If you pause an A/B test, and then later unpause it, the countdown to sending the winning email will resume from where it was paused. The time remaining before the winning email send is shown in the Winner section of the read repot for the A/B test.
- If you enable A/B Testing on an email, then change your mind, you can turn it off before sending the email. See How do I turn off A/B Testing for complete instructions.
- Prospects added to recipient lists after testing has begun will receive the winning version of the email.
- If you schedule an email, any changes you make to the prospect list before the scheduled send time will be reflected on the email send.
- Change One Element: Try only changing one element between version A and version B (for example: subject line, an image in the email body, or greeting). If too many elements are different you won't know what it was about the winning version that actually 'made' it win.
- Confirm Send Date/Time: Bear in mind the date and time your winning email is scheduled to be sent. If you choose to start your test at 11:00 AM and you set the test period to be 9 hours then your winning email will go out at 8:00 PM. We've highlighted this time to ensure you are ok with it before you click 'send' or 'schedule'.
- Balance Test Audience Size and Test Period Length: Test duration is really up to you as the sender. It's impossible to give precise instructions as test results will vary by industry, client base and topic. It's best to base your initial AB tests off of what has worked well in the past or any data you may have for your particular company. In general, test duration should be long enough that there is a reasonable expectation that an email recipient has had time to interact (open/click through) the email. In the case of a 24-hour sale, you might want to send the email 2 days out, with a 1-day test duration. For a case study, or industry news type of email, you might want to test opens over the course of a week. Test duration will not affect the validity of your A/B test, as long as all emails are sent at the same time, and data collection ends at the specified time. To ensure that the performance of the winning email stays the same between the test period and the full send, it is important to make sure that the conditions of the sending time are similar. For example, if you send the test on a Monday, you will also want your remaining emails to send on a weekday. You don't want to test until Friday and then send all your emails over the weekend.
- Test Every Email: More testing can only lead to more knowledge. Knowledge can only lead to better content. We believe in you!
InstructionsTo start an Email A/B Test:
- Navigate to Marketing > Emails > Drafts.
- Click +Send New List Email, or select an existing email draft to launch Email Flow.
- In the Building step, click Basic Info, then mark Enable A/B Testing to turn on A/B Testing for that email. This will copy your email, and creates an A version and B version of the same email.
- Next, click the A/B tab at the top of the editor to toggle bewtween versions. As a best practice, we recommend you change only one element in the email. Too many differing elements makes it difficult to pinpoint what made the winning version the winner.
- Once you're ready to send, click the Sending tab.
- Add your recipient and suppression lists. You must have at least 4 prospects on your recipient list to send the A/B email.
- Pardot will automatically designate 25% of the recipient list as the test audience. This is randomized. To adjust the percentage, either move the slider bar or enter a new percentage in the "What percent of your audience do you want to use for testing? box.
- View both versions of your email (A and B) in the right sidebar. Mouseover the desired version to view the email at full size.
- Choose to schedule your email or send immediately.
- To Schedule the Email:
- Click the Schedule button. Select a future date and time and click
the Schedule button again to confirm. The email is now scheduled to send
to your test audience on the chosen date, and will send to your winning audience after
the test period has ended.
- Click the Schedule button. Select a future date and time and click the Schedule button again to confirm. The email is now scheduled to send to your test audience on the chosen date, and will send to your winning audience after the test period has ended.
- To Send the Email:
- Click the Send button. The email will send immediately to your test audience, and will send to your winning audience after the test period has ended.
- To Schedule the Email:
Note: If you enable A/B Testing on an email, then change your mind, you can turn it off before sending the email. See How do I turn off A/B Testing for complete instructions.
Determining and Sending the Winner
Determining the Winner
Your test audience recipients, determined by your given percentage, will receive one of the two versions of your email. Half the test audience will receive Version A, and half the test audience will receive Version B. Pardot will determine the winner based on your chosen criteria. In the below example, the criteria is link clicks. So the email version with the most email link clicks in 1 day will be the winning email. Please note, link clicks or email opens that take place after the test has finished running will not be counted towards the winning email totals.
Note: In the unlikely event that emails A and B receive the exact same email click percentage or email open percentage during the test, Pardot will send email A.
Sending the Winning Email
The winning email is determined by the test audience's interaction with your email during the chosen test run time. If you chose the "Send Now" button when executing your test, the winning version will send immediately at the conclusion of the test.
Scenario: On Monday at 2:00 pm, you create your email and set up an A/B test. You choose to base the test on link clicks and provide a test period of 2 days. You click the "Send Now" button. Your Test Audience will immediately receive either Version A or Version B. The link clicks will be tallied and the winning email will be sent to the Winning Audience at 2:00 pm on Wednesday.
Sending the Winning Email if Scheduled
The winning email is determined by the test audience's interaction with your email during the chosen test run time. If you chose the "Schedule" button when executing your test, the test version will send once the scheduled date and time occurs. The winning version will send immediately at the conclusion of the test.
Scenario: On Monday at 2:00 pm, you create your email and set up an A/B test. You choose to base the test on link clicks and provide a test period of 2 days. You click the "Schedule" button and choose to send your email on Wednesday at 3:30 pm. Your Test Audience will receive either Version A or Version B on Wednesday at 3:30 pm. The link clicks will be tallied and the winning email will be sent to the Winning Audience at 3:30 pm on Friday, allowing for the test period.
Not what you're looking for? Check out these other articles:
A/B Test Reporting
4 Steps to Successful A/B Email Testing