The ABC’s of A/B Testing

Email on Acid recently posted a great blog post on using A/B testing to increase your clickthrough rate on copies of webpages, forms, and email. An A/B test involves taking similar renditions of a piece of content and changing a specific variable. It’s an easy way to try and better understand which elements of your copy or design resonated best with your audience based on their actions (i.e., clicks, time on site, etc.) At Sailthru, we agree that A/B testing is an effective way to better tailor your message to your audience, so we offer you the environment to leverage the A/B testing both for transactional and mass campaign mailings.

Sailthru’s A/B functionality allows:

  • Easy A/B Optimization: You can automatically divide up your list into percentages you wish to send to. Currently, you can split a mass mail campaign or transactional template up to 5 times to test variables across a single campaign. You can use an A/B split that measures two versions of a campaign against one another (a 50/50 split), or you can measure up to four versions of a campaign against one another and send a “final” version to a remaining percentage of your list (10/10/80 split).
  • Clear Reporting: Using our campaign reporting metrics, you can determine what campaign within your A/B test was most successful in regards to open, click, and purchase rates.
  • Coming Soon: Configuring an auto winner for the A/B test and automatically sending the final version of a campaign dependent on which test had the better click or open rates.

An example of a 5 section A/B test

As Email on Acid pointed out, a simple change such as a different subject line or the color of a button, can lead to a huge variance in response. The rule of thumb: keep it simple. Test just one variable at a time to isolate which resonates best with users. Then use those results to send out even more relevant messages that yield the greatest ROI on your email campaigns.