The ease of A/B testing makes it one of the most time-tested tools for digital marketers. Marigold Engage by Sailthru by Marigold has always been a big believer in test-and-learn strategies because of their value to make informed decisions that are rooted in data. Approximately 59% of companies perform email A/B tests for good reason. In the short term testing can improve open and click rates. In the long term, A/B testing can improve the user experience and increase a customer’s lifetime value. 

What Can A/B Testing Help Me Do? 

The first step to creating your test-and-learn strategy is to define your objective. Some of the most common objectives our customers focus on include: 

  • Growing the brand’s database
  • Improving viewer engagement
  • Increasing conversion rates
  • Enhancing cross-channel communications
  • Improving the customer experience with better insights
  • Retaining and winning back lapsed customers and subscribers

Setting Up Your Testing Environment

Creating a hypothesis, or more simply stated as “What question are we trying to answer?” is the most vital component when thinking about how to set up your test and control groups. 

Once you’ve established your hypothesis, it’s time to choose how you will design your test. Marigold Engage by Sailthru by Marigold offers traditional A/B testing and multivariate testing. Traditional A/B testing looks at one specific question like which subject line is more engaging: the one with a person’s name in the subject line, or a simple emoji instead? Multivariate testing looks at many factors at one time to see what combination of factors yields the best results. Multivariate testing is typically used for more complex hypotheses like how email cadence can impact opt-out rates, or what incentives are driving loyalty program sign-ups. Multivariate tests are executed over a longer period, either weeks or months, versus one or two days. 

The Rules for Testing

Sailthru by Marigold recommends five guidelines for effective testing: 

  • Test one thing and keep everything else constant: Testing personalization in the subject line? Keep the send time the same for each stage of the testing process.
  • Test enough times: We recommend at least four rounds of testing for every hypothesis.
  • Test on a big enough audience: Your sample sizes should be more than 1000 contacts, otherwise your test is not statistically significant.
  • Choose a random sample across all segments: Random sampling means that any visitor of your website has the same probability to be chosen to see a variation of your A/B test. Biased sampling can unintentionally occur if you test only for a couple of days during a promotional period, like back-to-school shopping, or only on weekdays. This could lead to bias because your customers who see the A/B promotion or email would only be the ones who shop on weekdays and could unintentionally not reach your shoppers who tend to shop on weekends.
  • Test the same thing again in 12 months: Things change. When emojis first appeared, they were perceived as unprofessional and unsophisticated. Nowadays emojis are a sure-fire way to boost brand awareness if they are used on the right target audience and align with your brand’s tone of voice. 

A/B testing is critically important to improving your email marketing campaigns. To learn more about getting the most out of your testing strategies, download the guide from Campaign Monitor, also a part of the Marigold family, The Ultimate Guide to A/B and Multivariate Testing. 

 

Download the Guide