We’re Expanding A/B Testing to Our Mobile Platform So You Can “Always Be Testing”Apr 3, 2018 - by Vanessa Sorenson
Did you know A/B testing stands for “Always be testing?” OK, that’s not technically true, but it might as well be. We believe in always testing and encourage our customers to think the same way. And we’re thrilled to announce that we’ve introduced a new A/B testing functionality to our mobile platform.
With a goal of redefining personalization, Sailthru’s key differentiator is our integrated data sources. Because we combine mobile, email and web activity, our customers get the most complete view of their customers. There is one drawback there. If you can dynamically personalize all content in real time… where do you even begin? A/B testing is a great way to narrow that down, comparing everything from copy to images to calls-to-action to messaging times.
How Sailthru Mobile’s A/B Testing Works
Far too often, A/B tests are run too hastily to be effective. To make the most of our new functionality, you should have statistical significance. If the samples are too small, you won’t be able to determine any real incremental gains. Good A/B tests also include holdout groups excluded from the testing, much like a traditional control group.
Our A/B testing feature extends to push notifications, in-app messaging and message streams. Our functionality divides the audiences up, allowing you to create up to five different tests, essentially mirroring the Sailthru platform itself.
Users can select how much of the audience sees each version. Do you want an even split? Would you rather a larger cohort receives one message stream over another? All of that is customizable within the message composer app interface, with the added simplicity of a drop-down menu. That way, if you see Version C performing exceptionally well, you can adjust your test and send that out to the rest of your audience.
With push notifications, you can even use multivariate testing, an evolved version of an A/B test that allows testers to simultaneously test multiple variables. Multivariate testing is already available on Lifecycle Optimizer and will be open to the rest of the mobile platform later this year.