Wouldn’t it be great if we could run an A/B testing campaign and have instant data that allows us to make changes on the fly?
In some cases, that’s possible.
Setting Expectations Around A/B Testing
For example, if you’re A/B testing an email marketing campaign, as soon as one version or the other wins, the rest of the campaign can go out with the winning message.
And within 48 hours, you should have enough data to tell whether the campaign worked or not.
But other digital marketing platforms require more time before you can make a judgment on the success or the failure the campaign.
For example, it takes about 90 days before you can get a good read on whether or not paid search is a viable channel for your business.
Sure, early on, you’ll have some winning paid search ads and some losing ads, but you won’t get a statistically viable read until you’ve collected data over a longer period of time.
The Importance of Remaining Patient When A/B Testing
Even though some campaigns can provide you a quick, early indication on whether or not they work, other campaigns require much more time to fully play out. That’s not to say that you shouldn’t optimize along the way.
It’s just to say that in some cases, you’ll need a longer period of time before you decide to pull the plug (or increase the budget).
- Be realistic about how soon to expect results.
- Don’t end a trial before you have conclusive results.
- Don’t spend resources on a trial if you can’t see it all the way through.
In a similar sense, online display campaigns take about 90 days before they’re effectively optimized for success.
Because there are adjustments you should be making on your ads and where they run that can happen only as you collect data on the results.
In other words, as the ads run across 1000 websites, you’ll learn which websites are your top performers.
The only way to do that is to let the ads run and then make adjustments as the data comes in.
Search engine optimization also takes about 90 to 180 days to see the full effects.
If you’re developing content around a certain set of keywords, then it will require three to six months to see if search engines adjusted your rankings based on your changes.
Correct This Mistake In Three Simple Steps
It’s always a good idea to try out new tactics, but not at the expense of tried-and-true technologies and channels.
With that in mind, carve off a small percentage of your budget to test new technologies or tactics while continuing to optimize your traditional approaches.
By doing so, you’ll have a much more balanced and even-keeled campaign, while still taking advantage of emerging trends and tech – to see if they’re right for you.
- Keep doing what works for your company, even if it’s not the trending thing.
- Keep part of your budget for experiments. Expect a percentage of them to fail.
- Introduce change in small doses, so you can find the right balance of old standbys and new hot techniques.