Intro to ab testing

Ab testing is the science of identifying an important element of your marketing, and creating a duplicate version of it with 1 noticeable thing changed.

This new change is called the variant.

When you run the original (control) with the variant to the same audience, during the same time frame then you can using statistical testing to confirm which version performed better.
‍

Where most people go wrong with ab testing

These three things are the biggest mistakes we see in ab testing.

1. Most people test multiple things at once thus making it impossible to confirm what change caused the increase/decrease in performance.
2. They test something so minor that it’s very difficult to tell the difference between the two versions.
3. Last, they don’t use consistent statistical testing methods to confirm if the change that saw an increase will always show the increase.

If you follow this guide, you will be miles ahead of others who do "ab testing."
‍

Components of an ab test

Do you remember in high school when you took statistics?

This is where it comes in handy.

The short and sweet of statistical testing is that we need to select a confidence interval (90 or 95%).

Then we need to select a variable to test - this is what we are hoping to improve.

Last we need to select a performance indicator - this is how we will measure success.

With these we can run an ab test.
‍

Types of ab tests

We like to classify ab tests into 3 different kinds of ab tests (campaign level, ad level, and landing page level tests).
‍

Campaign level tests

Starting out at the campaign level we include these because they have to do with audience targeting.

A lot of our clients don’t know if certain audiences perform better than others. So this is the place to test just that.

We also like to test bidding strategies, locations, bidding modifiers and more at this level.

Most ab testing agencies don’t even consider testing these which baffles our minds.

These ab tests can significantly move the needle on your campaigns performance.

The primary objective here is to measure conversions / qualified leads - since the audience targeting dictates a huge percentage of who will interact with your ads.
‍

Ad level tests

This is where most agencies go wrong and test multiple ad graphics with multiple differences in each just to call it ab testing.

These ab tests should be methodical and strategic.

At this level, you can change 3 major components (Imagery, messaging, or the CTA)

The primary objective at this level is CTR since it is the ads job to get someone to the landing page.
‍

Landing page tests

Another place where ab testing agencies go wrong is here.

They will only test things that are small and hardly noticeable (like changing the color of a button that doesn’t show until 50% down the page).

The kinds of ab tests at the landing page level that we have seen do best are Hero/Headline changes.

These can be in the form of the headline message, the hero imagery, or hero CTA (color and language)

Another test we like to run at the landing page level is short vs long landing pages.

The primary objective here is conversion rate for qualified leads.

It’s not conversions because we are focused on the landing page’s effectiveness at turning qualified clicks into qualified leads.
‍

The first 3 ab tests we run in month 1

We strategically run 3 ab tests all at the same time while ensuring appropriate controls are in place so that we don’t get confounding results.
‍

Bid strategy ab test

At the campaign level, we like to test a bidding strategy change.

On google ads this can be in the form of maximize clicks vs ECPC.

On LinkedIn Ads this can be in the form of maximum delivery vs manual bidding.

This test is foundational for the overall setup optimization of your campaigns.

If we can uncover an objectively better bidding strategy then we are miles ahead of standard campaigns.
‍

Headline ad tests

The first ad level ab tests we like to run revolve around the message.

So we will create multiple iterations of ads that have the same imagery and CTA but noticeable changes in headlines.

An example could be the headlines including stats whereas the variant describes the problem without stats.

This helps us understand if our ICP is driven by technical messaging or not.
‍

Landing page tests

Last, we like to test different imagery on the landing page.

This means that the imagery needs to be significantly different in the header.

And example can be something like the hero has a picture of the software whereas the test is someone using the software.

This helps us understand if our ICP is interested in the details of the software or the result the software brings.
‍

Why you should work with a paid media agency for your ab testing needs

There are ab testing agencies out there but we recommend not working with them for two big reasons.
‍

Ab testing agencies don’t help you drive traffic

One thing often not explained to clients before working with an ab testing agency is that there needs to be significant, quality traffic being driven to your website for ab testing to happen.

So if you aren’t driving hundreds of clicks to your site each month then you will be not getting a lot out of your ab testing efforts.
‍

If ab testing agencies offer paid media, they likely aren’t that good at it

This shouldn’t come as a surprise but you wouldn’t hire your dentist to do your taxes.

Ab testing agencies offer ways to get more clicks to your site but they will not be ahead of the game in terms of executing quality, and strategic paid media campaigns
‍

Why working with a paid media optimization agency is the best choice for performance

At the end of the day, you want increased marketing performance.

Doesn’t matter if it comes from ab testing, seo, paid media etc

The short and sweet of it is this...

Ab testing agencies help squeeze out value from campaigns that have already been proven and built out by a paid media agency.

If you don’t have proven campaigns already then ab testing findings will be useless because even if you have a lot of traffic but it’s from the wrong target audience then your findings won’t translate to the winning audience you eventually find from working with paid media experts.
‍

The difference of ab testing approaches

When you work with Optimize, you find winning campaigns and you find them through strategic ab testing to find the best performing campaigns

We win when we find optimized campaigns and the only way we do that is through strategy and ab testing.

Ab testing agencies build off of paid media agencies that are already dialed in.

A unique advantage of working with Optimize is our unique understanding of paid media and how ab testing can be used to find winning campaigns.

And we stay dialed in when we are driving serious traffic that helps make decisions on more in depth ab tests.

If you need help navigating whether or not your should work with us, then book some time with us to learn more.

Until next time.