So, you know all about the importance of email nurturing and conversational Marketing when it comes to targeting leads. You’ve also mastered the A/B email subject line test. And you know exactly what character count gets you a 0.0001% uplift in your email open rate. What now?

As I’m sure you’ve found, in modern Marketing, there’s a lot more to effective email testing than just switching up your subject lines or the colours of your CTAs. But getting inspiration for your email testing is often far easier said than done!

That’s why, in today’s article, we’re going to be exploring the golden rules of creative, effective email testing for targeting leads and driving more inbound sales. 

But before we dive into the details, let’s take a quick look at the 5 best practices for email testing:

‍

#1 Always have a hypothesis

#2 Thou shalt not test for testing’s sake
‍
#3 Make sure your findings are statistically significant

#4 Be patient and give it time

#5 Test one variable per KPI

‍

Now that I’ve given you a little overview of what’s to come, let’s go ahead and address the above points in a little bit more detail. And, as always, I’ll be throwing in some examples for you along the way!
‍

#1 Create a hypothesis‍

A few years back, I had a boss who was always talking about the importance of hypotheses, and I’m very grateful to them for drilling this lesson into me as I’ve carried it with me ever since!

Creating a hypothesis is vital because simply saying “I will test Subject line A vs B” teaches us nothing. There has to be a reason behind it. And that reason can’t be “Because my boss asked me to!”

So often, when I ask companies what they have learnt from a test that they’re carrying out, all I get in return is that sometimes a stock subject wins over a personalised one and vice versa. The reason for this lack of broader learning is that no hypothesis was put behind the test itself.

Think about it: We drink water because we are thirsty, we eat delicious bacon because we are hungry... it’s the human condition! Why, then, would we test our emails without a reason?

So, what makes a good hypothesis? Let’s take a look at a couple of great examples.

Here’s an excellent hypothesis for an email subject line test: If I have a promotion and use a percentage discount, rather than a monetary value, in my subject line, I will get a higher open rate.‍

And here’s a great example of a hypothesis that could be put behind testing the CTAs within an email: Our CTA clicks are position-led, meaning the CTAs at the top of the email will always get more opens than those at the bottom.

‍

#2 Thou shalt not test for testing's sake‍

In some ways, this golden rule is a continuation of best practice #1, but I felt it pertinent enough to have a section of its own.

‘Testing for testing’s sake’ is a pet hate of mine, as I’ve seen far too many companies wasting their valuable time testing things week on week simply because they think they have to. 

As impressive as it might sound to some, testing things like ‘personalised subject lines vs static ones’ every week on a 10%/10% basis is actually a terrible idea! Why not just determine which one wins most on average, rather than testing the same thing week on week? 

If your personalised subject lines are consistently getting a higher open rate, then just send personalised subject lines to all of your subscribers right from the get-go! 

Adding a button here or there or changing your branding every week is a waste of time. If you tested yellow buttons a couple of weeks ago and the results were statistically significant, then use yellow buttons and retest this theory again in 6 months to see if your audience’s appetite has changed. Doing it again two weeks later is just a waste of your time!

In other words, test only when you need to - and when you have a hypothesis. Make sure all of your testing is done with a particular outcome in mind and that it will give you important learnings that you can then use in future.
‍

#3 Make sure your findings are statistically significant‍

Of course, we know that testing is important. Very, very important! But that doesn’t necessarily mean that every test’s results will be. If your findings from a test aren’t particularly significant, don’t feel the need to make up some hoo-ha about how influential the learnings are. 

Remember that a result is only significant if the test group’s results will make a real world difference to your numbers when applied to your entire database.

To better explain this, let’s walk through a little example: 

Say my contact database is made up of 100,000 people, and my average numbers are:

• 20% open rate (20,000 opens)
• 5% click rate (5,000 clicks)
• 0.2% conversion rate (200 conversions)

In a test group of 10% of my contacts (10,000 people), I would need to change my numbers somewhat to equate to big results when replicated to the rest of the database.

So, if I increased my conversion rate to 0.3% on the sample, that would equate to 10 extra conversions - 100 over the total database. On an example average order value of £36, that would mean an extra £3600 per email, which could ultimately bring me (based on 3 emails per week and the wind behind our backs) more than £500,000 in additional revenue per year.

On the other hand, increasing the conversion rate to 0.21% would mean 1 extra conversion per email, 10 over the total database and £5000+ per year. Yeah, it’s still something, but it might not be worth all the extra time, efforts and resources. See what I mean? ‍

‍

#4 Be patient and give it time‍

I can’t tell you how many times in my Marketing career I’ve heard the words “This just isn’t working.” And not after 6 months - but a week! Be patient, friends, and the results will come. 

Of course, how long each test takes will vary depending on a number of factors. You should give some of your tests more time and others less, but always make sure there is a reason behind every timeframe that you choose. 

As always, test your hypothesis for a set amount of time and if you are testing for opens, clicks, conversions or a change in behaviour, do not let things creep in which could eschew the results! 

For example, running a welcome program to get people to go from first to second purchase sooner will offer unclear and sometimes erroneous results if you dump a sale into the middle of the timeframe.

I also recommend giving your tests time as testing one thing from week to week is great, but not if you pull reports for last week’s email a week later and compare it to a test you started doing the day before. My rule of thumb is to check when the majority of your audience would have completed an action.

For example, if you are testing email opens, then you would want to see when 50% or more of your openers have done the task. In essence, if I were to email 100,000 people and have a 20% open rate (20,000 people), with 10,000 of those opens happening in the first 24 hours, I would use that as my measuring stick for opens. And the same goes for clicks, conversions, etc. Boom, real results!‍

‍

#5 Test one variable per KPI‍

Last but not least, I have to finish with the old multivariate warning. Remember, this is email… not the web!

Email is a soft, delicate beast which you cannot poke or prod too much. In other words, you can test many things at once but make sure you are testing just one thing at a time when it comes to opens, clicks and conversions.

For example, consider that subject lines and pre-headers are used to test opens, while CTAs and images are used to test clicks and conversions.

This means that while you can test for opens and clicks in the same email, you shouldn’t switch up both your CTAs and images when testing for conversions. You have to make sure you’re comparing apples to apples, or you won’t know which change has delivered you your results!  ‍

‍

Go forward and test‍

And there you have it. The five golden rules of email testing, to help you target leads, generate more inbound sales and identify more hot leads. 

Never underestimate the importance of a thorough email testing plan - but equally, take the time to get it right and follow these best practices without fail. The customers you attract will thank you for it later. ‍

‍

To find out more about how our online voice recording and messaging platform can supplement your business’ conversational Marketing strategy, get in touch with the team at telbee today.