Why You Need To Run Growth Experiments In 2022

It’s the new year. After recharging and reflecting, you’re ready to dive into work again and find new ways to make a difference.

When it comes to your marketing strategy, you’ll be tempted to make changes. Maybe change the channel spend, maybe tweak the creative, maybe adjust the performance bid strategy. Ask your agency to come up with a new campaign idea or ask your SEO team to target some different keywords.

That’s good, it’s good to keep things fresh. But if you’re doing a lot of tinkering, how are you measuring what’s working and what’s not? How are you scoping and planning changes? Without a proper process, you could find yourself in a position where you know that something has improved the marketing performance, but you’re not sure what or why. Or worse, you’ll find yourself in a position where the marketing performance has absolutely tanked and you don’t know where to start. Happy new year, eh?

It’s to stop clients getting into this kind of pickle that we always recommend having a growth experiment process in place. Here’s how it helps:

1)  Organisation

A huge reason to run growth experiments in 2022 is organisation. With a proper framework in place to plan, execute, measure and evaluate your experiments, you’ll find it so much easier to keep on top of what works and what doesn’t. 

Here’s how that framework might look:

Step 1: Objectives and key results. Based on analysis of your marketing funnel, what needles do you want to move, how much do you want to move them by, and when do you need to move them by?

Step 2: Hypothesis. Write a ‘because we have observed X, we hypothesise Y’ overview of what experiments you want to run to impact your objectives and key results.

Step 3: ICE scores. Assign Ice (Impact, Confidence, Effort) scores to your experiments. How much impact could it have, how much confidence do you have in that impact, and how easy will it be to run? Prioritise experiments with higher ICE scores. 

Step 4: Assign and manage. Assign execution of the experiments to the relevant people and ensure the experiments have proper project management in your software of choice (we prefer Asana). 

We’ve written more about how to build a growth framework here. 

2) Speed

By having defined experiments, project owners and timeframes, things tend to move quicker. There’s no more vague ‘let’s try this for a while and see if it works’. Instead you’ll be saying ‘if by this date, this needle hasn’t moved, we re-think and try something else’.

The result, almost always, is a faster process of marketing experimentation and improvement. You’ll be able to brainstorm, deploy, measure and evaluate ideas in a much faster cadence. 

3) Motivation

There’s definitely such a thing as staleness in marketing. With a new campaign, project or client there’s a huge amount of buzz and excitement, and everyone comes in full of ideas and enthusiasm.

But after a while, people can get bored. Marketing staff can fall into a routine, and the productivity of the marketing effort stalls. It becomes less fun and engaging, and results tend to suffer.

A rigorous process of running growth experiments is a fantastic way to keep things fresh. There’s always a new idea to get excited about trying and it keeps teams very task and mission oriented. And this isn’t about cracking the whip – we’ve found that marketing teams who are running growth experiments consistently find the process far more rewarding. 

4) Results

Fundamentally the biggest reason to run growth experiments in 2022 is to get results. It’s not just about project management and team engagement – it’s about improving the performance of your marketing.

We find growth experiments get us better results for these reasons:

• You’re thinking through what you’re testing. You’re not just throwing ideas out willy nilly, you are carefully considering what your objectives are and how to reach them. Applying ICE scores far reduces the amount of time wasted on fruitless experiments.
• You never settle. Once one round of experiments is done, you gear up and deploy the next. Running growth experiments gets you into a cycle of constantly finding marginal gains.
• You find new opportunities. Because of the pressure to run new experiments regularly, you will leave no stone unturned. To use a cliche, growth experiments encourage ‘thinking outside the box’.

 

Pitfalls To Avoid When Running Growth Experiments

It is possible to run into problems when running growth experiments. The main one being a poor framework leading to unreliable results, or false negatives. False positives can also happen, but false negatives are more common.

For example, a poorly set-up experiment may yield poor results in, say, PR. You may take that to mean the channel doesn’t work and close it down prematurely. And then find later down the line that actually it’s a great channel for you, and that the badly designed experiment gave you misleading information. 

The main ways you can avoid false negatives are:

• Use a proper growth framework, as outlined above. By using a clearly defined process of idea > hypothesis > ICE score > prioritisiation > project management you’ll have a much better chance of getting good data.
• Have realistic objectives. You are moving the needle, not changing the world. If your objectives are unrealistic (such as, half our CPA in 3 weeks) you experiments are bound to fail. These failures would almost certainly be false negatives.
• Run the experiments for long enough. You can’t run an experiment for a short period of time and assume the results would extrapolate out if run longer. Marketing sometimes takes time to get going – it takes time for ads to build memory structures in your audience’s mind, and it takes time for performance campaigns to gather enough data to optimise. If your experiment is too short you won’t have statistically significant data. There’s not really any set length they should run for, but for larger experiments we tend to run them for 3 months.
• Don’t rely too much on digital attribution. Especially in the post-IOS14 world, attribution can be hit and miss. Don’t necessarily believe Google Analytics all the time if it’s telling you that one channel is working and one isn’t. You’ll need to use a smart, nuanced approach to interpreting the data. 

We’ve written more on the subject of avoiding false negatives in your experiments here. 

Conclusion

We’re a growth marketing agency, so we’re bound to love growth experiments and suggest it as the medicine for everyone’s marketing department.

But the reason for this is we’ve tested it. We’ve run them for so many clients now, and got such great results, that we’ve become evangelical about the process.

So if you’re going to make one change in your marketing department this year, make it this one: run growth experiments, and do it properly!

---

Talk to a Growth Advisor

We create a clear, focused marketing strategy by combining our expertise with your knowledge of your business.