The Marketer’s Guide to Models

By Will Critchlow

Clients desperately want forecasts and as a result, we hear these questions all the time – all boiling essentially down to some variant on:

These are scary questions to hear from a client – and they are easy ones to push back on – often with some variant on the common reasons reputable SEO firms don’t promise guaranteed search rankings. Especially early in our careers as consultants, it is incredibly easy to fall into the trap of pushing back on the specific thing the client asked for (because we’ve learned that we need to be a trusted advisor rather than just a paid lackey who’ll say yes to any request). Well, I’m here today to tell you that’s a mistake.

Very rarely is a client looking for a contractual guarantee of future performance (if they are, the only way this can work is on a pay-for-performance model which would have to be the topic of its own post. But suffice it to say that there’s no such thing as a successful cheap pay-for-performance contract – any that work are going to be very expensive so plan accordingly). And we shouldn’t fall back on the Google position that reputable firms don’t offer ranking guarantees. That’s not what smart clients are looking for.

So, if they’re not looking for a contractual guarantee, what are they looking for? In my experience, it boils down to needing the tools to better make or better justify decisions:

• Those who hold budgets are typically looking at a range of options and trying to figure out which is best, and whether any of them have a chance of meeting the objectives at a reasonable cost
• Anyone who has to build organisational support for a plan needs answers to questions about expected outcomes – what exactly this looks like will vary by organisation (see for example Amazon’s memo-based process) but will fundamentally involve the output of this kind of process

In both these cases, I’d argue that the true need is more like a range of likely scenarios than it is a single forecast. Everyone understands that things might not work perfectly as planned or precisely as in the best case scenario – but by presenting some scenarios, they get a feel for the range of outcomes and a sense of the risk associated with them.

But I’m getting ahead of myself. Let’s step back to some definitions.

The difference between models, scenarios, and forecasts

There are three terms that I hear thrown around somewhat interchangeably – model, scenario, and forecast – that I’d like to give my definition for to make sure we are talking about the same things:

Model

A model in this context is similar to the concept of a “scale model” – in other words, a smaller, simplified version of the real thing that shows how it works. Typically built in spreadsheets in a business context, this is a mechanism for turning inputs and assumptions into outputs or outcomes – it essentially encapsulates an opinion on “what the levers do”.

In the context of digital marketing, it might be a spreadsheet that captures how much more traffic we might get over time if we change the pace of publishing different kinds of content on our site.

Scenario

A scenario is a model along with a specific set of inputs and assumptions – not necessarily the set of assumptions that the author believes to be the most likely – it’s more a case of “what happens if the world looks like this and we do that?”. Scenarios are most useful and interesting when we produce a range of them in order to discuss the likelihood and attractiveness of different outputs.

In the context of digital marketing, this might include assumptions around how average tests change conversion rates and what proportion might turn out to be successful along with inputs about how many experiments we might be able to run per quarter.

Forecast

A forecast is essentially an opinionated scenario – it’s the outcome you think is likely if you follow a certain path and try to execute a certain plan. It is possible to have a group of forecasts – either under varying plans (inputs) or with different assumptions (essentially a group of forecast scenarios – e.g. conservative / aggressive).

The power of scenarios

For the most part, when people ask for “a” forecast, the best answer is a range of scenarios that show possible outcomes and the sensitivity to assumptions and inputs. It’s much easier to put these scenarios together if you first focus on the much lower-pressure task of building a model without the stress of having to calibrate the right assumptions and perfect plan.

A range of scenarios helps in most requests for a forecast – it’s useful for getting sign-off as you can focus on how much of the range of outcomes has sufficient return to justify the investment. It’s also good for attempting to constrain the limited number of cases where the activity turns out not to have been worthwhile – and even allows you to change the plan to reduce the impact of those failure cases if needed.

Earlier in the process, it’s good for figuring out if something is a vaguely sensible plan before getting too committed to it – does the top end of the range of possible outcomes have a big enough impact to be worthwhile? If you can’t even model a result that’s worth it, you need to act immediately to revise the plan because it’s clearly not going to be something you’re going to want to pitch.

What do you do with the output?

It’s good practice to visualise the output of the model even as you are building it – we’re visual creatures, and having a couple of charts updating in the background makes it much easier to spot flawed assumptions, issues with the model, and outright bugs:

It’s also generally the case that you are going to want to summarise the output of the model to make it useful – there are two key ways that I tend to do this:

1. With charts and images – this is particularly powerful for comparing the outcomes of different scenarios or understanding the sensitivity to particular assumptions
2. To drive headline numbers – it can be powerful to open a presentation with a “wow” statistic (“this investment can drive £Xm of incremental revenue over the next three years”). You can do this by opening with a single number from the output of a scenario and then back it up by building up a “pyramid” of supporting information and assumptions – drilling only as deep down the stack as this particular audience needs you to

Note: Sometimes you will share the actual model – but generally only as a deeeeeep appendix. The model is how you create the scenario(s) which becomes the data at the bottom of the pyramid. Even the exact scenarios might only be an appendix – instead, you need to remember to tell the story that you discover this way.

How do you build a great model?

The first thing to note is that the pressure feels much lower as soon as you acknowledge that you are just building scenarios at first. The first scenarios don’t have to be close to realistic – you are just trying to capture the mechanics of how things change when you move input assumptions around. This stage is just about getting the levers in place. You can set sensible assumptions later.

Top-down vs bottom-up models

Broadly speaking, there are two ways of connecting the levers – you can build your model “top-down” or “bottom-up”. These are finance terms – where they define the differences as:

• Top-down model: is one which starts with the market as a whole and estimates market share, forecasts trends over time in % terms etc.
• Bottom-up model: is one which starts with the product or service itself, looks at units, at production capacity, at route to market etc.

Although the outputs of our work are often financial, they are not exclusively so – and certainly not in the sense of concepts like market share being consistently useful, so my versions for the kind of models marketers build are:

• Top-down model: looks at percentages and trends – which tends to make it better for modelling established sites and operations – these models are more likely to talk about the keyword universe and share of voice or to extrapolate from current performance with tweaks to the growth curve
• Bottom-up model: looks at raw numbers – typically thinking in pieces of content, individual channels, specific routes to users – hence these models tend to be better for modelling new sites, sections and markets. They are more likely to talk about individual keyword demand, production capacity, and other raw activities

Properties of good models

As you’re building your model, here are some pointers to help you stay on the right track:

1. Keep your inputs separate – I like to end up with them on a totally different sheet in Excel when I’m ready to ship my model, but while you’re building it you probably want to keep the assumptions physically close so you can tweak them easily. The key thing here is that you don’t want them scattered throughout the model. You want them separated and grouped together so that you can see them all, they are enumerated, and you can tweak them to understand their effects
2. Think about key inputs and what direction they move your key outputs. What’s the relationship? Does doubling the input double the output? More? Less?
3. Which are the biggest effects? Model them as monolithic effects initially and then later you can break them down. For example, you might say that the number of successful tests is a key factor. Start with that. You can break it down to attempted tests later and get closer to specifics that are in your control
4. As you start to see the model take shape, spend some time playing with the inputs and watching the outputs. Don’t worry about overt realism yet, but instead focus on the dynamics – does everything move in the direction you expect? Does it move in the ratios or proportions you expect? Explore the space of possible inputs a bit.

Specifically for models of marketing activities, there are a couple of very common specific themes:

• You will generally need some concept of a time delay – I like to build models with time periods running left to right across the sheet. You will typically find that you end up setting up your model so that there is a cascading effect where changes in one row in a given time period cascade to future time periods as they move down through the rows
• You will often need to factor competition into your models – either as explicit assumptions of how fast they are going, or more often as a more generic “how hard will it be to achieve this portfolio of effects”. I tend to prefer to model it the second way – focusing on competition as an abstract dampener or as a range of difficulties

Define your inputs and assumptions

Inputs and assumptions look similar, but they have some key differences:

• Assumptions are things that you’re just trying to get right – and that are generally properties of your world that aren’t going to change. Something like “time for Google to recrawl and reindex my homepage” would be an example of an assumption in most models
• Inputs are explicitly designed to be changed – this is how you build scenarios – they correspond to different decisions. Something like “number of contract content writers” would be an example of an input

As you initially build the model, don’t worry at all about putting anything realistic in for the assumptions and inputs. In fact, I often put outlandish values in initially just to make sure that I see the effect I expect from very high or very low values. Get the mechanics working, then, before you start playing with the inputs, get your assumptions to reasonable levels:

1. True values: if you can get the actual values from your own site, then this do this. Remember that assumptions control the behaviour of your model, but they’re not the things you are trying to change to do better
2. Benchmark: if you don’t have your own data, can you lay your hands on industry data? You will often be able to find public data sources or reports that give you a sense of reasonable ranges, and/or you can often ask around in industry circles – see if your frenemies will tell you how they’re doing
3. Intuition: even if you don’t have accurate data for your own environment, and no-one you know will share yours, you may have a good intuitive sense of where the right answer might roughly lie. If you do, you can sense-check your ranges – socialise your views by discussing ranges with industry contacts – even if they won’t tell you their numbers, you can often check with them that you’re in the right ballpark
4. Use the model: failing all of that, you can sometimes still set sensible assumptions by playing with the model and getting familiar with how the output varies and making sure that the outputs seem sensible – this is most useful when you are trying to work backwards – “to achieve outcome X, we’d have to be getting Y% of tasks above Z threshold”

Run some scenarios

Now you’re ready to run some scenarios. Remember that a scenario is what I’m calling “a model along with a specific set of inputs and assumptions”. So you’ve set your assumptions above – meaning that the way you generate scenarios is by tweaking your inputs. I like to start by just playing about with unrealistic ranges of each input – typically I find at least one bug in my model this way when it completely fails to react to an input or does something unexpected.

More interestingly, even when you’ve built a highly-complex and detailed model, you will often find that its outputs are only truly sensitive to one or two key inputs and that the output changes very little as you play around with the others.

Even if it’s not perfect, you start to be able to shape your plan using a model in this state. In particular, you can focus your energy on pushing those specific parts of the plan that move the key inputs. I’m reminded of this quote:

“It’s important to make good decisions. But I spend much less time and energy worrying about “making the right decision” and much more time and energy ensuring that any decision I make turns out right.”

Sean McNealy – a co-founder of Sun Microsystems and its long-time CEO (see this HBR article)

The mechanics of building a model

You’ll notice that I haven’t really dug into the actual mechanics of getting models from idea to implementation. For most people, this will involve spreadsheets, and the great thing about spreadsheets is that you can use them with almost any level of computer skill – from typing each cell into a table all the way up to complex linked models. For an excellent primer, I still recommend an article that a colleague of mine at the time (Mike Pantoliano) wrote back in 2012 or so: Excel for SEO.

I wanted to share something real, but everything I’ve built recently has been too niche or too confidential, so I put together a completely made-up dummy one. When you take a look at it, you’ll realise that it’s obviously not designed to be useful – but rather to illustrate some of the points I have been making above. I have annotated it with notes to show where I have demonstrated key elements. Although it’s not large or sophisticated, I have done my best to show most of the key features you’d need to build something much more involved.

I’ll leave you with one final tip: formatting and colour-coding is your friend. You’ll find your model so much easier to work with (and easier to explain to and share with others) if you use formatting to make it clear what different cells do. Here’s the key from my example model:

I’d love to hear your own experiences building models. What tips and tricks do you use all the time? How do you prepare forecasts and scenarios that stand up to client/boss scrutiny?