5 reasons Binet and Field got it wrong on effectiveness

What do Sonic Youth, Carl Jung and the Bible have in common? They all think you should ‘kill your idols’ – that you shouldn’t be blinded by the shine of fame, that you need to keep your critical faculties intact when listening to ‘celebrities’

In the world of marketing effectiveness, there are two men idolised above all others – Les Binet and Peter Field.

Rightly so.

Their work stands as steady thinking amidst the storm of marketing commentary, and a corrective force against the autocracy of quarterly targets.

They are, in many ways, my heroes.

So, with a fanfare of distorted guitars, psychotherapy and spiritualism, here we go…

1. Effectiveness isn't about measurement, it's about improvement

First, and perhaps most importantly, the point of looking at effectiveness isn't to write books and academic papers. It isn’t even just to make the next three years as effective as possible. It's also to make the campaign you're working on right now as effective as possible. Test and learn is the order of the day, not set and forget. Practice, not just theory. That's true at a campaign level, and it's true for a medium-term plan. BiFi's famous golden ratios of short- and long-term spending are an impressive-sounding answer to a not-very-useful question: 'on average, and in general, how have people spent their marketing budget?' And speaking of averages...

2. The law of averages

Hussain's law of averages states that, on average, averages don't actually exist in the real world. Just ask the US Air force. FiBi's work reduces the complexities of IRL situations to a cornucopia of catchy graphs. These graphs advise the average company how to behave in an averaged market that's in an averaged context. Unfortunately, neither that company, nor that market, nor that context have ever existed. Short of an 'infinite-monkeys-meet-infinite-typewriters' approach to business (a very long game indeed), this renders the aforementioned graphs less useful than a single monkey using a single typewriter.

3. Small sample + many variables = unreliable results

LesTer, it seems, pick a relatively arbitrary sample from campaigns that have already been sold as 'very effective'. This is like judging the consistency of milk by sampling cream. More enjoyable, but not accurate. For more on why this is a problem, read this excellent article, in which I found the photo below. In summary, Dorothy is not a representative sample of the human population.

4. There aren't two types of marketing activity

Good vs Bad. Messi vs Ronaldo. Short vs Long. Sure, dualism is alluring (and is peculiarly enduring), and it can create nice tidy diagrams like the one below, but it leads, once again, to a wildly inaccurate abstraction of reality. There aren't two buckets for campaigns. There aren't sales activation campaigns. There aren't brand campaigns. (At least, not for anyone in the audience.) There are many campaigns, each unique in their approach and ambition. This doesn't make for tidy data though, so yep - I'm going with Messi being 'better' than Ronaldo. (For more on the problems of dualistic thinking, read this analysis from Tom Roach.)

5. Correlation does not equal causation

An oldie but a goodie. If correlation equalled causation, we might assume that an increase in margarine consumption is single-handedly responsible for divorces in Maine. No matter your objections to eating a substance so synthetic that even mould is scared of it, this isn't the case. Finet might be consummate graph-generators, but that doesn't make them irreproachable statisticians. Too often, their work shows correlation without showing a fundamental proof of cause and effect. For example, the famous graph that 'proves' Excess Share of Voice will grow Market Share seems to assume that the two have a linear relationship, where one thing happens (ESOV) and then another happens (increase in market share). In practice, they are two sides of a spinning wheel. The number of variables in a single marketing campaign -– let alone a growing business – is vast, and the interactions between them create a complex system. Graphs with a line or two going up and to the right will always be persuasive, but that doesn't make them reliable.

Oh, and by the way, in case I hadn’t already made it clear, Messrs Binet and Field are bloody brilliant and you should read everything they've ever written. Seriously. If you haven't already (why are you reading this article?), go do it. Just don't assume it's all perfectly correct. Don’t surrender your faculties to fame.

.