What’s the deeper worry around big data and creativity?

Data is wonderful at slicing options apart to discriminate the best choice within narrowly defined criteria. Creative thinking, on the other hand, isn’t about differentiating between options but about drawing together disparate notions that don’t initially seem to belong together. The more creative the idea, the more unexpected the connection. And the more innovative the idea, the less data there will be about whether people might respond well to it in the future. Hence the correlation between novelty and risk.

Many cite Netflix’s development of shows like House of Cards to show how data can ‘de-risk’ the creative process. They were able to safely invest millions upfront with the foreknowledge that the show would be a big hit -- the algorithms had indicated that it would be, partly because so many people had liked the original show. A similar strategy of course crops up around the idea of pre-testing ads and campaign concepts.

Such a ‘de-risked’ approach can be profitable of course. But if we choose our creative direction only by what users have (implicitly) told us they already like, our creative work is going to become increasingly self-referential, and circular. Simply by choosing which data sets to examine, we’ve already determined a particular direction of travel.

We might have reduced our risk, which is wonderful in a world of short term quarterly targets -- but we will never know what we might have missed out on. What Netflix did with House of Cards is clever, but it's at the shallow, incremental level of creativity. It’s variations on a familiar theme.

And it’s not just Netflix of course, the endless re-boots of superhero movie franchises and the mash-ups of digital culture, and even the repetitious nature of much pop music also treads the ever diminishing, data-led circuit.

It takes bravery to promote genuinely new stuff. The first few screenings of one the UK’s best loved TV shows ‘Only Fools and Horses’ for example, showed very little promise in terms of the initial data. Today, it might well have been axed after the first few episodes, or more likely, even before that point. It was only kept alive through a degree of stubborn creative bravery.

None of this is to say that data is irrelevant to creativity, it's just that it’s only one piece of stimulus - a highly useful one, but just one amongst many. When it comes to developing truly original, creative ideas, their very serendipitous nature means that we’ll never be able to find the answers we need in a database.

Perhaps one day we’ll build one big enough to do that. But not for aeons - as it would need to include pretty much every bit of data that exists - and then it would need to know how to find just the right unexpected, random connections between the data to generate something original and of value. But by that point, the map would have become the territory and we'll be streaming the 1000th season of our favourite box set from a remote hill station on Mars.

The increased use of data, however big, poses no logical challenge to creativity or innovation in general. However, it does raise a more worrying question, namely, when it comes to marketing - how genuinely creative or original does that really need to be?

Perhaps a lot of the navel gazing and angst around the advance of the algorithm stems from the fact that much successful marketing doesn't really need to be all that creative to still work pretty well. Of course, it must occasionally vary the tune to keep catching our attention but consistency not originality is more often the name of the game.

Perhaps the more unsettling existential question that the rise of big data prompts for marketeers is not “how can we continue being so creative” but rather, “how creative do we really need to be?”

Ewen Haldane is business director at The School of Life