Is big data too big?

I was quite shocked when I got my first job in advertising and all anybody talked about was data, algorithms, and metrics. There was a noticeable lack of whisky in meetings.

One thing is for sure: it’s an industry of extremes. As soon as a new technology appears, it becomes the be-all-end-all. ‘Brands who don’t have apps will die!’ ‘Social is the future!’ ‘AI will take all our jobs!’ ‘We will all live in the metaverse!’ So the hysteria goes.

The same thing happened with big data. The industry became obsessed. As a result, these days, pretty much anything you say is followed with: ‘where’s the data to back that up?’

The data deluge

Continuing advances in AI and big data have allowed us to capture and analyze huge amounts of information, but our ability (and inclination) to quantify everything is creating a problem. Simply put, big data is too big. We’re drowning in data. Marketers spend so much time being behind screens, monitoring numbers and making ‘data-driven’ decisions that they’re in real danger of forgetting who we’re creating, designing and building for: humans.

Even companies like Spotify and Netflix, which use big data in smart ways, can’t reveal the whole picture. Spotify’s algorithm can tell you that I flit between rap and classical music and never listen to techno. Netflix can tell you I binge murder documentaries on Sundays and never watch horror films. But they can’t tell you why.

Don’t get me wrong, data is important, and doing thorough research is crucial because it means we’re not designing with bias. Most agency people are friends with other agency people, who date agency people, who live with agency people. We’re not doing our jobs right if we assume that the behaviors we see directly around us apply to everyone in the world.

The world is bigger than (big) data

It’s about using data in the right way. It’s big data paired with ethnographic research and qualitative data (or ‘thick data’) that holds the most value. Thick data is usually what reveals the most unexpected and deeply human insights. Digging into subcultures and micro-communities, and speaking to people in their own environments, can provide you with truly insightful human truth.

You only need to look at Nike's ‘Nothing Beats A Londoner’, Dr Martens Presents or Always: Keep Going #LikeAGirl to understand how carrying out proper qualitative research can elevate campaigns and creative work. You can learn more from 100 thick data points than 10m quantitative ones.

Big data can find opportunity spaces (the metal detector); then, thick data can uncover insights or inspiration within these spaces (the gold). To paraphrase Steve Jobs, ‘market research can tell you what your customers think of something you show them, or what they want as an incremental improvement of what you have, but very rarely can your customers predict something that they don’t even know they want yet’.

We don't want to go back to the Mad Men days of pulling a campaign out of thin air because of a gut feeling, but there has to be a middle ground. We have to stop trying to tell human stories through numbers. It kills creativity and causes us to over-analyze brilliant and brave thinking.

Using only big data is a bit like turning off one of your senses. It can’t give you the full picture. We must take a holistic approach. Big data is a tool, but it’s not the tool. You need both to fill in the gaps and be able to connect the dots between what they think, feel, say and do. big data can tell you what’s happening. Human insight will tell you why.

For more on how the world of data-driven advertising and marketing is evolving, check out our latest Deep Dive.