How do you solve a problem like big data?

So much data…

The word ‘big’ really doesn’t do it justice; something closer to ‘colossal data’ might be a little more appropriate. Although impossible to state with complete accuracy, IBM estimates there to be around 2.7 zettabytes of data in the digital universe at present, with five exabytes generated every 48 hours. These figures might be difficult to comprehend for anyone more used to dealing in megabytes, gigabytes and terabytes, so think of it like this instead: Facebook users alone upload enough data to fill 6,400 basic iPads every day.

What’s more, the vast majority of data in existence is unstructured; it’s information that, by Gartner’s definition, “doesn’t conform to a specific, pre-defined data model”. What this means is that it won’t fit neatly into your database and analysing it effectively is likely to be difficult.

Simply put, businesses are drowning in data; its volume is growing by the day and marketers are struggling to keep up.

…so little time

Knowing where to begin with all of this information is one thing; having the time to actually get to work on it is completely different. So much of the data mentioned above is useful to marketers, but sifting through to identify and collect the necessary parts is an extremely long-winded task; far from ideal in an industry where spare hours are a rarity.

Unfortunately, this tedious aggregation process is a necessity for most marketers, despite the availability of so many useful tools. According to a January 2015 Econsultancy report, just over half (51 per cent) of organisations are using more than 20 digital marketing technologies at present. With such a collection of data sources to tend to, though, it’s no surprise that so much valuable time is being wasted.

We don’t know everything

The already complex marketing tech landscape is expanding by the day; according to research carried out by chiefmartec.com founder Scott Brinker, there were 1,876 vendors in existence at the start of 2015 – up from just 947 at the same point last year. How can anyone be genuinely expected to keep up with this and still have the time to dedicate to their campaigns?

While many of these providers’ tools have overlapping functions, most marketers still have no idea just how many data sources are available to them - and not all know which ones they should be focusing their attention on either. As a result, there will always be puzzle pieces missing, and when it’s impossible to get the complete picture of a website’s performance, it’s impossible to identify and capitalise on every opportunity for improvement.

As the number of data sources continues to grow, and the marketing tech sector becomes somewhat saturated, there’s little to suggest this issue won’t grow in prominence.

Valuable resources are going to waste – including your money

There’s no doubting data is invaluable to the modern marketer, but the manual task of managing it, as suggested above, is far from efficient. It takes time and it takes effort – both of which come at an indirect financial cost.

The good news is that the average modern agency will be equipped to handle your data needs, with specialist teams and tools ready to harvest, process and act on the information most relevant to your digital performance. The bad news is that it will likely take them just as much time – and it’s time you’re paying for.

At present, a significant chunk of your marketing budget is probably being spent on aggregation when it should instead be spent on the skills, interpretations and actions of knowledgeable, trained specialists. Until the menial tasks associated with data use can be removed, this will always be the case.

Problem solved?

The issues above are currently preventing brands and marketers across the world from realising the true potential of data, but it won’t always be the case. Technology is evolving at some pace, and it’s taking the marketing industry with it.

With the revolution’s spotlight firmly on the value of insight and intelligence, the industry has been forced to concentrate its efforts on innovation. As a result, we’re starting to see tools designed specifically to make the data-dependent marketer’s life easier and their skills more powerful.

As it stands, there’s too much distance between marketing and technology. While there’s no doubting that the latter has had a huge impact on the former already, the two still exist at opposite ends of the spectrum. You have traditional service-centric digital agencies on one side, serving their clients with the help of platforms provided by vendors on the other.

The marketer of the future, however, sits comfortably at the midway point. Here, they combine the expertise they’ve always been known for with proprietary intelligence tools to make the best possible use of data and maximise campaign performance.

Graeme Parton is brand journalist for Vertical Leap