Unlock the value of your Data Lake by connecting it to a Data Marketing Infrastructure
Every digital marketer knows that effective campaigns are built and run on accurate, real-time, actionable data and insights
Separately, but at the same time, data departments have been investing in data lakes (a centralised repository that stores all structured and unstructured data) and data scientists who develop machine learning solutions and algorithms within a data lake to predict future outcomes. Recently, it has become obvious that data is the key to both business and marketing success, with companies that have invested heavily in data having the highest stock exchange valuations. With data becoming hugely fashionable large businesses have created data departments, headed by a Chief Data Officer, that were the drivers of the creation of data lakes.
Where this strategy fell short is that marketing teams were already implementing personalisation techniques with existing tools, such as CRM platforms. When the two departments came together it became obvious that because the data flowed into the data lake in real-time the data science teams could build better models for optimising margins and CTR using the data in the data lake. The issue was that there were no pipes to connect the data lake to marketing activation tools. Essentially, it’s like having an amazing idea but no words to express it.
The irony is that by operating these systems in silos, brands and retailers are collecting more data than ever before but effectively doing very little with it because the systems are not integrated.
The curse of fragmentation
In computing terms, fragmentation causes a computer to use excessive resources (memory and CPU time) to complete tasks related to reading and writing files. This unnecessarily increases the work your computer must do to support the applications you are running. The exact same concept applies to data marketing, except the issue is not limited to the actual technology — it also significantly increases the amount of human resources and time needed to make sense of all the data and apply learnings to marketing campaigns.
Having a fragmented data marketing stack and isolated data lake means that essentially the advanced insights that have been gleaned from AI and ML within the data lake cannot be ingested into a CDP, DMP, or any other marketing activation tools. Essentially, it’s like having a fully functioning brain (i.e. a data lake with memory and processing capabilities) that has no connections to the muscles (i.e. marketing activation tools) of the body. Without these connections your muscles would move in random, unsynchronised ways leaving you unable to move in a coordinated way.
The challenge we face is how to create an open infrastructure that enables the DMP, CDP, and data lake to function as one working towards common goals and objectives.
The catalyst for change
Today, the digital marketing industry is in the midst of a fundamental change — the demise of the third-party cookie. This presents huge challenges for DMPs as this is the currency on which they operate. At the same time, there is an industry-wide concern over the disproportionate amount of ad spend that flows through the walled gardens. What we need are new ways of not just tracking but understanding consumer behaviour and applying that understanding to marketing activation. This will enable marketers to move away from reliance on Google’s intent data, Facebook’s look-a-like modelling, and Amazon’s purchase intent data and hone in on their first-party data to drive performance. The challenge is that in order to take control and leverage their own data effectively, brands need to build their own technology stacks.
Up to now brands that are able to offer a meaningful value exchange, e.g. retailers, have been pushing users to log in to their websites and apps and then store and activate that data using a CDP. Brands that are unable to build one-to-one relationships with consumers, e.g. FMCG, have used the small amount of first-party data they have layered on top of second-party data and built assumptive look-a-like models for audience targeting.
In programmatic terms, these audiences are small but they can be used as a seed audience to track user behaviour and profiles. Machine learning (ML) algorithms can then be used to build look-a-like models and define the best channels, geos, and audiences and tie those to users on the open web. When these seed audiences are also sent to walled gardens you overcome the issue of achieving scale in the absence of cookies.
Artificial intelligence (AI) and machine learning (ML) functions have the potential to transform current ways of working with marketing data without the limitations of ‘off-the-shelf’ programming which is just not physically able to take into consideration the enormous number of variables that influence consumer behaviour and marketing decisioning. When AI and ML operate in a data lake the algorithms created are based on all available data not just segments of data. In addition, because a data lake is built and controlled by data scientists, human insight and reasoning can be overlaid making the algorithms even better.
How one of the top 10 global retailers (food and non-food) created a centralised view of data with mediarithmics
A retail holding group comprising 25 retailers, with more than €60B annual revenue, built a data lake containing ‘cold’ data from the groups 100 million loyalty cards enabling them to gather valuable insights into consumer behaviour across all brands. This gave them a cross-brand single customer view but on its own the insights they had could not be applied to marketing.
The challenge for this retail group was to find a platform that could ingest all the data from their data lake with no compression or aggregation while simultaneously keeping each brands’ data separate with zero data leakage. mediarithmics developed a solution that gave them a platform with separate ‘compartments’, with unique IDs, for each brand’s CRM and a client ID that is common across all brands for each customer. This enabled the retail group to maintain a centralised view of the cross-brand data and control the distribution of data back out to each brand.
mediarithmics is then able to collect data from marketing campaigns, websites, and apps and uses a common matching key (the client ID, CRM ID, and hashed email addresses) and reconciles them within the data lake thus creating a 360 degree, cross-brand customer view. By creating this feedback loop between the data lake and the CDP the retail group’s data science team are able to refine the algorithms that calculate audience scores and export those scores to each brand for real-time optimisation.
Transforming advertising for better, next year and beyond
There is no doubt that the digital advertising industry is on the cusp of a new era. As we collectively navigate the changes coming over the next months and years it is imperative that we hone in on individual drivers of success using real-life use cases and move away from the culture of investing in technology because it’s there.
When evaluating technology partners, especially those underpinned by AI and ML, brands need to have a clear line of sight over what they want to achieve and dig deep into the infrastructure and operations of any platform that is going to touch their data. Unifying all data technology components enables brands to deliver all of their data marketing needs from collection, segmentation, activation, reporting, and insights.
Technology companies need to step up to the plate and offer solutions that are truly customisable and build platforms that can operate at scale, collaboratively, in a compliant way. This is the key to eliminating the spray and pray campaigns that damage user experiences and transitioning to a world where all marketing campaigns and decisioning is underpinned by robust data and builds relevant one-to-one relationships throughout the purchase funnel and creates amazing consumer experiences.
If you would like to discuss how to unlock the power of your data lake further, please visit www.mediarithmics.com.