This promoted content is produced by a publishing partner of Open Mic. A paid-for membership product for partners of The Drum to self-publish their news, opinions and insights on thedrum.com - Find out more
From butterfly to kaleidoscope: A new era of data collaboration
March 29, 2023
It is tempting to reminisce about the clean lines of the DMP / CDP “butterfly”. This beautiful, flexible diagram outlined the "data in" and “data out” powering marketing for a company. The technical architecture was certain and centralized around an enterprise’s goals. Large silos connected at different tiers, identity was (partly) resolved across known and pseudo-spaces, and software interconnectivity was (mostly) clear. Existential debates were about header bidders, pixels vs. UTM parameters, and how to manage simultaneous programmatic and direct sales efforts. Above all, the butterfly said I am in control. Robert Moses, the influential New York urban planner, would have loved it.
DMP / CDP butterflies may have been elegant, but they had limits. The butterfly was self-oriented, top-down architecture of an organization’s own technical setup, with fixed ideas for how data flows across systems. Those systems were controlled by the company itself. Outside partnerships were present, but at the margins, and walled gardens threw a wrench into an otherwise elegant slide. (“Unfortunately, Facebook doesn’t let you....”) At its best, we can say that the Data 60s was a purposeful time driven by leaders with grand ideas for the future.
Welcome to the Data 70s
The Data 60s had their charm, but it is time to accept that we are in a new era. Welcome to the Data 70s. Consumers and their elected representatives will no longer blindly trust, if they ever did. Major brands embrace those trends and encourage consumers to express their distrust at scale. Companies themselves want always-on guardrails when conducting essential collaborations. And yet, the tools built to navigate this new era of deliberate choice have opened up fresh opportunities for data collaboration in a way never before seen.
Even at its height, the Data 60s never accessed datasets deemed too sensitive to risk outside of a company, or even across a company’s own internal business lines. There was no way to access exactly what you wanted for only the precise operations and computes needed. You shared or you did not share. There was no middle ground. But new tools like data clean rooms and clean computes have, out of necessity, allowed all parties to tailor their data collaboration to the absolute minimum required. The ability for firms to decide on their own terms has allowed net new collaborations both within and beyond marketing use cases. We can state confidently from client innovators that this evolution in the end has benefited all parties.
Embracing this complexity means accepting a lack of complete control over some of the foundations owned by others. Success stems from interoperability and flexibility. Imagine your business partner runs on a different cloud platform than your own, and a third party (on yet another cloud / region pair) has transaction data that can inform new directions for your partnership. Collaboration requires a distributed compute across 3 parties, one on AWS, a second on Azure, a third on GCP. A fourth potential party (on Snowflake) could add additional depth when the partnership deepens. When one of your partners merges with a company with a Databricks posture, you can add a fifth connection to leverage their new data sets.
Tennis on top of Grand Central
You might imagine a group meeting for a doubles match at Grand Central. One takes Metro North, another tries out the brand new direct LIRR East Side access, a third arrives on the 7 (former IRT), and the fourth finds her way on the M (former BMT). Each brought something special to the court, and anyone’s absence would have diminished the experience. Each came to the partnership on their own terms, and when they were done, they headed home. The match they played together was ephemeral. Only the final score remains. They each gave up some control and relied on others to deliver a shared experience. And each is absolutely better off for making that choice.
It was not one butterfly, but four butterflies coming together on their own terms. A group of butterflies is called a kaleidoscope. What better metaphor for the Data 70s?
Habu’s clients and partners are focused on outcomes, and getting the insights they need to further improve those outcomes. Access to all the data was never the point. Control of everything was an artifact of an older ecosystem, tools, and mindset. Now we can do better. Talk with Habu about what happens when you are freed from rigid architecture, and focus instead on joining interoperable partnerships. It’s the real thing.